The first run of a query pulls pages from disk into the database's buffer pool and the operating system's cache; the second run finds them already there and can be ten times faster for no other reason. So timing a query once tells you almost nothing, and timing it twice tells you about your cache, not your query. The honest approach is to run it several times, report both the cold and the warm figure, and compare buffer counts rather than milliseconds when you need a measure that does not move.
Article: PostgreSQL Cold Cache: pg_prewarm and Restart Warm-Up