Once the plan shows you the expensive step, write down what you think is wrong and what you expect the fix to change, then apply exactly one change: an index, a rewrite, or fresh statistics. Re-run EXPLAIN ANALYZE and check two things, that the plan node you were aiming at actually changed, and that the time went down; a faster run with the same plan is usually just a warm cache. Changing three things at once leaves you with a faster query and no idea which change to keep, and you will be carrying the other two forever.
Article: PostgreSQL EXPLAIN ANALYZE: Read Plans & Fix Slow Queries