Your site takes nearly a second just to load the HTML http://www.webpagetest.org/result/181014_9A_744f6e41c62ea5be0a2e3d0dfff51def/1/details/#waterfall_view_step1 steps 1 &2 in the test above, so comparing your site performance to other sites doesn’t address the fundamental performance issues your site has that the others don’t.
To completion of the SSL handshake to your origin the timing looks like this:
DNS Lookup: 24 ms
Initial Connection: 105 ms
SSL Negotiation: 241 ms
The same handshake for the CDN1 assets through Cloudflare is:
DNS Lookup: 64 ms
Initial Connection: 30 ms
SSL Negotiation: 66 ms
So the SSL handshake is > 50% faster if your site was behind Cloudflare. Plenty of other room for optimization in your theme… upwork and fiver probably aren’t good peer comparisons as their site design is completely different from the one you’re testing.