In real world, AMP pages won’t always be served via Google CDN AMP cache as there are other platforms which can now auto detect AMP pages and serve them based on detected amphtml referenced AMP pages besides Google search. For example, Twitter on mobile Overview | Twitter Developer.
Twitter supports the standard AMP discovery mechanism. When a link is shared on Twitter, the Twitter crawler (user agent: Twitterbot/1.0) will look for a element in your document. If a valid AMP document is referenced, Twitter mobile clients will direct users to that AMP edition of your page.
So there are possibilities when AMP HTML pages aren’t cached outside of Google AMP cache.
Yeah using /amp instead of ?amp might help