As the NS record delegates authority over subdomain.example.com, I can see how the behavior might be reasonable. I don’t have any RFC or similar handy, but my gut feeling is that a strict implementation would respect the delegation, which should cover anything under/below that domain.
You can probably get around this if you don’t need to delegate at multiple levels. Like instead of…
example.com → subdomain.example.com → test.subdomain.example.com
… you could do…
example.com → test.subdomain.example.com
Essentially two Route 53 hosted zones instead of three. It can scale to a large number of delegations straight from the example.com Route 53 hosted zone as long as you can avoid overlap (as in “subdomain” NS record kind of hiding “test.subdomain”).
The feasibility depends entirely on your record structure. I’m thinking something like this, but your use case might be different.
example.com hosted zone:
prod.us.example.com NS <awsdns…>
prod.eu.example.com NS <awsdns…>
prod.us.example.com hosted zone:
mysql.prod.us.example.com A 127.0.0.1
microservice1.prod.us.example.com A 127.0.0.1