"Build comments on pull requests" not working

The option “Automatically add build comments on open pull requests” is enabled on my page’s build configuration but I don’t see any comment added to my PR’s. The Cloudflare GitHub App is installed in an Organization account, in case that matters. I don’t see much about this feature in help docs.

Any ideas?

I am also no longer getting comments on my open PRs.

Can now also confirm that the comment only gets added after additional commits are made, as mentioned here: PR comment only made if additional commits pushed