A little from column A and a little from column B. If you have the ‘All’ option selected in the rule then it should pick up new entries as Cloudflare adds them. If you have deselected the ‘all’ because you wish to be selective about what you add, net new entries wouldn’t be added automatically.
Also you can provide feedback in the app for apps you find and you can create a DNI rule for the hosts associated with it. Cloudflare may choose to add it to the list it maintains, but there are a lot of apps out there and not necessarily every app that uses Cert pining is one where either Cloudflare has enough data or it may be an app that the average company might not want to allow (e.g. if you find a
questionable content app that used cert pinning it is somewhat unlikely we’d add it to the list).