R2 breaks pyarrow (API incompatibility)

Developers Storage
1

Seems like R2 does not work well with pyarrow.
See pyarrow JIRA ticket: https://issues.apache.org/jira/browse/ARROW-18076

2

For context:

  • pyarrow is a library for reading/writing parquet files, but it also handles providing a S3 file system API for Python

This pyarrow file system API works fine for files stored in S3 but chokes on files stored in R2.