Petastorm not working due to PyArrow version hell #806
Description
I was trying to build something from petastorm dataset minimal example.
When I tried to launch my code, I got error:
from pyarrow.filesystem import LocalFileSystem
ModuleNotFoundError: No module named 'pyarrow.filesystem'
Because apparently it's called pyarrow.fs now?
After downgrading my pyarrow version to 13.0, then 12.0 as suggested here:
Traceback (most recent call last):
File "/.myenv/lib/python3.8/site-packages/petastorm/utils.py", line 21, in <module>
import pyarrow
File "/.myenv/lib/python3.8/site-packages/pyarrow/__init__.py", line 65, in <module>
import pyarrow.lib as _lib
File "pyarrow/compat.pxi", line 43, in init pyarrow.lib
File "/.myenv/lib/python3.8/site-packages/cloudpickle/__init__.py", line 1, in <module>
from . import cloudpickle
File "/.myenv/lib/python3.8/site-packages/cloudpickle/cloudpickle.py", line 73, in <module>
import uuid
File "/usr/lib/python3.8/uuid.py", line 60, in <module>
_platform_system = platform.system()
File "/usr/lib/python3.8/platform.py", line 892, in system
return uname().system
File "/usr/lib/python3.8/platform.py", line 858, in uname
processor = _syscmd_uname('-p', '')
File "/usr/lib/python3.8/platform.py", line 612, in _syscmd_uname
import subprocess
File "/usr/lib/python3.8/subprocess.py", line 218, in <module>
_PopenSelector = selectors.SelectSelector
AttributeError: module 'selectors' has no attribute 'SelectSelector'
What version of pyarrow is it supposed to use??