Thank you for sharing this! I haven't used Arrow tooo much but I have been running into lots of performance issues with CSVs and Pandas which is leading me into experimenting with Arrow / Polars. Never hurts to read up on the documentation & I assume I'll be using Arrow a lot more in the future.
this post was submitted on 05 Jul 2023
2 points (100.0% liked)
Data Engineering
185 readers
1 users here now
Discussion on Data Engineering topics. Data pipelines, tools and technologies, databases and DBMS, best practices:
Rules:
- Limited to data engineering, no general CS/programming posts.
- No technical questions. Example: how to fix this bug in my code.
- No marketing
- No resumes, jobs
- No PII
founded 1 year ago
MODERATORS