[feat] remove numpy dependencies in the core code path

[animeshtrivedi]

Remove numpy usage in the core I/O paths, which breaks pypy JIT'ing logic.