It seems from the test code that every time that Profile.main() gets called, once it gets to SentenceExtraction_test, it reloads the MITIE/Stanford model, which is obviously very, very slow. Am I reading that right? Is there either a way to pass a list of stories into Profile.main()? If not, it seems like it would make sense to load the NER once inside Profile.main() and pass it into SentenceExtraction_test with each sentence (same as the embedding model). Does that make sense? Am I missing an option?
It seems from the test code that every time that
Profile.main()gets called, once it gets toSentenceExtraction_test, it reloads the MITIE/Stanford model, which is obviously very, very slow. Am I reading that right? Is there either a way to pass a list of stories intoProfile.main()? If not, it seems like it would make sense to load the NER once insideProfile.main()and pass it intoSentenceExtraction_testwith each sentence (same as the embedding model). Does that make sense? Am I missing an option?