Scaffolding for contributing UDFs as extensions

[jaychia]

We would like to propose and create scaffolding for extending the Daft engine, starting with new UDFs/operators in a manageable repository structure.

[conceptofmind]

Hi all,

I think it would be interesting to be able to add support for different adapters that interface with api libraries.

For example, something such as GLM OCR has api support for text extraction: https://github.com/zai-org/GLM-OCR

Another integration that I would like to personally add is our own APIs for things such as classification and text segmentation.

For example, I currently have a free text segmentation api that can be easily integrated into a daft UDF:

client = TeraflopAI(api_key=api_key, url=url)

text = """
UNITED STATES of America, Appellee, v. Daniel Dee VEON, Appellant.
No. 72-1889.
United States Court of Appeals, Ninth Circuit.
Feb. 12, 1973.

Claude 0. Allen, Oakland, Cal., for appellant.
James L. Browning, Jr., U. S. Atty., F. Steele Langford and Jerry K. Cimmet, Asst. U. S. Attys., San Francisco, Cal., for appellee.
Before TRASK, GOODWIN and WALLACE, Circuit Judges.
PER CURIAM:

Daniel Dee Veon appeals from convictions of violations of 21 U.S.C. § 841 (a) (1) (possession of marijuana with intent to distribute); two counts of 21 U.S.C. § 843(b) (use of telephone to commit or to facilitate commission of felonies under Title 21); 21 U.S.C. § 846 (conspiracy to distribute and to possess with intent to distribute marijuana).

Defendant contends the district court erred: (1) in restricting the reeross-examination of a government witness; (2) in receiving into evidence a tape recording and a written transcript of the tape; and (3) in submitting the case to the jury, upon evidence which he asserts was insufficient to support the verdict.

Customs agents observed a load of 400 kilograms of marijuana being brought in from Mexico. The agents arrested the smugglers and had them make a “controlled delivery” to Donald Lynch, their employer. Lynch was arrested and agreed to cooperate with police. With police listening and taping the call, Lynch phoned Veon and told him that the “stuff” was in for “Gary.” Veon said, “Yeah, okay, uh?” Lynch replied, “Same old thing.” Veon signified assent. This call is the basis for the second count of the indictment.
"""

results = client.segment(text)

print(results["results"])

This returns a list of strings that can just be exploded.

It currently works well with daft batch UDFs, and I used it for the US caselaw dataset.

I know in the main library, there are adapters for LLM and embedding-based APIs.

So I was wondering what the best way would be to add this.

I would also like to add support for the jotunn codebase once I finish updating it to the new batch udfs.

Thank you,

Enrico

[rchowell]

Hey @conceptofmind,

Is there something you would like to do today which is difficult with our current extension points?

• DataSource: Interface for reading data into DataFrames
• DataSink: Interface for writing data from DataFrames
• Table: Combines a source & sink, resolvable via Catalogs
• Catalog: Table resolution interface for SQL
• Scalar UDF: value in -> value out functions (sync + async support)
• Batch UDF: batch in -> batch out functions (sync + async support)
• Generator UDF: Uses yield to produce multiple output rows

[conceptofmind]

Hey @conceptofmind,

Is there something you would like to do today which is difficult with our current extension points?

• DataSource: Interface for reading data into DataFrames
• DataSink: Interface for writing data from DataFrames
• Table: Combines a source & sink, resolvable via Catalogs
• Catalog: Table resolution interface for SQL
• Scalar UDF: value in -> value out functions (sync + async support)
• Batch UDF: batch in -> batch out functions (sync + async support)
• Generator UDF: Uses yield to produce multiple output rows

Currently, these cover what is needed when using our own UDFs.

I am talking more about adding official support for external APIs through adapters such as openai, glm, etc.

[rchowell]

Could you please elaborate more on what "official support for external APIs" means? If it's official support, then perhaps it is worth contributing upstream to Daft. For something that you wish to plugin, we have those extension points and can make more.

I would need to know more specifically what you need so that I can help out here.

[conceptofmind]

Could you please elaborate more on what "official support for external APIs" means? If it's official support, then perhaps it is worth contributing upstream to Daft. For something that you wish to plugin, we have those extension points and can make more.

I would need to know more specifically what you need so that I can help out here.

For OpenAI, there is a from daft.ai.openai.provider import OpenAIProvider. My thoughts here were that it could make sense to add additional adapters for these providers to include support for more external apis. I was not sure whether it would make sense to upstream these directly to Daft or have a separate repository where a user could select additional ones.

[rchowell]

I mistakenly forgot to include this. The provider system is designed to be extensible and you can add additional providers or implementations if you would like.

Do you have any examples of what you mean by additional adapters or more external APIs? That would help determine how this should fit in.

While the providers are designed to be extensible, it is intentionally over a subset of provider-like APIs such as embeddings and chat completions. Similar to how the Catalogs are designed to be extended, but for catalog-like capabilities such as creating schemas and resolving tables.

[conceptofmind]

I mistakenly forgot to include this. The provider system is designed to be extensible and you can add additional providers or implementations if you would like.

Do you have any examples of what you mean by additional adapters or more external APIs? That would help determine how this should fit in.

While the providers are designed to be extensible, it is intentionally over a subset of provider-like APIs such as embeddings and chat completions. Similar to how the Catalogs are designed to be extended, but for catalog-like capabilities such as creating schemas and resolving tables.

An example of an additional adapter would be the glm-ocr api found here: https://docs.bigmodel.cn/cn/guide/models/vlm/glm-ocr

https://github.com/zai-org/z-ai-sdk-python/tree/main

Maybe this would fit into the catalog category then?

[rchowell]

I am having a hard time understanding what is meant by "adapter" here. That glm-ocr API seems like a one-off and unique to something BigModel, but it's hard for me to tell since it is in Chinese which I do not read or speak.

If this is a standardized API like the OpenAI API specification, then it makes sense to define an interface within Daft. You can also use a class-based UDF (and async) for using a stateful client along with the functionality you need.

Catalogs are the traditional SQL abstraction for managing schemas and tables. I don't believe this OCR function belongs in the catalog interface.

[conceptofmind]

I am having a hard time understanding what is meant by "adapter" here. That glm-ocr API seems like a one-off and unique to something BigModel, but it's hard for me to tell since it is in Chinese which I do not read or speak.

If this is a standardized API like the OpenAI API specification, then it makes sense to define an interface within Daft. You can also use a class-based UDF (and async) for using a stateful client along with the functionality you need.

Catalogs are the traditional SQL abstraction for managing schemas and tables. I don't believe this OCR function belongs in the catalog interface.

Currently, there is support for protocols such as Google and OpenAI apis: https://github.com/Eventual-Inc/Daft/tree/main/daft/ai

I am trying to determine whether it would make sense to add an upstream PR to interface with Daft for something like the zai api there.

Or rather this should be entirely up to user to implement themselves instead.

[rchowell]

Yes that does make sense; you can implement a "zai" provider (ZaiProvider) following: https://docs.daft.ai/en/stable/contributing/contributing-ai-functions/

If there are additional capabilities you need or functionality missing, then we can see how that might fit into the provider interfaces. However, for bespoke functions then class-based async UDFs are best.

[conceptofmind]

Yes that does make sense; you can implement a "zai" provider (ZaiProvider) following: https://docs.daft.ai/en/stable/contributing/contributing-ai-functions/

If there are additional capabilities you need or functionality missing, then we can see how that might fit into the provider interfaces. However, for bespoke functions then class-based async UDFs are best.

Ok. Thank you for clarifying that it should be integrated directly into Daft!

[rchowell]

that it should be integrated directly into Daft

I don't believe this was the conclusion, what do you mean by "it" here? Sorry, all I am trying to say is that you can implement your own provider outside of daft for zai. The provider interface was intentionally designed for this.

[conceptofmind]

that it should be integrated directly into Daft

I don't believe this was the conclusion, what do you mean by "it" here? Sorry, all I am trying to say is that you can implement your own provider outside of daft for zai. The provider interface was intentionally designed for this.

Yes, misunderstanding there. I meant using the provider interface.

If we would want to contribute these provider interfaces to the community what would be the best way to do so?