Skip to content

Commit 58e18bd

Browse files
codetalker-aicodetalker-ai
authored
Merge branch 'TobikoData:main' into main
2 parents 33f133f + d4dfe0f commit 58e18bd

123 files changed

Lines changed: 980 additions & 234 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎docs/integrations/engines/clickhouse.md‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -421,4 +421,29 @@ If a model has many records in each partition, you may see additional performanc
421421
Choose a model's time partitioning granularity based on the characteristics of the data it will process, making sure the total number of partitions is 1000 or fewer.
422422

423423
## Local/Built-in Scheduler
424+
424425
**Engine Adapter Type**: `clickhouse`
426+
427+
| Option | Description | Type | Required |
428+
| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :----: | :------: |
429+
| `type` | Engine type name - must be `clickhouse` | string | Y |
430+
| `host` | ClickHouse server hostname or IP address | string | Y |
431+
| `username` | ClickHouse user name | string | Y |
432+
| `password` | ClickHouse user password | string | N |
433+
| `port` | The ClickHouse HTTP or HTTPS port (Default: `8123`) | int | N |
434+
| `cluster` | ClickHouse cluster name | string | N |
435+
| `connect_timeout` | Connection timeout in seconds (Default: `10`) | int | N |
436+
| `send_receive_timeout` | Send/receive timeout in seconds (Default: `300`) | int | N |
437+
| `query_limit` | Query result limit (Default: `0` - no limit) | int | N |
438+
| `use_compression` | Whether to use compression (Default: `True`) | bool | N |
439+
| `compression_method` | Compression method to use | string | N |
440+
| `http_proxy` | HTTP proxy address (equivalent to setting the HTTP_PROXY environment variable) | string | N |
441+
| `verify` | Verify server TLS/SSL certificate (Default: `True`) | bool | N |
442+
| `ca_cert` | Ignored if verify is `False`. If verify is `True`, the file path to Certificate Authority root to validate ClickHouse server certificate, in .pem format. Not necessary if the ClickHouse server certificate is a globally trusted root as verified by the operating system. | string | N |
443+
| `client_cert` | File path to a TLS Client certificate in .pem format (for mutual TLS authentication). The file should contain a full certificate chain, including any intermediate certificates. | string | N |
444+
| `client_cert_key` | File path to the private key for the Client Certificate. Required if the private key is not included the Client Certificate key file. | string | N |
445+
| `https_proxy` | HTTPS proxy address (equivalent to setting the HTTPS_PROXY environment variable) | string | N |
446+
| `server_host_name` | The ClickHouse server hostname as identified by the CN or SNI of its TLS certificate. Set this to avoid SSL errors when connecting through a proxy or tunnel with a different hostname. | string | N |
447+
| `tls_mode` | Controls advanced TLS behavior. proxy and strict do not invoke ClickHouse mutual TLS connection, but do send client cert and key. mutual assumes ClickHouse mutual TLS auth with a client certificate. | string | N |
448+
| `connection_settings` | Additional [connection settings](https://clickhouse.com/docs/integrations/python#settings-argument) | dict | N |
449+
| `connection_pool_options` | Additional [options](https://clickhouse.com/docs/integrations/python#customizing-the-http-connection-pool) for the HTTP connection pool | dict | N |

‎examples/ibis/models/ibis_full_model_python.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
from datetime import datetime
33

44
import ibis # type: ignore
5-
import pandas as pd
5+
import pandas as pd # noqa: TID253
66
from constants import DB_PATH # type: ignore
77
from sqlglot import exp
88

‎examples/sushi/models/items.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
from datetime import datetime
44

55
import numpy as np
6-
import pandas as pd
6+
import pandas as pd # noqa: TID253
77
from helper import iter_dates # type: ignore
88
from sqlglot.expressions import to_column
99

‎examples/sushi/models/order_items.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
from datetime import datetime
44

55
import numpy as np
6-
import pandas as pd
6+
import pandas as pd # noqa: TID253
77
from helper import iter_dates # type: ignore
88
from sqlglot import exp
99
from sqlglot.expressions import to_column

‎examples/sushi/models/orders.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
import typing as t
33
from datetime import datetime, timedelta
44

5-
import pandas as pd
5+
import pandas as pd # noqa: TID253
66
from helper import iter_dates # type: ignore
77

88
from sqlmesh import ExecutionContext, model

‎examples/sushi/models/raw_marketing.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
from datetime import datetime
44

55
import numpy as np
6-
import pandas as pd
6+
import pandas as pd # noqa: TID253
77
from sqlglot import exp
88

99
from sqlmesh import ExecutionContext, model

‎examples/wursthall/models/db/order_f.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
from datetime import datetime
44

55
import numpy as np
6-
import pandas as pd
6+
import pandas as pd # noqa: TID253
77
from models.src.shared import DATA_START_DATE_STR, set_seed # type: ignore
88
from sqlglot import parse_one
99

‎examples/wursthall/models/src/customer_details.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
from dataclasses import dataclass
44
from datetime import datetime, timedelta
55

6-
import pandas as pd
6+
import pandas as pd # noqa: TID253
77
from faker import Faker
88
from models.src.shared import DATA_START_DATE_STR, iter_dates, set_seed # type: ignore
99

‎examples/wursthall/models/src/order_item_details.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
from datetime import datetime
55

66
import numpy as np
7-
import pandas as pd
7+
import pandas as pd # noqa: TID253
88
from faker import Faker
99
from models.src.shared import DATA_START_DATE_STR, iter_dates, set_seed # type: ignore
1010
from sqlglot import parse_one

‎pyproject.toml‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,7 @@ dev = [
7676
"psycopg2-binary",
7777
"pydantic",
7878
"PyAthena[Pandas]",
79-
"PyGithub~=2.5.0",
79+
"PyGithub",
8080
"pyperf",
8181
"pyspark~=3.5.0",
8282
"pytest",
@@ -271,3 +271,9 @@ select = [
271271
"F401",
272272
"RET505",
273273
]
274+
extend-select = ["TID"]
275+
276+
[tool.ruff.lint.flake8-tidy-imports]
277+
banned-module-level-imports = [
278+
"pandas",
279+
]

0 commit comments

Comments
 (0)