Skip to content

Commit eaf4629

Browse files
Baowen648tuhaihe
Baowen648
authored and
tuhaihe
committed
Doc: update the deployment README.md
We have updated the old documents and improved the formatting to make them more user-friendly. Users can deploy the Cloudberry Database by following these steps.
1 parent 188cc1d commit eaf4629

1 file changed

Lines changed: 143 additions & 63 deletions

File tree

‎deploy/README.md‎

Lines changed: 143 additions & 63 deletions
Original file line numberDiff line numberDiff line change
@@ -1,84 +1,164 @@
1-
### Cloudberry Automation Build and Deployment
1+
# Auto-Build Cloudberry Database from Source Code
22

3-
#### Download the Cloudberry project source code into your working directory
4-
git clone git@github.com:cloudberrydb/cloudberrydb.git
3+
You can build Cloudberry Database from source code in two ways: manually or automatically.
54

6-
#### Enter the directory under workspace/cbdb, developer can deploy the automation build & test environment with vagrant virtual machine or docker container, vagrant and docker tool is required to be installed in your local environment before running the script.
5+
For the manual build, you need to manually set up many system configurations and download third-party dependencies, which is quite cumbersome and error-prone.
76

8-
##### 1.Deploy the automation build & test environment with vagrant virtual machine
7+
To make the job easier, you are recommended to use the automated deployment method blessed by virtualization technology. The automation method simplifies the deployment process, reduces time costs, and allows developers to focus more on business code development.
98

10-
##### Start vagrant virtual machine by following script
11-
cd ~/workspace/cbdb/deploy/vagrant
12-
vagrant up
9+
To go automated, you can choose either of the following ways based on your needs and project requirements:
1310

14-
##### After virtual machine vbox being launched, compiled and deployed automatically by "vagrant up" script, then log into vagrant virtual machine via ssh
15-
vagrant ssh
16-
psql postgres
11+
- [Vagrant deployment](#deploy-with-vagrant): Vagrant allows you to easily create and manage vbox virtual machine environments, streamlining the deployment process.
12+
- [Docker deployment](#deploy-with-docker): Docker provides a containerized environment, offering higher levels of isolation and ease of management for building and testing.
1713

18-
##### start regression test
19-
cd ~/workspace/cbdb_dev
20-
make installcheck-good
14+
This document introduces how to use Vagrant and Docker to auto-build Cloudberry Database from source code, respectively. In addition, it also introduces the file sharing mechanism that ensures that the Cloudberry Database is mapped to a specific path in the virtualized environment.
2115

16+
## Ready for Work
2217

23-
##### 2.Deploy the automation build & test environment with docker container
18+
Before you begin your automation build, make sure the following preparation works are done:
2419

25-
##### docker image repository configuration
26-
Configure the docker image repository URL by adding the following JSON format configuration in "Preferences->Docker Engine" tab in Docker tool.
27-
"insecure-registries": [
28-
"docker.artifactory.hashdata.xyz"
29-
]
20+
1. Download the Cloudberry Database source code into your local machine:
3021

31-
##### Start the docker container by following script
32-
cd ~/workspace/cbdb/deploy/docker
33-
bash docker_start.sh
22+
```bash
23+
git clone git@github.com:cloudberrydb/cloudberrydb.git
24+
```
3425

35-
##### After the docker container being launched, compiled and deployed automatically by docker_start.sh script, then log into the docker container. The ContainerId could be retrived by using "docker ps -a" command
36-
docker exec -it [ContainerId] /bin/bash
37-
psql postgres
26+
2. Install the required [Vagrant](https://developer.hashicorp.com/vagrant/docs/installation) or [Docker](https://docs.docker.com/desktop/install/mac-install/) in your local environment, depending on your deployment preference.
3827

39-
##### start regression test
40-
cd ~/workspace/cbdb_dev
41-
make installcheck-good
4228

43-
<br>
29+
## Deploy with Vagrant
4430

31+
1. Start your Vagrant virtual machine.
32+
2. Start the automatic compilation and deployment process.
4533

46-
### Script Introduction
34+
```
35+
$ cd /cloudberrydb/deploy/vagrant
36+
$ vagrant up
37+
```
4738

48-
The build & test environment of cbdb project relies on many systemconfigurations and third-party dependencies, therefore the manual deployment process is quite cumbersome and error-prone. For this reason, the automation build & test environment of cbdb is introduced by profitting the virtualization techologies such as vagrant or docker, which can be used to simplify the user's deployment efforts and time-costs to deploy the cbdb build & test environment. As a result the cbdb developers could be more concentrated on their business-level code development work. The cbdb project supports two automation deployment methods: the first one is to deploy vbox virtual machine with vagrant, and another one is to deploy container image with docker. Developers can choose the way of automation deployment depends on their own requirements. The entire automation build & test process is running on the virtualization environment managed by vagrant or docker. The cbdb code repository of the host machine is mapped into specified path in the virtualization environment through the sharing file mechanism for cbdb automation build & test.
39+
3. Log into your Vagrant virtual machine via `ssh`:
4940

50-
#### The whole script is subdivided into three main parts:
41+
```
42+
$ vagrant ssh
43+
```
5144

52-
##### 1.The automation deployment script running in the virtualization environment.
53-
##### Through the mechanism provided by vagrant vagrantfile provision or docker dockerfile ENTRYPOINTS
54-
* Provides subcommands that operates the build & test environment
55-
* Provides the necessary configuration variables to the internal script of the virtual machine and trigger specific operations
56-
##### Passing the following compilation options through /code/depoly/cbdb_ deploy.sh script
57-
* Compilation options
58-
* Deployment options
59-
* Test options
60-
##### Passing the environment variable parameters required for automation build & test through /code/depoly/cbdb_env.sh script.
45+
4. Connect to Cloudberry Database, and check whether the psql client can be started and connect to the database.
6146

62-
#### 2.Vagrant startup script
63-
##### Vagrantfile
64-
* Downloads the Vagrant virtual machine box and local virtual box creation.
65-
* The host code repository path is mounted on virtual machine with read-only permission, for example: source code path to /home/gpadmin/workspace/cbdb directory
66-
* Copy the source file directory to /home/gpadmin/workspace/cbdb_dev, which is the actual working directory for automation build & test.
67-
* Ensure that the login user is gpadmin
68-
##### vagrant_release.sh
69-
* Script which provides the packaging of vagrant box images.
47+
```
48+
$ su - gpadmin
49+
$ psql postgres
50+
postgres=# select * from gp_segment_configuration;
51+
```
7052

71-
#### 3.Docker startup script
72-
##### Dockerfile
73-
* Docker image file packaging
74-
* Docker container service startup configuration, such as SSH service
75-
##### docker_deploy.h
76-
* Generates SSH public keys and automatic SSH login configuration
77-
* Start docker automation deployment for build & test
78-
##### docker_start.sh
79-
* Downloads docker image version remotely
80-
* The host code repository path is mounted on virtual machine with read-only permission, for example: source code path to /home/gpadmin/workspace/cbdb directory
81-
* Copy the source file directory to /home/gpadmin/workspace/cbdb_dev, which is the actual working directory for automation build & test.
82-
* Ensure that the login user is gpadmin
53+
5. Start the regression test. Note that this test is expected to be performed with the `gpadmin` user.
54+
55+
```
56+
$ cd ~/workspace/cbdb_dev
57+
$ make installcheck-world
58+
```
59+
60+
## Deploy with Docker
61+
62+
1. Configure the Docker image repository.
63+
64+
Configure the Docker image repository URL by adding the following JSON format configuration in the **Preferences** -\> **Docker Engine** tab in your Docker Desktop setting.
65+
66+
```json
67+
"insecure-registries": [
68+
"docker.artifactory.hashdata.xyz"
69+
]
70+
```
71+
72+
2. Start the Docker container.
73+
74+
The `docker_start.sh` scripts will automatically launch the Docker container, and compile and deploy Cloudberry Database within the container.
75+
76+
```bash
77+
$ cd ~/workspace/cbdb/deploy/docker
78+
$ bash docker_start.sh
79+
```
80+
81+
3. Log into the Docker container.
82+
83+
```bash
84+
$ docker exec -it (cotainer ID) /bin/bash
85+
```
86+
87+
4. Connect to Cloudberry Database.
88+
89+
```bash
90+
$ su gpadmin
91+
$ psql postgres
92+
postgres=# select * from gp_segment_configuration;
93+
```
94+
95+
5. Start the regression test.
96+
97+
```bash
98+
$ cd ~/workspace/cbdb_dev
99+
$ make installcheck-world
100+
```
101+
102+
## About the automation script
103+
104+
The whole script is subdivided into three main parts:
105+
106+
```
107+
|-- cbdb_deploy.sh
108+
|-- cbdb_env.sh
109+
|-- docker
110+
| |-- Dockerfile
111+
| |-- docker_deploy.sh
112+
| `-- docker_start.sh
113+
|-- script
114+
| `-- install_etcd.sh
115+
`-- vagrant
116+
|-- Vagrantfile
117+
`-- vagrant_release.sh
118+
```
119+
120+
1. The automation deployment script running in the virtualization environment.
121+
122+
Through the mechanism provided by Vagrant vagrantfile provision or Docker dockerfile ENTRYPOINTS:
123+
124+
* Provides subcommands that operates the build & test environment
125+
* Provides the necessary configuration variables to the internal script of the virtual machine and trigger specific operations
126+
127+
Passing the following compilation options through `/code/depoly/cbdb_ deploy.sh` script:
128+
129+
* Compilation options
130+
* Deployment options
131+
* Test options
132+
133+
Passing the environment variable parameters required for automation build & test through `/code/depoly/cbdb_env.sh` script.
134+
135+
2. Vagrant scripts
136+
137+
2.1 `Vagrantfile`
138+
139+
* Downloads the Vagrant virtual machine box and local virtual box creation.
140+
* The host code repository path is mounted on virtual machine with read-only permission, for example: source code path to `/home/gpadmin/workspace/cbdb` directory
141+
* Copy the source file directory to `/home/gpadmin/workspace/cbdb_dev`, which is the actual working directory for automation build & test.
142+
* Ensure that the login user is `gpadmin`.
143+
144+
145+
146+
3. Docker scripts
147+
148+
3.1 `Dockerfile`
149+
150+
* Docker image file packaging
151+
* Docker container service startup configuration, such as SSH service
152+
153+
3.2 `docker_deploy.sh`
154+
155+
* Generates SSH public keys and automatic SSH login configuration
156+
* Start Docker automation deployment for build & test
157+
158+
3.3 `docker_start.sh`
159+
160+
* Downloads Docker image version remotely
161+
* The host code repository path is mounted on virtual machine with read-only permission, for example: source code path to `/home/gpadmin/workspace/cbdb` directory
162+
* Copy the source file directory to `/home/gpadmin/workspace/cbdb_dev`, which is the actual working directory for automation build & test.
163+
* Ensure that the login user is `gpadmin`
83164
84-
<br>

0 commit comments

Comments
 (0)