This is a project created to simulate satellite updates and allow you to change the status in real time.
Table of Contents
You will need to have docker installed and some knowledge of how to clone from github.
To run the application locally, once Docker is installed, run this command from the satellite-simulation folder to deploy with Docker compose.
$ docker compose up -- build
[+] Building 10.4s (27/27) FINISHED
...
=> [web] resolving provenance for metadata file
[+] Running 4/4
...
Attaching to postgres_db_container, redis-container, satellite-project-container, vue-frontend-container
After the application starts, navigate to http://localhost:8000 in your web browser to verify it is running.
Note: If it is not running, try building it again as sometimes I noticed the django.sh scripts running with /r when copied from windows.
If you want to create a superuser for django, you can run this command to create an admin user.
$ docker exec -it satellite-project-container python manage.py createsuperuser
You can also run this command in the django container exec to add an admin user instead.
python manage.py createsuperuser
This superuser will allow you to open the admin panel to add and manage spacecrafts: http://localhost:8000/admin
You can also directly access the API directly: http://localhost:8000/api/docs
Once you are done, you can stop and remove the containers using this command.
$ docker compose down
You are a spacecraft software engineer and your job is to create a simple satellite simulation.
Your task is to design a full-stack application that creates satellites, updates their status, and updates a database. CRUD
Please dockerize this application to make it easier to test. Language choice is up to you. If there are any questions, use your best judgment and explain why you chose to go that route. There are no wrong choices as long as the basic requirements are met.
Please give a description of your approach for this assignment.
- Add an endpoint that creates a spacecraft with an ID and a status (NOMINAL, MALFUNCTIONING). Add this spacecraft to a database (including status).
- Add endpoints that allow users to update the spacecraft status. (Other instances will see status updates through pub/sub, websocket, etc.)
- The statuses should also randomly change over time.
- Every time a status changes, send an alert to the frontend (pub/sub, websocket, etc.), and update the status in the database.
- The frontend can be very basic. Its only purpose is to show off the statuses for each spacecraft.
- List each spacecraft and its associated status. Add a button for each spacecraft that will update 'MALFUNCTIONING' statuses to 'NOMINAL'.
For time constraints, I will need to keep this project scoped to MVP and then add lower and mid-priority tasks only if I have time.
The main MVP concepts are:
- Allow CRUD operations for satellites, with an ENUM status NOMINAL, MALFUNCTIONING
- Create some sort of event driven architecture that alerts the client of a change.
- Containerize the application for easy testing
- A Basic front end
For the backend, I decided to use Django because it has built-in security roles, it uses python similar to this position, and I want to dig deeper into Django's abilities to handle real time communication.
I was debating on using websockets but to decouple the connection and less complexity, I decided to use Server Side Events. I also wanted to avoid using the cloud infrastructure at this time to keep it simple and SSE should be quick for a MVP that can be extended if needed. For the front-end I was thinking about adding a separate react front end. If I have time, I might either add it to the project or just build something quick using Django views.
Normally I would use a Kanban project like in https://kanbanflow.com/ to manage the priorities and backlog for personal projects but for ease of use/reading, I will summarize it here:
-
Research Current trends on real time communication.
-
WebTransport - pass for now
- Newest, but less support at this time. Would be a fun side project
-
HTTP polling - not efficient
- Maybe if we know there will always be an update at a certain time, but generally this is not recommended
-
AWS pub/sub - too much setup
- Eventbridge could work as well or other cloud options, but it’s too much setup for this simple project.
-
WebSockets - considered
- More setup than other solutions
- Harder to scale than SSE
-
Server Side Events (SSE) - chose this solution
- One directional reads from the server
- Updates can be done through HTTP calls
-
WebTransport - pass for now
- Research Django SSE
-
Allow CRUD operations for satellites, with a status NOMINAL or MALFUNCTIONING
- Setup CRUD’s using Ninja
- Containerize the application for easy testing
- A Basic front end
- Split Localhost and Production Template
- Basic unit test example
- Mock results for testing
- Update the database to Postgres
- React front end
- Create a Postman library to add regression tests
- Add Next.js
- Front end unit testing
-
AWS server setup and deployment
- IoC w/ terraform
- Create a websocket version or webTransport version for fun
- CI/CD Pipelines
- Linters and other development tools
- K8 Scaling
When using SSE, the ideal route would be to allow triggers from the database. At this point, the example I was using pushed updates from the endpoint but not from a trigger on the database. This can work, but is not ideal as the trigger should come from the database.
Ideally, create a CRON to update the database, the database will trigger an event that SSE is watching on. When that event happens then trigger the update…(Note CRON’s only run every minute, so went into creating tasks using Celery Beat’s and Redis. For now just going to add the update task into the simulation and maybe add beats later if time to decouple the simulation)
The current SSE structure is basically polling but from the server side at intervals similar to polling. This is better than polling on the client side because it is close to the data, but still not ideal as it is based on a timer.
A better solution would be watching the database and having it ‘signal’ the trigger from the insert. Not quite sure how to do this in Django at the moment, still looking for solutions.
Internal thought: for now run SSE based on a timer that looks up the database. The ideal solution here would only push the changes needed. I was thinking of storing the state on the backend then searching through it and only sending the updated items. This could be from the lastUpdatedDate compared to when the SSE was started. That would allow a lookup on the state to see what items changed. This doesn’t seem ideal, but could be developed quickly to meet the deadline and allow less network traffic from the front end.
Last Push: Goal is to get some data into the front end and refactor the event_stream after. There are many Websocket tutorials out there and it might have been an easier choice for a quicker delivery. That being said, I still like the one way connections with SSE opposed to bi-directional flow as I think it might be harder to load balance, use with firewalls and proxies.(It just takes more time than I thought to set up)
Last issue working on: celery is not working anymore once containerized. It doesn’t seem to be loading the models from django. For now, bypassing this by having vue push updates at random intervals and console logging for easy viewing.
This was a fun project that allowed me to dig deeper into researching, exploring different technologies and ramping up into some unfamiliar territories. Setting up my docker containers was frustrating at times, but fun. Keeping this project as minimal as possible was a challenge as there were so many things I wanted to add to it. When I created the github check-ins I tried to keep them grouped by feature, so hopeful this can be a good training examples for others.
As far as meeting the requirements for the project, I trust I met them and beyond. I created an CRUD API that can be accessed quite easily if other apps need to use it. Security can easily be added to it and CORS can be added per IP fairly easy. Django Admin allows quick CRUD actions instead of using the API if desired. The frontend is connecting to the SSE, and updating from the database. The front end also is creating random updates l. I trust the documentation of this project is good to give a flavor of my approach and thought process as well as my ability to show I can learn/implement new technologies in a short period of time.
I desired to get celery to work under containers but couldn’t get it to write to the correct database. When using sqlite, I noticed that it was writing outside the containers to my localhost database so I figured it was a setting. To get around this I tried to change where celery read from the django models, but couldn’t figure out how to change it. I googled suggestions and people pointed me to adding CELERY_RESULT_BACKEND_DB to the database directly. I tried this to no avail. I even set up Postgres to run in a separate container but was having issues with it connecting. This issue took longer than the whole rest of the project to debug and so I tabled it for the time being. (I even ran old containerized projects from github and they were having separate issues with missing celery library versions during compose up…) I decided to add the random updates from the front end instead.
I added vue.js on the front end as a separate SPA and with login permissions for future updates. The front end is now functioning with SSE loading after bypassing Celery.
I added Postgres to speed up the polling and cleaner.