GH-451: [Storage] Container baked with Document Data in MongoDB

[vibhatha]

This PR closes #451

[gemini-code-assist]

Summary of Changes

Hello @vibhatha, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the deployment of MongoDB and Neo4j services by implementing a build-time data ingestion strategy. Instead of restoring data at container runtime, the Docker images are now pre-populated with essential document and graph data during their construction. This change streamlines the container startup process, ensuring that instances are immediately operational with the necessary datasets, and standardizes data paths for both databases.

Highlights

• Build-time Data Ingestion for MongoDB: The MongoDB Dockerfile now incorporates a multi-stage-like build process to download, restore, and bake data into the /tmp/db directory during image creation, ensuring containers are pre-populated.
• MongoDB Path Relocation: MongoDB's primary database path (dbPath) has been shifted from /data/db to /tmp/db within the container, facilitating the build-time data baking process.
• Neo4j Data Path Standardization: Neo4j data directories have been consistently moved to /tmp/neo4j_data, /tmp/neo4j_logs, /tmp/neo4j_import, and /tmp/neo4j_plugins, aligning with the strategy of managing data within temporary paths during the build.
• MongoDB Service Type Update: The Choreo component configuration for MongoDB was updated from REST to TCP, correctly reflecting MongoDB's communication protocol.
• Backup Version Update: The default MongoDB backup version has been updated to 0.0.4, ensuring the latest data is used for pre-population.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces build-time data ingestion for the MongoDB container, aligning it with the setup for other databases in the project. This involves modifying the MongoDB Dockerfile to use /tmp/db for data, and adding a new RUN step to download and restore a database backup during the image build. The Neo4j Dockerfile is also updated to use /tmp for its directories.

My review focuses on the new functionality in the MongoDB Dockerfile. I've identified a logic bug that causes a function to report failure on success, and several critical security vulnerabilities where credentials are exposed on the command line during the build process. The feedback provides specific suggestions to address these issues.

[gemini-code-assist]

The function restore_from_github now returns 1 on success. A return code of 1 typically signifies an error, which will cause the calling logic (restore_from_github || log ...) to incorrectly report a failure. This should return 0 on success.

        return 0\n\

[gemini-code-assist]

Passing the password as part of the --eval string exposes it in the process list, which is a security risk. This is also inconsistent with how a user is created in the entrypoint script (line 203), which uses a 'here string' (<<<) to pass the command via standard input. Please use a 'here string' here as well to avoid exposing credentials.

    mongo admin --quiet <<< "db.createUser({user: '${MONGO_INITDB_ROOT_USERNAME}', pwd: '${MONGO_INITDB_ROOT_PASSWORD}', roles: [{role: 'root', db: 'admin'}]})" && \

References

1. Avoid passing sensitive information like passwords as command-line arguments to prevent them from being exposed in the process list. Instead, use methods like here strings (<<<) to pass the data via standard input, or write it to a temporary file that is securely handled and deleted.

[gemini-code-assist]

Passing credentials via command-line arguments to mongorestore is a security risk as they can be exposed in the process list. This violates a general security rule for this repository.

While mongorestore has limitations in how it accepts credentials, this is a significant security concern. Please investigate more secure alternatives, such as checking if this version of mongorestore can read credentials from a file or environment variables.

References

1. Avoid passing sensitive information like passwords as command-line arguments to prevent them from being exposed in the process list. Instead, use methods like here strings (<<<) to pass the data via standard input, or write it to a temporary file that is securely handled and deleted.

[gemini-code-assist]

Passing credentials via command-line arguments is a security risk as they can be exposed in the process list. Instead of using --username and --password flags, you can use a 'here string' to pass an authentication command followed by the shutdownServer command to the mongo shell via standard input.

    mongo admin --username=${MONGO_INITDB_ROOT_USERNAME} --password=${MONGO_INITDB_ROOT_PASSWORD} --authenticationDatabase=admin --quiet <<< "db.auth('${MONGO_INITDB_ROOT_USERNAME}', '${MONGO_INITDB_ROOT_PASSWORD}'); db.shutdownServer();" && \

References

1. Avoid passing sensitive information like passwords as command-line arguments to prevent them from being exposed in the process list. Instead, use methods like here strings (<<<) to pass the data via standard input, or write it to a temporary file that is securely handled and deleted.