See our CONTRIBUTING file.

Repository Structure

This repository is structured as a monorepo, containing the following projects:

• container-engine-lib: Library used to abstract away container engine being used by the enclave.
• core: Container launched inside an enclave to coordinate its state
• engine: Container launched to coordinate enclaves
• api: Defines the API of the Kurtosis platform (engine and core)
• cli: Produces CLI binary, allowing interaction with the Kurtosis system
• docs: Documentation that is published to docs.kurtosis.com
• internal_testsuites: End to end tests

Dev Dependencies (Nix)

Install the Nix package manager.

bash
1sh <(curl -L https://nixos.org/nix/install)

And enable some Nix flags (alternatively you can add --extra-experimental-features 'nix-command flakes' every time calling the nix command):

bash
1mkdir -p ~/.config/nix
2echo "experimental-features = nix-command flakes" >> ~/.config/nix/nix.conf

And to bring the environment up, just open a new shell terminal, go to the root folder of the repo and run:

bash
1nix  develop

This will download all dev deps and setup the environment accordingly.

You can also use the direnv to automatically load the environment when entering the main folder or using a plugin in your preferred IDE:

• vscode: mkhl.direnv
• jet brains: Direnv integration

Direnv can also be easily installed with Nix (or HomeBrew if you prefer):

bash
1nix-env -f '<nixpkgs>' -iA direnv

Now you just to add the direnv hook to your shell:

bash
1echo 'eval "$(direnv hook bash)"' >> ~/.bashrc
2# or for ZSH
3echo 'eval "$(direnv hook zsh)"' >> ~/.zshrc

Now next time you open a new shell terminal and go to repo's folder you environment will update and load automatically.

Dev Dependencies (Manual install)

The commands below assume that the env variable BREW_PREFIX contains the brew prefix.

bash
1BREW_PREFIX="$(brew --prefix)"

Bash (5 or above)

On MacOS:

bash
1# Install modern version of bash, the one that ships on MacOS is too old
2brew install bash
3# Allow bash as shell
4echo "${BREW_PREFIX}/bin/bash" | sudo tee -a /etc/shells
5# Optional: make bash your default shell
6chsh -s "${BREW_PREFIX}/bin/bash"

Git

On MacOS:

bash
1# Install modern version of git, the one that ships on MacOS can be too old
2brew install git

Docker

On MacOS:

bash
1brew install docker

Go (1.23 or above)

On MacOS:

bash
1brew install [email protected]
2# Add the Go binary dir to your PATH
3PATH="${BREW_PREFIX}/opt/[email protected]/bin:$PATH"
4# Add the GOPATH bin dir to your PATH
5PATH="${HOME}/go/bin:$PATH"

On Ubuntu:

bash
1wget https://go.dev/dl/go1.23.7.linux-amd64.tar.gz
2tar -C /usr/local -zxf go1.23.7.linux-amd64.tar.gz
3# Add the following to your bashrc or equivalent.
4export PATH=$PATH:/usr/local/go/bin

Goreleaser

On MacOS:

bash
1brew install goreleaser/tap/goreleaser

On Ubuntu:

bash
1echo 'deb [trusted=yes] https://repo.goreleaser.com/apt/ /' | sudo tee /etc/apt/sources.list.d/goreleaser.list
2sudo apt update
3sudo apt install goreleaser

Node (20.* or above) and Yarn

On MacOS, using NVM:

bash
1brew install nvm
2mkdir ~/.nvm
3nvm install 20.11.0
4npm install -g yarn

On Ubuntu, using NVM:

bash
1curl https://raw.githubusercontent.com/creationix/nvm/master/install.sh | bash
2source ~/.bashrc
3nvm install 20.11.0
4npm install -g yarn

Rust

On MacOS, Ubuntu:

bash
1curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Go and Typescript protobuf compiler binaries

On MacOS:

bash
1brew install protoc-gen-go
2brew install protoc-gen-go-grpc
3go install connectrpc.com/connect/cmd/protoc-gen-connect-go@latest
4yarn global add ts-protoc-gen
5yarn global add grpc-tools

On Ubuntu:

bash
1go install google.golang.org/grpc/cmd/protoc-gen-go@latest
2go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest
3go install connectrpc.com/connect/cmd/protoc-gen-connect-go@latest
4yarn global add ts-protoc-gen
5yarn global add grpc-tools

OpenAPI generators for Go and Typescript

On MacOS:

bash
1go install github.com/deepmap/oapi-codegen/cmd/[email protected]
2yarn global add [email protected]

On Ubuntu:

bash
1go install github.com/deepmap/oapi-codegen/cmd/[email protected]
2yarn global add [email protected]

Build Instructions

To build the entire project, run:

bash
1./scripts/build.sh

Note: If you encounter module import errors related to github.com/kurtosis-tech/kurtosis/kurtosis_version, you may need to generate the version constants first:

bash
1./scripts/generate-kurtosis-version.sh

This script generates the required kurtosis_version.go file that contains version constants used throughout the codebase. The file is automatically generated during the build process, but may need to be run manually in some development scenarios.

To only build a specific project, run the script on ./PROJECT/PATH/script/build.sh, for example:

bash
1./container-engine-lib/scripts/build.sh
2./core/scripts/build.sh
3./api/scripts/build.sh
4./engine/scripts/build.sh
5./cli/scripts/build.sh

If there are any changes to the Protobuf files in the api subdirectory, the Protobuf bindings must be regenerated:

bash
1./api/scripts/regenerate-protobuf-bindings.sh

If you are developing Kurtosis over a Podman cluster, run:

bash
1./scripts/build.sh false true # no debug image, use podman

This will use the Podman image builder for building images. See documentation in scripts for more details on build options.

Running Dev Version

After building the project, run ./cli/cli/scripts/launch-cli.sh just like you would the kurtosis command. This will launch the latest locally built version of the CLI, which will also start the engine and core containers using their latest built images.

You can verify this by running ./cli/cli/launch-cli.sh engine status and

1A Kurtosis engine is running with the following info:
2Version:   53d823 <-- or `-dirty` depending on the commit

The version will be identical to the version on the latest dev versions of the engine image created - (can verify with docker images). Enclaves started by the engine will be started with the same version as the engine.

If you'd like to specify a different core image version than that of the engine, you can do so with the --api-container-version flag on enclave add (e.g. ./cli/cli/scripts/build.sh enclave add --api-container-version <image tag>).

If you are working on multiple dev versions of the engine at a time, you can use engine restart --version <image tag> to specify exactly what version of the engine to use.

For frequent contributors, we recommend attaching an alias to kurtosis and ./cli/cli/scripts/launch-cli.sh.

bash
1alias kt="kurtosis"
2alias dkt="$(pwd)/cli/cli/scripts/launch-cli.sh"

If you want tab completion on the recently built CLI, you can alias it to kurtosis:

bash
1alias kurtosis="$(pwd)/cli/cli/scripts/launch-cli.sh"
2kurtosis enclave add

Unit Test Instructions

Build scripts also run unit tests as part of the build process.

For all Go modules, run go test ./... on the module folder. For example:

bash
1cd cli/cli/
2go test ./...

E2E Test Instructions

Each project's build script also runs the unit tests inside the project. Running ./script/build.sh will guarantee that all unit tests in the monorepo pass.

To run the end-to-end tests:

1. Make sure Docker is running

console
1$ docker --version
2Docker version X.Y.Z

2. Make sure Kurtosis Engine is running

console
1$ kurtosis engine status
2A Kurtosis engine is running with the following info:
3Version:   0.X.Y

3. Run test.sh script

console
1$ ./internal_testsuites/scripts/test.sh

If you are developing the Typescript test, make sure that you have first built api/typescript. Any changes made to the Typescript package within api/typescript aren't hot loaded as of 2022-09-29.

Run Debug Instructions (for Golang code so far)

For running CLI with Golang remote debug:

1. Build the CLI dev binary and run the command you want to debug (kurtosis version in this example), this will start the debug server and will wait for a client connection

bash
1cli/cli/scripts/build.sh
2source ./scripts/set_kt_alias.sh
3ktdebug version

2. Open the command's file you want to debug
3. Add the breakpoint in the line where you want to stop the cursor
   
4. Then choose the "CLI-remote-debug" run configuration in the "run panel"
5. Press the "debug" button
   
6. Use the debug panel to inspect the variables value and continue with the debug flow
   

For running CLI with Delve debug client:

1. Build the CLI dev binary and run the command you want to debug (kurtosis version in this example), but first pass "dlv-terminal" as the first argument (this will start the Delve client in the terminal)

bash
1cli/cli/scripts/build.sh
2source ./scripts/set_kt_alias.sh
3ktdebug dlv-terminal version

2. You can add a new breakpoint using the terminal client and the break command

bash
1(dlv) break version.run:1

3. You can move the cursor to the breakpoint created in the previous step with the continue command

bash
1(dlv) continue

For running Kurtosis engine with Golang remote debug:

1. Run the main build script with the first argument debug_mode as true. This will generate a new Kurtosis engine container image which will contain the debug suffix in the name.

bash
1scripts/build.sh true 

2. Add the breakpoint in the line where you want to stop the cursor
   
3. Run the engine in debug mode with the ktdev engine start --debug-mode or the ktdev engine restart --debug-mode commands

bash
1source ./scripts/set_kt_alias.sh
2ktdev engine start --debug-mode 

4. Then choose the "Engine-remote-debug" run configuration in the "run panel"
5. Press the "debug" button
   
6. Make a call to the engine's server (you can use the Kurtosis CLI or Postman) in order to reach out the breakpoint in the code
7. Use the debug panel to inspect the variables value and continue with the debug flow
   
8. You can debug the CLI and the Kurtosis engine's server at the same time by running it with ktdebug instead of ktdev mentioned in a previous step, remember to run both remote debug configurations in the Goland IDE.

bash
1source ./scripts/set_kt_alias.sh
2ktdebug engine start

Additional steps if you are debugging Kurtosis engine in K8s:

1. Upload the engine's image for debug to the K8s cluster

bash
1# for example:
2k3d image load kurtosistech/engine:5ec6eb-dirty-debug

2. Run the port-forward script before pressing the debug button in Golang (in another terminal instance) to bind the host's port to the container's debug server port

bash
1scripts/port-forward-engine-debug.sh

3. Do not forget to run the Kurtosis gateway after calling the engine's server (in another terminal instance also)

bash
1ktdev gateway

For running Kurtosis APIC with Golang remote debug:

1. Run the main build script with the first argument debug_mode as true. This will generate a new Kurtosis APIC container image which will contain the debug suffix in the name.

bash
1scripts/build.sh true 

2. Add the breakpoint in the line where you want to stop the cursor.
   
3. Run the Kurtosis engine in debug more or not depending on if you want to also debug the engine.

bash
1source ./scripts/set_kt_alias.sh
2ktdev engine start --debug-mode
3
4OR
5
6ktdev engine start # you will have to build the engine in the regular way `engine/scripts/build.sh` if you choose this version

4. Add a new enclave in debug mode with the enclave add command and passing the debug-mode flag. This will create a new APIC container with the debug server port bounded and waiting for a connection. IMPORTANT: You can only run one enclave in debug mode so far, if you want to run another one it will fail due the debug port is already in use,

bash
1ktdev enclave add --debug-mode 

5. Then choose the "APIC-remote-debug" run configuration in the "run panel"
6. Press the "debug" button
   
7. Find the APIC's GRPC server port in the host machine (you can check it in Docker Desktop or using the Docker CLI, it's the one bounded with the container's 7443 port)
8. Make a call to the APIC's server (you can use the Kurtosis CLI or Postman) in order to reach out the breakpoint in the code
9. Use the debug panel to inspect the variables value and continue with the debug flow
   
10. You can debug the CLI, the Kurtosis engine's server and the Kurtosis APIC's server at the same time by running it with ktdebug instead of ktdev mentioned in a previous step, remember to run the three remote debug configurations in the Goland IDE.

bash
1source ./scripts/set_kt_alias.sh
2ktdev engine start --debug-mode
3ktdebug enclave add

Additional steps if you are debugging Kurtosis engine in K8s:

1. Upload the APIC's image for debug to the K8s cluster

bash
1# for example:
2k3d image load kurtosistech/core:5ec6eb-dirty-debug

2. Run the port-forward script before pressing the debug button in Golang (in another terminal instance) to bind the host's port to the container's debug server port

bash
1scripts/port-forward-apic-debug.sh enclave-name

3. Do not forget to run the Kurtosis gateway after calling the APIC's server (in another terminal instance also)

bash
1ktdev gateway