Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 37 additions & 49 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,10 @@ You can learn more about the Protokit framework at the [official documentation](
- pnpm `v9.8.0`
- nvm
- (optional) For running with persistance / deploying on a server
- docker `>= 24.0`
- docker-compose `>= 2.22.0`
- docker `>= 24.0`
- docker-compose `>= 2.22.0`

**Run the following commands to get started:**

```zsh
# clone the repository
git clone https://github.com/proto-kit/starter-kit my-chain
Expand All @@ -30,21 +29,19 @@ pnpm env:development prisma:generate
# starts both the UI and the sequencer (file watcher / live reload enabled)
pnpm env:inmemory dev
```

Visit http://localhost:3000 to view the example UI, or http://localhost:8080/graphql to explore the sequencer's GraphQL APIs.

### Structural overview

The starter kit contains the following files and folders:

```
├── apps
│ └── web // example UI that connects to the app-chain's sequencer
│ ├── components // display components
│ ├── containers // smart components ("containers")
│ └── lib
│ └── stores // data stores for interacting with the app-chain's sequencer
├── docker
│ └── data // mounted as a volume for the docker containers
Expand All @@ -56,7 +53,7 @@ The starter kit contains the following files and folders:
│ │ └── processor // processor configuration (handlers, graphql resolvers, graphql server, ...)
│ ├── protocol // protocol modules (transaction fees, ...)
│ └── runtime // runtime modules (your app-chain's business logic)
│ └── modules
│ └── modules
│ ├── balances.ts // built-in example runtime module for Balances, with a faucet
│ └── withdrawals.ts // withdrawal functionality module
└── test // tests for various app-chain components
Expand All @@ -72,9 +69,8 @@ The starter-kit offers different environments to run you appchain.
You can use those environments to configure the mode of operation for your appchain depending on which stage of development you are in.

The starter kit comes with a set of pre-configured environments:

- `inmemory`: Runs everything in-memory without persisting the data. Useful for early stages of runtime development.
- `development`: Runs the sequencer locally and persists all state in databases running in docker.
- `development`: Runs the sequencer locally and persists all state in databases running in docker.
- `sovereign`: Runs your appchain fully in docker (including the UI) for testnet deployments without settlement or bridging.

Every command you execute should follow this pattern:
Expand Down Expand Up @@ -117,7 +113,7 @@ pnpm env:development prisma:migrate

#### Pruning data

Persisted data is stored under `docker/data`, you can delete this folder in case you're experiencing issues with persistence and need to reset your environment setup entirely.
Persisted data is stored under `docker/data`, you can delete this folder in case you're experiencing issues with persistence and need to reset your environment setup entirely.

However to prune data during development, you should use the `--pruneOnStartup` CLI option [documented here](#cli-options)

Expand All @@ -127,7 +123,7 @@ Ensure you've successfully started the dockerized dependencies, generated and mi

#### With live reload

> ⚠️ Be aware, the dev command will automatically restart your application when your sources change.
> ⚠️ Be aware, the dev command will automatically restart your application when your sources change.
> Please keep in mind that running the components below in `dev` mode (with live reload / watchersr) is advisable only when you're working on that specific component. In case you're experiencing issues with watches cross-triggering reload of different components, you can use the `start` command instead.

```zsh
Expand All @@ -145,16 +141,14 @@ pnpm env:development sequencer:start --filter=chain

Protokit has the ability to report metrics, logs and traces to a Grafana instance for visualisation.
These can be configured by the following environment variables

```zsh
OPEN_TELEMETRY_TRACING_URL=
OPEN_TELEMETRY_TRACING_ENABLED=

OPEN_TELEMETRY_METRICS_URL=
OPEN_TELEMETRY_METRICS_ENABLED=
OPEN_TELEMETRY_METRICS_SCRAPING_FREQUENCY=
```

````
Note that the functionality is not configured for the `in-memory` mode.

### Running the UI
Expand All @@ -165,6 +159,7 @@ pnpm env:development dev --filter=web

> You can also build/start the UI as well, instead of using `dev` command with live-reload.


### Running the indexer

⚠️ Indexer only runs with docker-enabled environments, therefore it is not available with the `inmemory` environment
Expand Down Expand Up @@ -221,46 +216,38 @@ You can define which resolvers are available in `chain/src/processor/api/resolve
At this point in time, the starter-kit offers settlement & bridging integration with lightnet (local mina network). You can enable these features by setting the `PROTOKIT_SETTLEMENT_ENABLED` environment variable to `true` in development .env file.

Follow these steps to get the sequencer to settle & bridge:

- Initialize the lightnet process, fund the sequencer operator & deploy settlement+bridging contracts:

```
pnpm env:development lightnet:start -d
pnpm protokit lightnet initialize --env development
```
```
pnpm env:development lightnet:start -d
pnpm protokit lightnet initialize
```

- Run a worker, alongside with the sequencer in separate shell instances

```
pnpm env:development worker:dev
pnpm env:development sequencer:dev
```
```
pnpm env:development worker:dev
pnpm env:development sequencer:dev
```

- Fund a testing account on lightnet (defined in the .env file)

```
pnpm protokit lightnet faucet B62qkVfEwyfkm5yucHEqrRjxbyx98pgdWz82pHv7LYq9Qigs812iWZ8 --env development
```
```
pnpm lightnet faucet B62qkVfEwyfkm5yucHEqrRjxbyx98pgdWz82pHv7LYq9Qigs812iWZ8
```

- Bridge the L1 $MINA to your app-chain, and observe your app-chain $MINA balance change after the next settlement lifecycle has been completed by the sequencer

> Token ID of MINA is `1` on both the L1 and app-chain

```
pnpm protokit bridge deposit 1 TEST_ACCOUNT_1_PRIVATE_KEY TEST_ACCOUNT_1_PUBLIC_KEY 100 --env development
```
> Token ID of MINA is `1` on both the L1 and app-chain
```
pnpm protokit bridge deposit 1 TEST_ACCOUNT_1_PRIVATE_KEY TEST_ACCOUNT_1_PUBLIC_KEY 100
```

- Withdraw your app-chain $MINA tokens back to the L1

```
pnpm protokit bridge withdraw 1 TEST_ACCOUNT_1_PRIVATE_KEY 100 --env development
```

For more detailed information about the protokit CLI commands used in this section, refer to the [Protokit CLI documentation](http://github.com/proto-kit/framework/tree/develop/packages/cli#proto-kit-cli).
```
pnpm protokit bridge withdraw 1 TEST_ACCOUNT_1_PRIVATE_KEY 100
```

## Deployments (sovereign environment)

When deploying to a server, you should push your code along with your forked starter-kit to some repository,
When deploying to a server, you should push your code along with your forked starter-kit to some repository,
then clone it on your remote server and execute it.

> Don't forget to run `pnpm env:sovereign docker:build` to build the required images.
Expand All @@ -275,7 +262,6 @@ UI will be accessible at `https://localhost` and GQL inspector will be available
### Configuration

Go to `docker/proxy/Caddyfile` and replace the `*` matcher with your domain.

```
yourdomain.com {
...
Expand All @@ -295,8 +281,7 @@ The caddy reverse-proxy automatically uses https for all connections, use this g

### Monitoring

Protokit offers monitoring via three different kinds of data and a collection of preconfigured services:

Protokit offers monitoring via three different kinds of data and a collection of preconfigured services:
- Logs via Promtail and Loki
- Metrics via OpenTelemetry and Prometheus
- Traces via OpenTelemetry, OTel Collector and Tempo
Expand All @@ -307,11 +292,10 @@ Protokit offers monitoring via three different kinds of data and a collection of
In Development mode, monitoring is disabled by default.

To enabled, edit the `development/.env` file in the following way:

1. Add the monitoring profile to `COMPOSE_PROFILES`
2. Uncomment `...DefaultModules.metrics()` in the sequencer's module definition.
Important: This has to be in front of all other modules (i.e. has to be first in the modules record)
3. Uncomment `...DefaultConfigs.metrics()` in the configuration call.
2. Uncomment `...metricsSequencerModules` in the sequencer's module definition.
Important: This has to be in front of all other modules (i.e. has to be first in the modules record)
3. Uncomment `...metricsSequencerModulesConfig` in the configuration call.

Then, run `pnpm env:development docker:up` like usual. This should start all the services needed for monitoring.
Grafana is available at `localhost:3000`.
Expand All @@ -328,10 +312,14 @@ If you want to remove the monitoring services, remove the docker profile `monito

More information about monitoring can be found [here](https://github.com/proto-kit/framework/pull/272).

*Deploying contracts*

`pnpm protokit lightnet initialize --env-path ./packages/chain/src/core/environments/sovereign/scripts.env`

## Building the framework from source

1. Make sure the framework is located under ../framework from the starter-kit's location
2. Adapt your starter-kit's `packages/chain` and `apps/web` package.json to use the file:// references to framework, including
references to `o1js` and `tsyringe`. Important: Make sure to update references in both chain and web, otherwise the location of the node_modules will be different and lead to errors
references to `o1js` and `tsyringe`. Important: Make sure to update references in both chain and web, otherwise the location of the node_modules will be different and lead to errors
3. Go into the framework folder, and build a docker image containing the sources with `docker build -f ./packages/deployment/docker/development-base/Dockerfile -t protokit-base .`
4. Replace the first line of `docker/base/Dockerfile` and `docker/web/Dockerfile` to use `FROM protokit-base:latest`
6 changes: 3 additions & 3 deletions docker/base/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# For regular deployment
# FROM node:18
# FROM node:22

# For locally built and imported framework references
# Make sure you build the base-image first
# Make sure you build the base image first
FROM protokit-base:latest

WORKDIR /app
Expand Down Expand Up @@ -37,4 +37,4 @@ RUN dos2unix ./entry.sh
RUN chmod +x ./entry.sh

# Migrate + execute
ENTRYPOINT ["./entry.sh"]
ENTRYPOINT ["./entry.sh"]
10 changes: 6 additions & 4 deletions docker/base/entry.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,26 +3,28 @@
if [[ ${PROTOKIT_SHOULD_ATTEMPT_DB_MIGRATION} ]]; then
echo "Migrating on ${DATABASE_URL}"
cd packages/chain/node_modules/@proto-kit/persistance
pnpm dlx prisma migrate deploy
pnpx prisma@5.18.0 migrate deploy
cd ../../../../..
fi

if [[ ${PROTOKIT_SHOULD_ATTEMPT_INDEXER_DB_MIGRATION} ]]; then
echo "Migrating on ${INDEXER_DATABASE_URL}"
cd packages/chain/node_modules/@proto-kit/indexer
pnpm dlx prisma migrate deploy
pnpx prisma@5.18.0 migrate deploy
cd ../../../../..
fi

if [[ ${PROTOKIT_SHOULD_ATTEMPT_PROCESSOR_DB_MIGRATION} ]]; then
echo "Migrating on ${PROCESSOR_DATABASE_URL}"
cd packages/chain/src/processor
pnpm dlx prisma migrate deploy
pnpx prisma@5.18.0 migrate deploy
cd ../../../../..
fi

cd /app
cd packages/chain

echo $@

# We can't use start:headless here bcs that task hardcodes the .js file, we want it to be extensible
node --experimental-vm-modules --experimental-wasm-modules --es-module-specifier-resolution=node $@
node --loader ts-node/esm --experimental-vm-modules --experimental-wasm-modules --es-module-specifier-resolution=node --no-warnings $@
2 changes: 1 addition & 1 deletion docker/indexer/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ services:
- 8081:8080
command:
[
"./packages/chain/dist/start.js start ./environments/${PROTOKIT_ENV_FOLDER}/indexer.config.js",
"./dist/start.js start ./core/environments/${PROTOKIT_ENV_FOLDER}/indexer.config.js",
]
labels:
logging: "promtail"
Expand Down
4 changes: 2 additions & 2 deletions docker/processor/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,10 @@ services:
ports:
- 8082:8080
working_dir: /app
command: ["./packages/chain/dist/start.js start ./environments/${PROTOKIT_ENV_FOLDER}/processor.config.js",]
command: ["./dist/start.js start ./core/environments/${PROTOKIT_ENV_FOLDER}/processor.config.js",]
labels:
logging: "promtail"
logging_jobname: "processor"

networks:
processor-net:
processor-net:
7 changes: 5 additions & 2 deletions docker/sequencer/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,9 @@ services:
- PROTOKIT_DISPATCHER_CONTRACT_PUBLIC_KEY=${PROTOKIT_DISPATCHER_CONTRACT_PUBLIC_KEY}
- PROTOKIT_MINA_BRIDGE_CONTRACT_PRIVATE_KEY=${PROTOKIT_MINA_BRIDGE_CONTRACT_PRIVATE_KEY}
- PROTOKIT_MINA_BRIDGE_CONTRACT_PUBLIC_KEY=${PROTOKIT_MINA_BRIDGE_CONTRACT_PUBLIC_KEY}
- PROTOKIT_PROOFS_ENABLED=${PROTOKIT_PROOFS_ENABLED}
- PROTOKIT_BLOCK_ARGUMENT_BATCH_SIZE=${PROTOKIT_BLOCK_ARGUMENT_BATCH_SIZE}
- PROTOKIT_STATE_TRANSITION_BATCH_SIZE=${PROTOKIT_STATE_TRANSITION_BATCH_SIZE}

- OPEN_TELEMETRY_TRACING_ENABLED=${OPEN_TELEMETRY_TRACING_ENABLED}
- OPEN_TELEMETRY_TRACING_URL=${OPEN_TELEMETRY_TRACING_URL}
Expand All @@ -61,10 +64,10 @@ services:
- lightnet-net
ports:
- 8080:8080
command: ["./packages/chain/dist/start.js start ./environments/${PROTOKIT_ENV_FOLDER}/chain.config.js"]
command: ["./dist/start.js start ./core/environments/${PROTOKIT_ENV_FOLDER}/chain.config.js"]
labels:
logging: "promtail"
logging_jobname: "sequencer"

networks:
chain-net:
chain-net:
50 changes: 47 additions & 3 deletions docker/worker/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ services:
- REDIS_PORT=${REDIS_PORT}
- REDIS_PASSWORD=${REDIS_PASSWORD}


- PROTOKIT_TRANSACTION_FEE_RECIPIENT_PRIVATE_KEY=${PROTOKIT_TRANSACTION_FEE_RECIPIENT_PRIVATE_KEY}
- PROTOKIT_TRANSACTION_FEE_RECIPIENT_PUBLIC_KEY=${PROTOKIT_TRANSACTION_FEE_RECIPIENT_PUBLIC_KEY}
- PROTOKIT_LOG_LEVEL=${PROTOKIT_LOG_LEVEL}
Expand All @@ -22,14 +21,59 @@ services:
- PROTOKIT_DISPATCHER_CONTRACT_PUBLIC_KEY=${PROTOKIT_DISPATCHER_CONTRACT_PUBLIC_KEY}
- PROTOKIT_MINA_BRIDGE_CONTRACT_PRIVATE_KEY=${PROTOKIT_MINA_BRIDGE_CONTRACT_PRIVATE_KEY}
- PROTOKIT_MINA_BRIDGE_CONTRACT_PUBLIC_KEY=${PROTOKIT_MINA_BRIDGE_CONTRACT_PUBLIC_KEY}
- PROTOKIT_SETTLEMENT_INTERVAL=${PROTOKIT_SETTLEMENT_INTERVAL}
- PROTOKIT_SETTLEMENT_ENABLED=${PROTOKIT_SETTLEMENT_ENABLED}
- PROTOKIT_PROOFS_ENABLED=${PROTOKIT_PROOFS_ENABLED}
- PROTOKIT_BLOCK_ARGUMENT_BATCH_SIZE=${PROTOKIT_BLOCK_ARGUMENT_BATCH_SIZE}
- PROTOKIT_STATE_TRANSITION_BATCH_SIZE=${PROTOKIT_STATE_TRANSITION_BATCH_SIZE}
- PROTOKIT_WORKER_VARIANT=${PROTOKIT_WORKER_VARIANT}
profiles:
- worker
- worker-specialized
depends_on:
redis:
condition: service_healthy
networks:
- db-net
command: ["./packages/chain/dist/start.js start ./environments/${PROTOKIT_ENV_FOLDER}/worker.config.js"]
- lightnet-net
volumes:
- ${HOME}/.cache/o1js:/home/node/.cache/o1js
command: ["./dist/start.js start ./core/environments/${PROTOKIT_ENV_FOLDER}/worker.config.js"]

worker-2:
image: starterkit-chain-base
build:
dockerfile: ./docker/base/Dockerfile
context: ../..
container_name: starterkit-worker-2
environment:
- REDIS_HOST=${REDIS_HOST}
- REDIS_PORT=${REDIS_PORT}
- REDIS_PASSWORD=${REDIS_PASSWORD}

- PROTOKIT_TRANSACTION_FEE_RECIPIENT_PRIVATE_KEY=${PROTOKIT_TRANSACTION_FEE_RECIPIENT_PRIVATE_KEY}
- PROTOKIT_TRANSACTION_FEE_RECIPIENT_PUBLIC_KEY=${PROTOKIT_TRANSACTION_FEE_RECIPIENT_PUBLIC_KEY}
- PROTOKIT_LOG_LEVEL=${PROTOKIT_LOG_LEVEL}
- PROTOKIT_SEQUENCER_PRIVATE_KEY=${PROTOKIT_SEQUENCER_PRIVATE_KEY}
- PROTOKIT_SEQUENCER_PUBLIC_KEY=${PROTOKIT_SEQUENCER_PUBLIC_KEY}
- PROTOKIT_SETTLEMENT_CONTRACT_PRIVATE_KEY=${PROTOKIT_SETTLEMENT_CONTRACT_PRIVATE_KEY}
- PROTOKIT_SETTLEMENT_CONTRACT_PUBLIC_KEY=${PROTOKIT_SETTLEMENT_CONTRACT_PUBLIC_KEY}
- PROTOKIT_DISPATCHER_CONTRACT_PRIVATE_KEY=${PROTOKIT_DISPATCHER_CONTRACT_PRIVATE_KEY}
- PROTOKIT_DISPATCHER_CONTRACT_PUBLIC_KEY=${PROTOKIT_DISPATCHER_CONTRACT_PUBLIC_KEY}
- PROTOKIT_MINA_BRIDGE_CONTRACT_PRIVATE_KEY=${PROTOKIT_MINA_BRIDGE_CONTRACT_PRIVATE_KEY}
- PROTOKIT_MINA_BRIDGE_CONTRACT_PUBLIC_KEY=${PROTOKIT_MINA_BRIDGE_CONTRACT_PUBLIC_KEY}
- PROTOKIT_SETTLEMENT_ENABLED=${PROTOKIT_SETTLEMENT_ENABLED}
- PROTOKIT_PROOFS_ENABLED=${PROTOKIT_PROOFS_ENABLED}
- PROTOKIT_BLOCK_ARGUMENT_BATCH_SIZE=${PROTOKIT_BLOCK_ARGUMENT_BATCH_SIZE}
- PROTOKIT_STATE_TRANSITION_BATCH_SIZE=${PROTOKIT_STATE_TRANSITION_BATCH_SIZE}
- PROTOKIT_WORKER_VARIANT=${PROTOKIT_WORKER_VARIANT_2}
profiles:
- worker-specialized
depends_on:
redis:
condition: service_healthy
networks:
- db-net
- lightnet-net
volumes:
- ${HOME}/.cache/o1js:/home/node/.cache/o1js
command: ["./dist/start.js start ./core/environments/${PROTOKIT_ENV_FOLDER}/worker.config.js"]
3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -41,12 +41,13 @@
"turbo": "^2.1.2"
},
"dependencies": {
"@proto-kit/cli": "../framework/packages/cli",
"@types/graphql-fields": "^1.3.9",
"chain": "^0.4.0",
"dotenv-cli": "^7.4.2",
"tslib": "^2.7.0",
"type-graphql": "2.0.0-rc.2",
"@proto-kit/cli": "../framework/packages/cli"
"ts-node": "^10.9.2"
},
"packageManager": "pnpm@9.8.0+sha256.56a9e76b51796ca7f73b85e44cf83712862091f4d498c0ce4d5b7ecdc6ba18f7",
"name": "starter-kit"
Expand Down
Loading