Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add state dump and loading for the sequencer #159

Closed
wants to merge 8 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,8 @@ starknet_api = { git = "https://github.com/starkware-libs/starknet-api", rev = "
# Other
async-trait = "0.1.58"
bytes = "1"
cairo-vm = "0.8.2"
cairo-lang-starknet = "2.2.0"
chrono = { version = "0.4.26", features = ["serde"] }
ctor = "0.2.4"
dotenv = "0.15.0"
Expand Down
6 changes: 6 additions & 0 deletions crates/sequencer/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,17 @@ license.workspace = true
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
anyhow = "1.0.75"
# Starknet
# TODO: remove the blockifier patch on the workspace once we can remove Katana.
blockifier = { package = "blockifier", git = "https://github.com/starkware-libs/blockifier.git", tag = "v0.3.0-rc0" }
cairo-lang-starknet = { workspace = true }
cairo-vm = { workspace = true }
serde.workspace = { workspace = true }
serde_json.workspace = true
starknet_api = { workspace = true }
starknet = { workspace = true }
thiserror.workspace = true

# Other
tracing = { workspace = true }
Expand Down
5 changes: 4 additions & 1 deletion crates/sequencer/src/constants.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,18 @@ pub mod test_constants {
use starknet::core::types::FieldElement;
use starknet_api::{
block::{BlockNumber, BlockTimestamp},
core::{ClassHash, CompiledClassHash, ContractAddress, PatriciaKey},
core::{ClassHash, CompiledClassHash, ContractAddress, Nonce, PatriciaKey},
hash::StarkFelt,
state::StorageKey,
};

lazy_static::lazy_static! {
pub static ref TEST_CONTRACT_ADDRESS: ContractAddress = ContractAddress(*ONE_PATRICIA);
pub static ref TEST_CONTRACT_ACCOUNT: ContractAddress = ContractAddress(*TWO_PATRICIA);
pub static ref TEST_ADDRESS: StarkFelt = *ONE_FELT;
pub static ref TEST_CONTRACT: StarkFelt = *TWO_FELT;
pub static ref TEST_STORAGE_KEY: StorageKey = StorageKey(*ONE_PATRICIA);
pub static ref TEST_NONCE: Nonce = Nonce(*ONE_FELT);
pub static ref SENDER_ADDRESS: FieldElement = FieldElement::from(2u8);
pub static ref SEQUENCER_ADDRESS: ContractAddress = ContractAddress(TryInto::<PatriciaKey>::try_into(StarkFelt::from(1234u16)).unwrap());
pub static ref FEE_TOKEN_ADDRESS: ContractAddress = ContractAddress(TryInto::<PatriciaKey>::try_into(StarkFelt::from(12345u16)).unwrap());
Expand Down
1 change: 1 addition & 0 deletions crates/sequencer/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@ pub mod commit;
pub mod constants;
pub mod execution;
pub mod sequencer;
pub mod serde;
pub mod state;
248 changes: 248 additions & 0 deletions crates/sequencer/src/serde/contract.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,248 @@
// This file has been taken from katana -> https://github.com/dojoengine/dojo/blob/main/crates/katana/core/src/db/serde/contract.rs

use std::collections::HashMap;
use std::sync::Arc;

use serde::{Deserialize, Serialize};
use starknet_api::core::EntryPointSelector;
use starknet_api::deprecated_contract_class::{EntryPoint, EntryPointOffset, EntryPointType};

use super::program::SerializableProgram;

#[derive(Debug, Clone, Serialize, Deserialize)]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you annotate which code is taken from katana? for credit but also to know which code is technically reviewed, tested and works in katana currently

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, completed via this commit.

pub enum SerializableContractClass {
V0(SerializableContractClassV0),
V1(SerializableContractClassV1),
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct SerializableContractClassV0 {
pub program: SerializableProgram,
pub entry_points_by_type: HashMap<EntryPointType, Vec<SerializableEntryPoint>>,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct SerializableContractClassV1 {
pub program: SerializableProgram,
pub entry_points_by_type: HashMap<EntryPointType, Vec<SerializableEntryPointV1>>,
pub hints: HashMap<String, Vec<u8>>,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct SerializableEntryPoint {
pub selector: EntryPointSelector,
pub offset: SerializableEntryPointOffset,
}

impl From<EntryPoint> for SerializableEntryPoint {
fn from(value: EntryPoint) -> Self {
Self {
selector: value.selector,
offset: value.offset.into(),
}
}
}

impl From<SerializableEntryPoint> for EntryPoint {
fn from(value: SerializableEntryPoint) -> Self {
Self {
selector: value.selector,
offset: value.offset.into(),
}
}
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct SerializableEntryPointOffset(pub usize);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has been taken from Katana directly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but can it be changed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is required for the serialization.


impl From<EntryPointOffset> for SerializableEntryPointOffset {
fn from(value: EntryPointOffset) -> Self {
Self(value.0)
}
}

impl From<SerializableEntryPointOffset> for EntryPointOffset {
fn from(value: SerializableEntryPointOffset) -> Self {
Self(value.0)
}
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct SerializableEntryPointV1 {
pub selector: EntryPointSelector,
pub offset: SerializableEntryPointOffset,
pub builtins: Vec<String>,
}

impl From<SerializableEntryPointV1> for blockifier::execution::contract_class::EntryPointV1 {
fn from(value: SerializableEntryPointV1) -> Self {
blockifier::execution::contract_class::EntryPointV1 {
selector: value.selector,
offset: value.offset.into(),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it looks like the only problem is offset!? that sounds cursed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has been taken from Katana directly.

builtins: value.builtins,
}
}
}

impl From<blockifier::execution::contract_class::EntryPointV1> for SerializableEntryPointV1 {
fn from(value: blockifier::execution::contract_class::EntryPointV1) -> Self {
SerializableEntryPointV1 {
selector: value.selector,
offset: value.offset.into(),
builtins: value.builtins,
}
}
}

impl TryFrom<SerializableContractClass> for blockifier::execution::contract_class::ContractClass {
type Error = anyhow::Error;

fn try_from(value: SerializableContractClass) -> Result<Self, Self::Error> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you rework this? You are using a TryFrom trait but I don't see any error being thrown and I see an unwrap in the implementation.

Copy link
Contributor Author

@bajpai244 bajpai244 Sep 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@greged93 this whole file is borrowed from Dojo, LamdbaClass said they are working on serialization for these types, if we still need this file in sometime even after merging this, then might be we can refactor this otherwise we will probably not need these serde types in sometime!

Ok(match value {
SerializableContractClass::V0(v0) => {
blockifier::execution::contract_class::ContractClass::V0(
blockifier::execution::contract_class::ContractClassV0(Arc::new(
blockifier::execution::contract_class::ContractClassV0Inner {
program: v0.program.into(),
entry_points_by_type: v0
.entry_points_by_type
.into_iter()
.map(|(k, v)| (k, v.into_iter().map(|h| h.into()).collect()))
.collect(),
},
)),
)
}
SerializableContractClass::V1(v1) => {
blockifier::execution::contract_class::ContractClass::V1(
blockifier::execution::contract_class::ContractClassV1(Arc::new(
blockifier::execution::contract_class::ContractClassV1Inner {
hints: v1
.hints
.clone()
.into_iter()
.map(|(k, v)| (k, serde_json::from_slice(&v).unwrap()))
.collect(),
program: v1.program.into(),
entry_points_by_type: v1
.entry_points_by_type
.into_iter()
.map(|(k, v)| {
(
k,
v.into_iter()
.map(Into::into)
.collect::<Vec<
blockifier::execution::contract_class::EntryPointV1,
>>(),
)
})
.collect::<HashMap<_, _>>(),
},
)),
)
}
})
}
}

impl From<blockifier::execution::contract_class::ContractClass> for SerializableContractClass {
fn from(value: blockifier::execution::contract_class::ContractClass) -> Self {
match value {
blockifier::execution::contract_class::ContractClass::V0(v0) => {
SerializableContractClass::V0(SerializableContractClassV0 {
program: v0.program.clone().into(),
entry_points_by_type: v0
.entry_points_by_type
.clone()
.into_iter()
.map(|(k, v)| (k, v.into_iter().map(|h| h.into()).collect()))
.collect(),
})
}
blockifier::execution::contract_class::ContractClass::V1(v1) => {
SerializableContractClass::V1(SerializableContractClassV1 {
program: v1.program.clone().into(),
entry_points_by_type: v1
.entry_points_by_type
.clone()
.into_iter()
.map(|(k, v)| {
(
k,
v.into_iter()
.map(Into::into)
.collect::<Vec<SerializableEntryPointV1>>(),
)
})
.collect::<HashMap<_, _>>(),
hints: v1
.hints
.clone()
.into_iter()
.map(|(k, v)| (k, serde_json::to_vec(&v).unwrap()))
.collect(),
})
}
}
}
}

#[cfg(test)]
mod tests {
use blockifier::execution::contract_class::ContractClass;
use starknet::core::types::contract::SierraClass;

use super::*;
use crate::serde::utils::{contract_class, rpc_to_inner_class};

#[test]
fn serialize_and_deserialize_legacy_contract() {
let original_contract = contract_class(include_str!(
"../test_data/contracts/compiled/universal_deployer.json"
));

let serializable_contract: SerializableContractClass = original_contract.clone().into();
assert!(matches!(
serializable_contract,
SerializableContractClass::V0(_)
));

let bytes = serde_json::to_vec(&serializable_contract).unwrap();
let serializable_contract: SerializableContractClass =
serde_json::from_slice(&bytes).unwrap();

let contract: ContractClass = serializable_contract
.try_into()
.expect("should deserialize");
assert_eq!(contract, original_contract);
}

#[test]
fn serialize_and_deserialize_contract() {
let class = serde_json::from_str::<SierraClass>(include_str!(
"../test_data/contracts/compiled/cairo1_contract.json"
))
.expect("should deserialize sierra class")
.flatten()
.expect("should flatten");

let (_, original_contract) =
rpc_to_inner_class(&class).expect("should convert from flattened to contract class");

let serializable_contract: SerializableContractClass = original_contract.clone().into();
assert!(matches!(
serializable_contract,
SerializableContractClass::V1(_)
));

let bytes = serde_json::to_vec(&serializable_contract).unwrap();
let serializable_contract: SerializableContractClass =
serde_json::from_slice(&bytes).unwrap();

let contract: ContractClass = serializable_contract
.try_into()
.expect("should deserialize");
assert_eq!(contract, original_contract);
}
}
4 changes: 4 additions & 0 deletions crates/sequencer/src/serde/mod.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
pub mod contract;
pub mod program;
pub mod state;
pub mod utils;
Loading
Loading