pseudoyu

Introduction#

Because the graduation Case Study project is mainly based on the Ethereum public chain and does not target enterprise application scenarios, my previous understanding of Hyperledger Fabric was mostly limited to its permission management mechanism, channels, and flexible smart contract writing. I had only a superficial understanding of its architecture, the roles of various nodes, and its operating mechanisms. Recently, while taking the HKU course <FITE3011 Distributed Ledger and Blockchain>, the professor provided a detailed explanation of the working principles of Hyperledger Fabric, network setup, and chaincode-related knowledge, which was very beneficial. This article aims to summarize these insights, and I welcome any corrections or discussions.

Overview of Hyperledger#

To learn about Hyperledger Fabric, let's first take a look at its parent project, Hyperledger.

Enterprise applications have complex business logic and participant role divisions, with high demands for business execution efficiency and security. Additionally, for common scenarios such as payments and data/information transactions, privacy protection is also a top priority. Therefore, common public chains like Bitcoin and Ethereum do not meet the needs of most enterprise applications. However, the distributed and tamper-proof historical ledger characteristics of blockchain can avoid complex operational processes caused by varying laws, regulations, and currencies in different countries/regions in scenarios like traceability and cross-border e-commerce, greatly improving efficiency. Thus, enterprise consortium chains are continuously evolving.

In a strict sense, consortium chains are not truly "decentralized." They introduce a permission management mechanism (combined with the roles of enterprises in real business) to weaken the preventive measures against node malfeasance, thereby improving efficiency and addressing complex business logic.

Among them, Hyperledger is a set of open-source projects focused on cross-industry distributed technologies maintained by the Linux Foundation, aimed at creating enterprise-grade, open-source, distributed classification frameworks and codebases to support business use cases, providing neutral, open, and community-driven infrastructure; establishing a technical community and promoting the development of blockchain and shared ledger proof of concepts, use cases, experiments, and deployments; establishing industry standards to encourage more enterprises to participate in the construction and application of distributed ledger technology, forming an open ecosystem; and educating the public about market opportunities in blockchain technology.

Design Philosophy#

Hyperledger has several core design philosophies:

1. It enhances efficiency for specific business scenarios of enterprises and has unique advantages in traceability and other scenarios. Each enterprise can maintain its own independent Hyperledger project tailored to its scenarios, thus it does not need to incentivize user participation in the blockchain system through digital currency like public chains.
2. Enterprise application scenarios are relatively complex, and often Hyperledger only participates in certain segments, making interaction with other existing systems essential. Therefore, Hyperledger emphasizes equipping complete APIs for other systems to call and interact with.
3. The framework structure of Hyperledger is modular and extensible, allowing enterprises to choose different modules based on specific business needs, avoiding complex business logic and bloated systems.
4. The security of enterprise applications is paramount, especially since many application scenarios involve high-value transactions or sensitive data. Therefore, it provides many mechanisms to ensure security (such as the channel mechanism in Fabric).
5. In addition to interacting with existing systems, future blockchain applications for enterprises may also interact with many different blockchain networks. Therefore, most smart contracts/applications should possess cross-blockchain network portability to form more complex and powerful networks.

Framework#

Under Hyperledger, there are several projects, among which Fabric is currently the most widely used, and this article will mainly introduce the Fabric blockchain network:

• Burrow
• Fabric
• Grid
• Indy
• Iroha
• Sawtooth

Tools#

1. Hyperledger Cello. Mainly used to facilitate the setup and management of blockchain services, reducing the complexity of project framework deployment and maintenance; can be used to build blockchain BaaS platforms; allows for the creation and management of blockchains through a Dashboard, making it easier for technical personnel to develop and deploy; can introduce the SaaS deployment model into blockchain systems, helping enterprises further develop frameworks.
2. Hyperledger Explorer. A visual blockchain operation tool that can be used to create user-friendly web applications; it is the first Hyperledger blockchain explorer, allowing users to view/call/deploy/query transactions, networks, smart contracts, storage, and other information.

Hyperledger Fabric#

We will focus on the most widely used Fabric project, which is a modular and extensible blockchain consortium chain project maintained by the Linux Foundation, not relying on any cryptocurrency. It provides protection for business transactions between entities with common goals (business needs) but incomplete information, such as cross-border e-commerce, fund transactions, and traceability.

Architecture#

In most public chains, the architecture is Order - Execute - Validate - Update State. For example, in the Bitcoin blockchain, when a new transaction occurs, it first uses the PoW mechanism to sort the block, then each node in the Bitcoin network verifies it one by one, and finally updates the state. Because verification must be done sequentially, this method results in relatively low execution efficiency.

In contrast, Fabric adopts the Execute - Order - Validate - Update State architecture.

Upon receiving a new transaction, it is first submitted to endorsing nodes for local simulation of transaction execution (and endorsement), then the endorsed transactions are sorted and broadcasted, and each node verifies the transactions before updating the state.

As mentioned in the characteristics of consortium chains, joining the Fabric network requires permission (identity verification), and each node in the Fabric network has its own identity.

In summary, Fabric supports complex business scenarios for enterprises through a modular and pluggable architecture, weakens node malfeasance through identity verification (binding to real identities), and greatly enhances system security and privacy protection through the channel mechanism.

MSP Membership Service Provider#

So, how is the identity of participants in the Fabric network managed?

Fabric has an MSP (Membership Service Provider) that mainly manages CA certificates to verify which members are trusted. The Fabric CA module is independent and can manage certificate services as well as allow third-party CA access, greatly expanding the system's application scope.

As shown in the figure above, Fabric CA provides two ways to interact with CA: client and SDK. Each Fabric CA has a root CA or intermediate CA. To further enhance the security of CA, clusters can be used to set up intermediate CAs.

Looking more specifically at the CA hierarchy, it generally adopts a three-tier tree structure of root CA, business CA, and user CA, where all lower-level CAs inherit the trust system of the upper-level CA. The root CA is used to issue business CAs, while business CAs are used to issue specific user CAs (identity authentication CA, transaction signing, secure communication CA, etc.).

Channels#

As mentioned earlier, Fabric uses the channel mechanism to ensure the security and privacy of transactions. Essentially, each channel is an independent ledger and also an independent blockchain with different world states. A node in the network can join multiple channels simultaneously. This mechanism effectively delineates different business scenarios without worrying about transaction information leakage.

Chaincode#

Fabric also has smart contracts similar to Ethereum, called Chaincode, which allows external applications to interact with the ledger in the Fabric network. Unlike Ethereum, Fabric uses Docker instead of a specific virtual machine to store chaincode, providing a secure and lightweight language execution environment.

Chaincode is mainly divided into system chaincode and user chaincode. System chaincode is embedded in the system and provides support for configuring and managing the system, while user chaincode runs in separate Docker containers and provides support for upper-layer applications. Users can write user chaincode through chaincode-related APIs to update the state in the ledger.

After installation and instantiation, chaincode can be invoked. During installation, it is necessary to specify which Peer node to install it on (some nodes may not have chaincode), and during instantiation, the channel and endorsement policy must also be specified.

Chaincode can also call each other, creating more flexible application logic.

Consensus Mechanism#

The broad consensus mechanism in Fabric includes endorsement, ordering, and validation, while the narrow consensus refers to ordering.

In the Fabric blockchain network, transactions between different participants must be written to the distributed ledger in the order they occur, relying on the consensus mechanism, which mainly includes three types:

• SOLO (limited to development)
• Kafka (a messaging platform)
• Raft (more centralized compared to Kafka)

Network Protocol#

So how is the state distribution among various nodes in the Fabric network conducted?

External clients make remote calls to various nodes in the Fabric network through gRPC, while synchronization among nodes in the P2P network is conducted via the Gossip protocol.

The Gossip protocol is mainly used for data exchange among multiple nodes in the network, which is relatively easy to implement and has a high fault tolerance. The principle is that the data sender randomly selects several nodes from the network to send the data to. Once a few nodes receive this data, they randomly send it to several other nodes except the sender, repeating this process until all nodes reach consensus (complexity is LogN).

Distributed Ledger#

Ultimately, all transactions will be recorded in the distributed ledger, which is also a core characteristic of blockchain. In Fabric, transactions can store relevant business information, and a block is a collection of ordered transactions. Linking blocks together through cryptographic algorithms forms the blockchain. The distributed ledger mainly records the world state (the latest state of the distributed ledger, generally using CouchDB for easy querying) and transaction logs (the historical updates of the world state, recording the blockchain structure, using LevelDB). Every operation on the ledger is recorded in the logs and is tamper-proof.

Application Programming Interface#

For applications based on Fabric, two main interaction methods are provided: SDK development kits and CLI command lines.

Core Roles in Fabric Blockchain#

First, it should be noted that the roles in the Fabric network are logical roles. For example, Peer node A may be both an ordering node and an endorsing node in certain business scenarios, and a single role may not be held by a single node.

Next, let's introduce the functions and responsibilities of each role.

Clients mainly sign transactions, submit transaction proposals to endorsing nodes, and receive endorsed transactions to broadcast to ordering nodes; endorsing nodes locally simulate the execution of transaction proposals to verify transactions (the strategy is determined by Chaincode), sign them, and return the endorsed transactions; ordering nodes package transactions into blocks and broadcast them to all nodes, not participating in transaction execution and verification. Multiple ordering nodes can form an OSN; all nodes maintain the blockchain ledger.

Summary of Advantages#

Fabric enhances efficiency and security by distributing various complex aspects of enterprise applications to different logical role nodes (endorsement, ordering, etc.), eliminating the need for all nodes to undertake resource-intensive operations like ordering, thus removing network bottlenecks. After role allocation, certain transactions are only deployed and executed on specific nodes, allowing for concurrent execution, significantly improving efficiency and security while concealing some business logic. Therefore, various flexible allocation schemes can be formed based on different business needs, greatly enhancing the system's scalability.

By making the consensus mechanism, permission management, encryption mechanism, ledger, and other modules pluggable, and allowing different chaincodes to set different endorsement strategies, the trust mechanism becomes more flexible, enabling the establishment of an efficient system tailored to business needs.

The membership identity management of Fabric CA, as a separate project, can provide more functionalities and can directly interact with many third-party CAs, making it more powerful and suitable for complex enterprise scenarios.

The multi-channel feature isolates data between different channels, enhancing security and privacy protection.

Chaincode supports various programming languages such as Java, Go, and Node, making it more flexible and supporting more third-party extension applications, thereby reducing business migration and maintenance costs.

Fabric Application Development and Interaction#

The above diagram illustrates the development and interaction process of a blockchain developer using the Fabric blockchain.

Developers are mainly responsible for developing applications and smart contracts (chaincode). Applications interact with smart contracts through the SDK, and the logic of smart contracts can perform get, put, delete, and other operations on the ledger.

Fabric Workflow#

Next, let's clarify the working principles of the Fabric network through a complete transaction flow.

0. Before all operations, it is necessary to obtain a legitimate identity from CA and specify the channel.
1. First, the Client submits a transaction proposal (containing its signature) to the endorsing node.
2. After receiving the transaction proposal, the endorsing node simulates execution using the local state, endorses the transaction, signs it, and returns it (including Read-Write Set, signatures, etc.).
3. After the Client collects enough endorsements (the strategy is determined by Chaincode, for example, obtaining 2 endorsements as shown in the figure), it submits the endorsed transaction to the ordering node (OSN).
4. The ordering node packages the transactions into blocks, sorts them (without executing or verifying the correctness of the transactions), and broadcasts them to all nodes.
5. All nodes verify the new blocks and submit them to the ledger.

Next, let's break down each step in detail.

Execution/Endorsement Phase#

After the Client submits the transaction proposal, the endorsing node first verifies the Client's signature, simulates execution using the local state, and returns the signed Read-Write Set to the Clients. The R-W Sets mainly contain three attributes: key, version, and value. The Read-Set includes all variables read during transaction execution and their version. If a write operation is performed on the ledger, the version will change. The Write-Set includes all edited variables and their new values.

When executing the transaction, the endorsing node checks whether the chaincode is correct based on the local blockchain state and executes it, returning the result.

Fabric supports various endorsement strategies. Before submitting to the ordering node, the Client verifies whether the endorsement requirements are met. It is worth noting that if only a query operation on the ledger is performed, the Client will not submit it to the OSN.

The transaction proposal mentioned earlier mainly includes the chaincode, input values for the chaincode, and the Client's signature, while the information returned to the Client by the endorsing node includes the return value, the R-W Set from the simulated execution, and the signature of the endorsing node, collectively forming the endorsed transaction.

Endorsement is the recognition of the transaction by relevant organizations, meaning that the relevant nodes sign the transaction. For a chaincode transaction, the endorsement strategy is specified when the chaincode is instantiated. A valid transaction must be signed by the organizations specified in the endorsement strategy to take effect. Essentially, transaction verification in the Fabric blockchain is based on trust in the endorsing nodes, which is one reason why Fabric is not strictly decentralized.

Here is a simple example of chaincode execution:

func (t *SimpleChaincode) InitLedger(ctx contractapi.TransactionContextInterface) error {
    var product = Product { Name: "Test Product", Description: "Just a test product to make sure chaincode is running", CreatedBy: "admin", ProductId: "1" }

    productAsBytes, err := json.Marshal(product)

    err = ctx.GetStub().PutState("1", productAsBytes)

    if err != nil {
        return err
    }
}

In this simple example, the main operation of the chaincode is to update the key-value pair, and after this operation, the version will change.

The R-W Set returned after execution is:

key: 1
value: JSON representation of Product { Name: "Test Product", Description: "Just a test product to make sure chaincode is running", CreatedBy: "admin", ProductId: "1" }

Ordering Phase#

The Client submits the endorsed transaction to the ordering node (the ordering node can be composed of several consensus strategies to form an OSN). After receiving the transaction, the ordering node packages it into blocks and sorts them according to the rules specified in the configuration. During this process, only the ordering operation is executed, without any execution or verification. Once sorting is complete, it sends the blocks to all nodes.

The ordering service is responsible for achieving consensus on transactions across the network, only ensuring agreement on the order of transactions, which avoids bottlenecks in the entire network and makes it easier to scale horizontally to improve network efficiency. Currently, it supports Kafka and Raft. The consistency of the Fabric blockchain network relies on the consistency of the ordering nodes.

The Raft consensus mechanism is a non-Byzantine consensus mechanism that uses a leader and follower model. When a leader is elected, log information is copied unidirectionally from the leader to the followers, making it easier to manage. The design allows all nodes to be considered ordering nodes, making it more centralized compared to Kafka. It also allows for the use of PBFT consensus mechanisms, but performance is often poor.

Validation Phase#

When a node receives a block sent by the ordering node, it verifies all transactions within the block and marks them as trustworthy or not. The main aspects of verification are: 1. whether the endorsement strategy is met; 2. the legality of the transaction structure, such as whether there are state conflicts, e.g., whether the version in the Read-Set is consistent.

Conclusion#

The above is a summary of the architecture of Hyperledger Fabric. Although it sacrifices some decentralization principles, as an open-source consortium chain aimed at enterprise applications, it encourages more enterprises to participate in the construction and application of distributed ledger technology. Many self-developed consortium chain platforms, such as Ant Chain and Quchain, have emerged in China, and I believe more enterprises will join this open ecosystem in the future!

References#

1. FITE3011 Distributed Ledger and Blockchain, Allen Au, HKU
2. Enterprise-level Blockchain Practical Tutorial, Zhang Yingping