Testing a Blockchain application (implemented as Public or Private) requires non-traditional testing approach. Magic’s test methodology for Blockchain applications heavily relies on DevOps expertise to drive automation for testing Blockchain Infrastructure. In addition, Magic brings functional experts who bring a strong knowledge of the domain to drive functional testing of the Smart Contracts.
Magic’s methodology looks at three broad categories for devising an appropriate solution.
a. P2P network establishment along with support for scaling & controlling volatility
b. Consensus algorithms to not only ensure transaction legitimacy but also provide fault tolerance for the network (e.g. RAFT, PBFT, PAXOS & many more derived from these algorithms etc)
c. Data & transaction encryption
d. Fault tolerance of multiple customer infrastructure participating in a blockchain network
e. Access control for synchronization of information to authorized nodes
f. Data syncing across nodes to ensure states are maintained based on the consensus algorithm in different disaster scenarios - Complete network failure, Individual node failure, Specific service failure on specific node etc. etc
a. Infrastructure
b. Performance
c. Security (Network security not Encryption)
d. Integration
e. Many more – Our website can be referred for more specific detail
a. Verification of Customer specific requirement for a specific domain
b. Verification of Smart Contract business logic
c. Security & Audit
Each of the above-mentioned categories has many more functionalities. Learn More
1. Create Test scenarios bank for each function defined above.
2. Demarcate all the scenarios based on the level of automation that can be achieved. Automation alone is both impossible and insufficient and our belief is that automation combined with Explorative Testing is the best approach to provide high test coverage.
3. Build generic Automation framework for all the scenarios, chosen for automation. We have taken a modular approach to building a module for each function within each category. Each module has its own setup and is driven by its own config file so it is very easy to use any specific module without triggering the entire automation framework.
Our automation framework operates in multiple ways:
1. Interactive Mode (UI based)
2. Batch Mode
3. Each Module comes up with the flexibility to schedule (In progress)
Here is a list of salient automation modules that we have developed for some of our clients.
Based on required network topology which can differ from customer to customer, our UI based solution takes up a config file that is used for defining the number of nodes within a network. The following diagram effectively explains the flow:
Once Network is setup, MagicVolatilityModule randomly kills services (keeping a handful of services alive based on the consensus algorithm) to observe the behavior of the network. In addition, it verifies logging of all expected messages followed by sending an adequate message of its unavailability to their peers. This is followed by restarting the service to verify whether the data is being synchronized from the moment it was killed till the current state of the network. This is repeated by killing a varying number of nodes in the network.
With time, there is a need to add new nodes in the network. Out automated solution spins off new nodes ready to network to its peers. New services are installed and data synchronization is verified based on defined access controls. It also verifies if the new node is able to participate in the consensus
Developed python based solution performs load/stress testing on the network to verify the throughput of the network. MagicPerformanceModule provides monitoring for all performance runs through Prometheus & Grafana. Performance test runs are continuous and this helps in comparing the performance amongst various builds. We have recently added monitoring for Docker containers and have provided more granularity for tracking resource consumption at application/service level.
MagicBlockchainQA has built an automated solution for RAFT based consensus algorithm using leader based protocol. Our approach is to send messages over the network which are routed to all nodes via the leader. Each node sends out the vote based on their current state and the Leader collects the vote from each node and takes appropriate actions.
Our module is currently capable of verifying Smart contract built on Solidity on Ethereum platform. However, this module can be extended to other blockchain development platforms based on specific business needs. Smart Contract business logic is very specific to a customer and its business model. Once we understand underlying platform on which smart contract is to be developed we can quickly design & develop automation framework for following languages - Solidity (Requires existing module tweak based on customer specific requirement), Python, Java & JavaScript.
Our Automated solution is designed to verify data from any RDBMS as well as key-value storage LevelDB, RocksDB & BadgerDB. However, Key-value storage verification is currently only limited to these 3 DBs. Within Private Blockchain, implementation data needs to be synchronized based on the access given to each node. Config driven MagicDBAnalyzerModule module reads config file for each node to check access. It then verifies data for all the nodes from the node with full chain data.
MagicBlockchainQA team is now focused on developing a comprehensive network security module to detect any vulnerabilities within the network and all running applications and services. In addition, we are working to make MagicSmartContModule a bit more generic as existing implementation supports verification for Smart contract written in Solidity on Ethereum. Our journey will continue to enrich various other modules based on the market need and recent technological advances. Our vision is to make this framework easy to use by the testing community.
We are open to suggestions & feedback to take this to the next level of maturity where this can help the industry to seamlessly test a Blockchain application.
Security Testing
Platform Partner
