Merge branch 'tronprotocol:develop' into develop

Author: 317787106

README.md

@@ -65,72 +65,70 @@ TRON enables large-scale development and engagement. With over 2000 transactions
 # Building the source
 Building java-tron requires `git` and `Oracle JDK 1.8` to be installed, other JDK versions are not supported yet. Make sure you operate on `Linux` and `MacOS` operating systems.
 
-## Getting the Source Code
+Clone the repo and switch to the `master` branch
 
   ```bash
   $ git clone https://github.com/tronprotocol/java-tron.git
+  $ cd java-tron
   $ git checkout -t origin/master
   ```
-
-## Build
-
+then run the following command to build java-tron, the `FullNode.jar` file can be found in `java-tron/build/libs/` after build successful.
 ```bash
-$ cd java-tron
 $ ./gradlew clean build -x test
 ```
 
-The `FullNode.jar` file can be found in `java-tron/build/libs/FullNode.jar` after build successful.
 
 # Running java-tron
-
-Get the mainnet configurate file: [main_net_config.conf](https://github.com/tronprotocol/tron-deployment/blob/master/main_net_config.conf), other network configuration files can be find [here](https://github.com/tronprotocol/tron-deployment).
-
 Running java-tron requires `Oracle JDK 1.8` to be installed, other JDK versions are not supported yet. Make sure you operate on `Linux` and `MacOS` operating systems.
 
+Get the mainnet configurate file: [main_net_config.conf](https://github.com/tronprotocol/tron-deployment/blob/master/main_net_config.conf), other network configuration files can be find [here](https://github.com/tronprotocol/tron-deployment).
 ## Hardware Requirements
-* Minimum:
-  * CPU with 8 cores
-  * 16GB RAM
-  * 1TB free storage space to sync the Mainnet
-* Recommended:
-  * CPU with 16+ cores(32+ cores for a super representative)
-  * 32+ GB RAM(64+ GB for a super representative)
-  * High Performance SSD with at least 1.5TB free space
-  * 100+ MB/s download Internet service
+Minimum:
+* CPU with 8 cores
+* 16GB RAM
+* 1TB free storage space to sync the Mainnet
+
+Recommended:
+* CPU with 16+ cores(32+ cores for a super representative)
+* 32GB+ RAM(64GB+ for a super representative)
+* High Performance SSD with at least 1.5TB free space
+* 100+ MB/s download Internet service
 
 
 ## Running a full node for mainnet
-  Full node has full historical data, it is the entry point into the TRON network , it can be used by other processes as a gateway into the TRON network via HTTP and GRPC endpoints. You can interact with the TRON network through full node：transfer assets, deploy contracts, interact with contracts and so on. `-c ` parameter specifies a configuration file to run a full node:
+Full node has full historical data, it is the entry point into the TRON network , it can be used by other processes as a gateway into the TRON network via HTTP and GRPC endpoints. You can interact with the TRON network through full node：transfer assets, deploy contracts, interact with contracts and so on. `-c ` parameter specifies a configuration file to run a full node:
    ```bash
    $ nohup java -Xms9G -Xmx9G -XX:ReservedCodeCacheSize=256m \
-    -XX:MetaspaceSize=256m -XX:MaxMetaspaceSize=512m -XX:MaxDirectMemorySize=1G \
-    -XX:+PrintGCDetails -XX:+PrintGCDateStamps  -Xloggc:gc.log \
-    -XX:+UseConcMarkSweepGC -XX:NewRatio=2 \
-    -XX:+CMSScavengeBeforeRemark -XX:+ParallelRefProcEnabled \
-    -XX:+HeapDumpOnOutOfMemoryError \
-    -XX:+UseCMSInitiatingOccupancyOnly  -XX:CMSInitiatingOccupancyFraction=70 \
-    -jar FullNode.jar -c main_net_config.conf >> start.log 2>&1 &
+                -XX:MetaspaceSize=256m -XX:MaxMetaspaceSize=512m \
+                -XX:MaxDirectMemorySize=1G -XX:+PrintGCDetails \
+                -XX:+PrintGCDateStamps  -Xloggc:gc.log \
+                -XX:+UseConcMarkSweepGC -XX:NewRatio=2 \
+                -XX:+CMSScavengeBeforeRemark -XX:+ParallelRefProcEnabled \
+                -XX:+HeapDumpOnOutOfMemoryError \
+                -XX:+UseCMSInitiatingOccupancyOnly  -XX:CMSInitiatingOccupancyFraction=70 \
+                -jar FullNode.jar -c main_net_config.conf >> start.log 2>&1 &
    ```
-## Running a super representative node for mainnet  
-  Adding the `--witness` parameter to the startup command, full node will run as a super representative node. The super representative node supports all the functions of the full node and also supports block production. Before running, make sure you have a super representative account and get votes from others，once the number of obtained votes ranks in the top 27, your super representative node will participate in block production.
+## Running a super representative node for mainnet
+Adding the `--witness` parameter to the startup command, full node will run as a super representative node. The super representative node supports all the functions of the full node and also supports block production. Before running, make sure you have a super representative account and get votes from others，once the number of obtained votes ranks in the top 27, your super representative node will participate in block production.
 
-  Fill in the private key of super representative address into the `localwitness` list in the `main_net_config.conf`, here is an example:
+Fill in the private key of super representative address into the `localwitness` list in the `main_net_config.conf`, here is an example:
    ```
     localwitness = [
         650950B193DDDDB35B6E48912DD28F7AB0E7140C1BFDEFD493348F02295BD812
     ]
    ```
 
-  then run the following command to start the node:
+then run the following command to start the node:
   ```bash
   $ nohup java -Xms9G -Xmx9G -XX:ReservedCodeCacheSize=256m \
-  -XX:MetaspaceSize=256m -XX:MaxMetaspaceSize=512m -XX:MaxDirectMemorySize=1G \
-  -XX:+PrintGCDetails -XX:+PrintGCDateStamps  -Xloggc:gc.log \
-  -XX:+UseConcMarkSweepGC -XX:NewRatio=2 \
-  -XX:+CMSScavengeBeforeRemark -XX:+ParallelRefProcEnabled \
-  -XX:+HeapDumpOnOutOfMemoryError \
-  -XX:+UseCMSInitiatingOccupancyOnly  -XX:CMSInitiatingOccupancyFraction=70 \
-  -jar FullNode.jar --witness -c main_net_config.conf >> start.log 2>&1 &
+               -XX:MetaspaceSize=256m -XX:MaxMetaspaceSize=512m \
+               -XX:MaxDirectMemorySize=1G -XX:+PrintGCDetails \
+               -XX:+PrintGCDateStamps  -Xloggc:gc.log \
+               -XX:+UseConcMarkSweepGC -XX:NewRatio=2 \
+               -XX:+CMSScavengeBeforeRemark -XX:+ParallelRefProcEnabled \
+               -XX:+HeapDumpOnOutOfMemoryError \
+               -XX:+UseCMSInitiatingOccupancyOnly  -XX:CMSInitiatingOccupancyFraction=70 \
+               -jar FullNode.jar --witness -c main_net_config.conf >> start.log 2>&1 &
    ```
 
 ## Quick Start Tool
@@ -143,16 +141,16 @@ Here are some common use cases of the scripting tool
 
 For more details, please refer to the tool [guide](./shell.md).
 
-## Run inside Docker container 
+## Run inside Docker container
 
 One of the quickest ways to get `java-tron` up and running on your machine is by using Docker:
 ```shell
 $ docker run -d --name="java-tron" \
-              -v /your_path/output-directory:/java-tron/output-directory \
-              -v /your_path/logs:/java-tron/logs \
-              -p 8090:8090 -p 18888:18888 -p 50051:50051 \
-              tronprotocol/java-tron \
-              -c /java-tron/config/main_net_config.conf
+             -v /your_path/output-directory:/java-tron/output-directory \
+             -v /your_path/logs:/java-tron/logs \
+             -p 8090:8090 -p 18888:18888 -p 50051:50051 \
+             tronprotocol/java-tron \
+             -c /java-tron/config/main_net_config.conf
 ```
 
 This will mount the `output-directory` and `logs` directories on the host, the docker.sh tool can also be used to simplify the use of docker, see more [here](docker/docker.md).
@@ -165,9 +163,8 @@ This will mount the `output-directory` and `logs` directories on the host, the d
 [tronprotocol/allcoredev](https://gitter.im/tronprotocol/allcoredev) is the official Gitter channel for developers.
 
 # Contribution
-If you'd like to contribute to java-tron, please read the following instructions.
+Thank you for considering to help out with the source code! If you'd like to contribute to java-tron, please see the [Contribution Guide](./CONTRIBUTING.md) for more details.
 
-- [Contribution](./CONTRIBUTING.md)
 
 # Resources
 * [Medium](https://medium.com/@coredevs) java-tron's official technical articles are published there.
@@ -180,3 +177,11 @@ If you'd like to contribute to java-tron, please read the following instructions
 
 # License
 java-tron is released under the [LGPLv3 license](https://github.com/tronprotocol/java-tron/blob/master/LICENSE).
+
+
+
+The minimum hardware requirements are `CPU with 8 cores`,`16GB RAM` and `1TB free storage space` to sync the Mainnet, the recommended hardware requirements:
+* CPU with 16+ cores(32+ cores for a super representative)
+* 32+ GB RAM(64+ GB for a super representative)
+* High Performance SSD with at least 1.5TB free space
+* 100+ MB/s download Internet service

chainbase/src/main/java/org/tron/common/storage/leveldb/LevelDbDataSourceImpl.java

@@ -352,6 +352,7 @@ public Map<byte[], byte[]> getNext(byte[] key, long limit) {
     }
   }
 
+  @Override
   public Map<WrappedByteArray, byte[]> prefixQuery(byte[] key) {
     resetDbLock.readLock().lock();
     try (DBIterator iterator = getDBIterator()) {

chainbase/src/main/java/org/tron/common/storage/rocksdb/RocksDbDataSourceImpl.java

@@ -431,6 +431,7 @@ public Map<byte[], byte[]> getNext(byte[] key, long limit) {
     }
   }
 
+  @Override
   public Map<WrappedByteArray, byte[]> prefixQuery(byte[] key) {
     if (quitIfNotAlive()) {
       return null;

chainbase/src/main/java/org/tron/common/utils/WalletUtil.java

@@ -131,4 +131,12 @@ public static byte[] getSelector(byte[] data) {
     return ret;
   }
 
+  public static Long isLong(String s) {
+    try {
+      return Long.parseLong(s);
+    } catch (NumberFormatException ignored) {
+      return null;
+    }
+  }
+
 }

chainbase/src/main/java/org/tron/core/db/TronDatabase.java

@@ -110,13 +110,7 @@ public T getUnchecked(byte[] key) {
   }
 
   public Map<WrappedByteArray, byte[]> prefixQuery(byte[] key) {
-    Map<WrappedByteArray, byte[]> result = new HashMap<>();
-    if (dbSource.getClass() == LevelDbDataSourceImpl.class) {
-      result = ((LevelDbDataSourceImpl) dbSource).prefixQuery(key);
-    } else if (dbSource.getClass() == RocksDbDataSourceImpl.class) {
-      result = ((RocksDbDataSourceImpl) dbSource).prefixQuery(key);
-    }
-    return result;
+    return dbSource.prefixQuery(key);
   }
 
   public abstract boolean has(byte[] key);

chainbase/src/main/java/org/tron/core/db/common/DbSourceInter.java

@@ -46,4 +46,6 @@ public interface DbSourceInter<V> extends BatchSourceInter<byte[], V>,
 
   void stat();
 
+  Map<WrappedByteArray, byte[]> prefixQuery(byte[] key);
+
 }

chainbase/src/main/java/org/tron/core/db2/core/Chainbase.java

@@ -12,8 +12,8 @@
 import java.util.Set;
 import java.util.stream.Collectors;
 
-import com.google.common.primitives.Bytes;
 import org.tron.common.utils.ByteUtil;
+import org.tron.common.utils.Pair;
 import org.tron.core.capsule.utils.MarketUtils;
 import org.tron.core.db2.common.IRevokingDB;
 import org.tron.core.db2.common.LevelDB;
@@ -350,13 +350,14 @@ private Map<byte[], byte[]> getNext(Snapshot head, byte[] key, long limit) {
   }
 
   public Map<WrappedByteArray, byte[]> prefixQuery(byte[] key) {
-    Map<WrappedByteArray, byte[]> resultSnapshot = prefixQuerySnapshot(key);
-    Map<WrappedByteArray, byte[]> resultDB = prefixQueryDB(key);
-    resultDB.putAll(resultSnapshot);
-    return resultDB;
+    Map<WrappedByteArray, byte[]> result = prefixQueryRoot(key);
+    Map<WrappedByteArray, byte[]>  snapshot = prefixQuerySnapshot(key);
+    result.putAll(snapshot);
+    result.entrySet().removeIf(e -> e.getValue() == null);
+    return result;
   }
 
-  private Map<WrappedByteArray, byte[]> prefixQueryDB(byte[] key) {
+  private Map<WrappedByteArray, byte[]> prefixQueryRoot(byte[] key) {
     Map<WrappedByteArray, byte[]> result = new HashMap<>();
     if (((SnapshotRoot) head.getRoot()).db.getClass() == LevelDB.class) {
       result = ((LevelDB) ((SnapshotRoot) head.getRoot()).db).getDb().prefixQuery(key);
@@ -369,16 +370,10 @@ private Map<WrappedByteArray, byte[]> prefixQueryDB(byte[] key) {
   private Map<WrappedByteArray, byte[]> prefixQuerySnapshot(byte[] key) {
     Map<WrappedByteArray, byte[]> result = new HashMap<>();
     Snapshot snapshot = head();
-    while (!snapshot.equals(head.getRoot())) {
-      Iterator<Map.Entry<byte[], byte[]>> iterator = snapshot.iterator();
-      while (iterator.hasNext()) {
-        Map.Entry<byte[], byte[]> entry = iterator.next();
-        WrappedByteArray ks = WrappedByteArray.of(entry.getKey());
-        if (Bytes.indexOf(entry.getKey(), key) == 0 && !result.containsKey(ks)) {
-          result.put(ks, entry.getValue());
-        }
-      }
-      snapshot = snapshot.getPrevious();
+    if (!snapshot.equals(head.getRoot())) {
+      Map<WrappedByteArray, WrappedByteArray> all = new HashMap<>();
+      ((SnapshotImpl) snapshot).collect(all, key);
+      all.forEach((k, v) -> result.put(k, v.getBytes()));
     }
     return result;
   }

chainbase/src/main/java/org/tron/core/db2/core/SnapshotImpl.java

@@ -4,10 +4,12 @@
 import com.google.common.collect.Iterators;
 import com.google.common.collect.Maps;
 import com.google.common.collect.Streams;
+import com.google.common.primitives.Bytes;
 import java.util.HashMap;
 import java.util.HashSet;
 import java.util.Iterator;
 import java.util.Map;
+import java.util.Objects;
 import java.util.Set;
 import lombok.Getter;
 import org.tron.core.db2.common.HashDB;
@@ -129,6 +131,17 @@ synchronized void collect(Map<WrappedByteArray, WrappedByteArray> all) {
     }
   }
 
+  synchronized void collect(Map<WrappedByteArray, WrappedByteArray> all, byte[] prefix) {
+    Snapshot next = getRoot().getNext();
+    while (next != null) {
+      Streams.stream(((SnapshotImpl) next).db).filter(e -> Bytes.indexOf(
+              Objects.requireNonNull(e.getKey().getBytes()), prefix) == 0)
+          .forEach(e -> all.put(WrappedByteArray.of(e.getKey().getBytes()),
+              WrappedByteArray.of(e.getValue().getBytes())));
+      next = next.getNext();
+    }
+  }
+
   /**
    * Note: old --> new
    * In the snapshot, there may be same keys.

framework/build.gradle

@@ -1,5 +1,6 @@
 plugins {
     id "org.sonarqube" version "2.6"
+    id "com.gorylenko.gradle-git-properties" version "2.4.1"
 }
 
 apply plugin: 'application'

framework/src/main/java/org/tron/common/overlay/server/Channel.java

@@ -61,6 +61,7 @@ public class Channel {
   private InetSocketAddress inetSocketAddress;
   private Node node;
   private long startTime;
+  @Getter
   private TronState tronState = TronState.INIT;
   private boolean isActive;
   @Getter