Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature derby raft #2

Open
wants to merge 80 commits into
base: develop
Choose a base branch
from
Open
Changes from all commits
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
6a2ce0c
feat: Create a lifecycle interface for the management of related reso…
chuntaojun Jan 13, 2020
870b2b2
fix: fix LifeCycleUtils method should throw exception
chuntaojun Jan 13, 2020
7264d04
refactor: Adjust the thread pool resource management implementation
chuntaojun Jan 13, 2020
0daf2cd
refactor: Code rule optimization
chuntaojun Jan 13, 2020
662f522
Merge pull request #1 from chuntaojun/feature/life_cycle
chuntaojun Jan 13, 2020
be44209
Merge branch 'develop' of https://github.com/alibaba/nacos into develop
chuntaojun Jan 20, 2020
e76def8
(refactor) refactor raft protocol
chuntaojun Jan 26, 2020
80bda0b
(fix) fix JRaftServer can't listener node change event
chuntaojun Jan 26, 2020
8e48b1b
refactor(core): implement distributed lock, adjust Raft protocol impl…
chuntaojun Jan 27, 2020
709e548
refactor: core
chuntaojun Jan 27, 2020
e24942d
refactor(config): v2.0存储
chuntaojun Jan 31, 2020
b640449
fix(core): fix NodeManager initialization resource order
chuntaojun Feb 1, 2020
62c0382
refactor: config
chuntaojun Feb 2, 2020
c239a60
feat(config): support derby do snapshot
chuntaojun Feb 3, 2020
36a8b0b
fix: config
chuntaojun Feb 5, 2020
bd12425
fix(config): embedded distributed database: publishing configuration …
chuntaojun Feb 5, 2020
87d4594
feat(core): initially simple sink the AP protocol to the Core module
chuntaojun Feb 6, 2020
a615363
feat(core): storage module abstraction
chuntaojun Feb 6, 2020
9ba6b73
fix(core): fix the problem that protocolMetaData returns data in JRaf…
chuntaojun Feb 6, 2020
6afcf0f
fix(core): protocolMetaData return copy
chuntaojun Feb 6, 2020
97aef2f
feat(core): storage interface abstraction
chuntaojun Feb 6, 2020
1d59055
refactor(core): jRaft state machine optimization, Log processing is p…
chuntaojun Feb 7, 2020
efb09f7
refactor(core): none
chuntaojun Feb 7, 2020
0a27866
refactor(consistency): consistency protocol layer re-abstracted, LogC…
chuntaojun Feb 8, 2020
f65e42c
refactor(core): fix jraft use
chuntaojun Feb 8, 2020
19d19a8
refactor(consistency): tuning Consistency Protocol Abstract Interface
chuntaojun Feb 8, 2020
4554f00
fix(core): commit should be taken when the request is forwarded
chuntaojun Feb 8, 2020
d8873b4
refactor(core): jRaft optimization
chuntaojun Feb 9, 2020
a7aad1b
refactor(core): none
chuntaojun Feb 9, 2020
e5c0d9d
refactor(core): jraft use adjustment
chuntaojun Feb 10, 2020
9ee47c3
refactor(consistency): modify the consistency protocol interface defi…
chuntaojun Feb 12, 2020
1a02fbc
refactor(consistency): protocolMetaData internal logic optimization
chuntaojun Feb 12, 2020
2acab44
none
chuntaojun Feb 14, 2020
d0f7f71
refactor(core): the distro protocol sinks to the core module
chuntaojun Feb 15, 2020
109ba65
refactor(core): distro 一致性下沉到core模块完成
chuntaojun Feb 16, 2020
ba05a61
refactor(core): the ap protocol specifies the implementation of the M…
chuntaojun Feb 16, 2020
198adf6
refactor(consistency): mapper and ProtocolMetaData modify
chuntaojun Feb 16, 2020
e9613c6
refactor(consistenct): modify GetRequest
chuntaojun Feb 16, 2020
5aa8b26
refactor(core): raftKVStore finished
chuntaojun Feb 16, 2020
51a23ec
refactor(core): cluster management implementation
chuntaojun Feb 17, 2020
f8241b5
refactor(config): the config module has embedded distributed storage …
chuntaojun Feb 19, 2020
8366ae2
feat(config): embedded distributed storage is complete
chuntaojun Feb 20, 2020
0d92ae4
feat(core): add invalid node clear logic
chuntaojun Feb 20, 2020
b64b3e6
refactor(core): partial code adjustments
chuntaojun Feb 21, 2020
55adcdc
fix(core): fix bugs
chuntaojun Feb 21, 2020
2b99a0d
refactor(core): code refactor
chuntaojun Feb 21, 2020
5d6e07c
refactor(core): adjust code
chuntaojun Feb 22, 2020
251bde6
refactor(naming): naming module data adjustment
chuntaojun Feb 22, 2020
94f27e5
feat(config): derby snapshot mechanism implementation
chuntaojun Feb 22, 2020
a21a3cc
style: fix code style
chuntaojun Feb 22, 2020
225648a
feat(core): Raft version Meituan Leaf implementation
chuntaojun Feb 24, 2020
4843913
add naming remove event #2378
yuanzhixiang Feb 27, 2020
5dbc8e9
Remove unnecessary comments #2378
yuanzhixiang Feb 27, 2020
3d62725
refactor(core): Data recovery related to server startup needs to comp…
chuntaojun Feb 27, 2020
9c9542e
Fix: #2378 add copyright notice
yuanzhixiang Feb 28, 2020
c189c6e
refactor(core): cluster.conf解析调整,配置格式调整为URL
chuntaojun Feb 28, 2020
8d18d8d
refactor(core): 添加Raft默认端口信息
chuntaojun Feb 29, 2020
7ae8a88
fix: 修复方法调用问题
chuntaojun Feb 29, 2020
0931643
Fix: #2378 add unit test
yuanzhixiang Mar 1, 2020
5edbb35
Fix: #2378 remove log file
yuanzhixiang Mar 1, 2020
12b36e1
Merge pull request #2391 from zhixiangyuan/develop_1.2.0_fix#2378
chuntaojun Mar 2, 2020
606ec4b
Update config auth test cases
Mar 2, 2020
b5a3ad3
refactor(Upstream code synchronization):
chuntaojun Mar 3, 2020
db42154
fix(core): Fix the DefaultIdGenerator error
chuntaojun Mar 5, 2020
2020a3a
refactor(core): logic optimization
chuntaojun Mar 7, 2020
6ddd8d0
refactor: DistroMapper remove from consistency
chuntaojun Mar 8, 2020
513a504
Merge branch 'develop' of https://github.com/alibaba/nacos into featu…
chuntaojun Mar 8, 2020
ec02ff4
fix: bug fix
chuntaojun Mar 8, 2020
2e24bae
refactor: merge upstream develop
chuntaojun Mar 9, 2020
025a4d4
fix(all): Fix snapshot load, save errors
chuntaojun Mar 9, 2020
d5c4713
fix(config): derby snapshot bug
chuntaojun Mar 10, 2020
d0b29b4
refactor: Derby snapshot recovery adjustment
chuntaojun Mar 11, 2020
c2d9778
refactor(core): The thread pool closes gracefully and the protocol cl…
chuntaojun Mar 12, 2020
756064f
refactor(core): Cluster synchronization optimization
chuntaojun Mar 13, 2020
e039a62
perf(common): Optimize the HttpClient resource application
chuntaojun Mar 14, 2020
d639fd6
feat(all): Function is basically determined
chuntaojun Mar 16, 2020
0451475
style: Repair code style
chuntaojun Mar 17, 2020
f5f7d3f
fix(naming): fix kvStore bug
chuntaojun Mar 18, 2020
421ad22
fix(all): memger upstream develop
chuntaojun Mar 19, 2020
dd2a500
refactor(all): Overall optimization, reduce serialization times
chuntaojun Mar 22, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -11,5 +11,7 @@ target
*.iml
node_modules
test/derby.log
test/log
test/work
derby.log
work
4 changes: 2 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -177,7 +177,7 @@
* Fully integrate with and seamlessly support Spring framework, Spring Boot and Spring Cloud
* Separate nacos-api from nacos client implementation
* Support high available cluster mode
* Fix cluster node health check abnormality
* Fix cluster member health check abnormality
* Fix stand-alone mode gets the change history list exception
* Fix Pulling does not exist configuration print io exception
* Optimized log framework
@@ -190,7 +190,7 @@
#### FEATURES:

* separate nacos-api from nacos client implementation
* Cluster node health check abnormality
* Cluster member health check abnormality
* Stand-alone mode gets the change history list exception
* Pulling does not exist configuration print io exception
* Optimized log framework
91 changes: 91 additions & 0 deletions NEW_FEATURE_USE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
#### 命令行参数

```shell script
-Dnacos.serializer-type=kryo
# -Dnacos.serializer-type=protostuff
```

#### application.properties

```properties
#*************** Core Related Configurations ***************#

### Whether to turn on inter-member discovery, If this configuration is enabled, the cluster.conf configuration
### of the newly added member requires that a member of the known cluster be added to build the discovery channel
nacos.core.member.self-discovery=true

### Which nacos embedded distributed ID is turned on,
### If an external implementation is provided, the external implementation is automatically selected
nacos.core.idGenerator.type=default
### The step size for each fetch of the embedded distributed ID
nacos.core.idGenerator.default.acquire.step=100

### If nacos.core.idGenerator.type=snakeflower, You need to set the dataCenterID manually
#nacos.core.snowflake.data-center=
### If nacos.core.idGenerator.type=snakeflower, You need to set the WorkerID manually
#nacos.core.snowflake.worker-id=

#*************** Embed Storage Related Configurations ***************#

### Whether to open embedded distributed storage in nacos cluster mode
embeddedDistributedStorage=true

#*************** Consistency Related Configurations ***************#

# About Raft

### Sets the Raft cluster election timeout, default value is 5 second
nacos.core.protocol.raft.data.election_timeout_ms=5000
### Sets the amount of time the Raft snapshot will execute periodically, default is 30 minute
nacos.core.protocol.raft.data.snapshot_interval_secs=30
### Requested retries, default value is 1
nacos.core.protocol.raft.data.request_failoverRetries=1
### raft internal worker threads
nacos.core.protocol.raft.data.core_thread_num=8
### Number of threads required for raft business request processing
nacos.core.protocol.raft.data.cli_service_thread_num=4
### raft linear read strategy, defaults to index
nacos.core.protocol.raft.data.read_index_type=ReadOnlySafe
### rpc request timeout, default 5 seconds
nacos.core.protocol.raft.data.rpc_request_timeout_ms=5000
### Maximum size of each file RPC (snapshot copy) request between members, default is 128 K
nacos.core.protocol.raft.data.max_byte_count_per_rpc=131072
### Maximum number of logs sent from leader to follower, default is 1024
nacos.core.protocol.raft.data.max_entries_size=1024
### Maximum body size for sending logs from leader to follower, default is 512K
nacos.core.protocol.raft.data.max_body_size=524288
### Maximum log storage buffer size, default 256K
nacos.core.protocol.raft.data.max_append_buffer_size=262144
### Election timer interval will be a random maximum outside the specified time, default is 1 second
nacos.core.protocol.raft.data.max_election_delay_ms=1000
### Specify the ratio between election timeout and heartbeat interval. Heartbeat interval is equal to
### electionTimeoutMs/electionHeartbeatFactor,One tenth by default.
nacos.core.protocol.raft.data.election_heartbeat_factor=10
### The tasks submitted to the leader accumulate the maximum batch size of a batch flush log storage. The default is 32 tasks.
nacos.core.protocol.raft.data.apply_batch=32
### Call fsync when necessary when writing logs and meta information, usually should be true
nacos.core.protocol.raft.data.sync=true
### Whether to write snapshot / raft meta-information to call fsync. The default is false. When sync is true, it is preferred to respect sync.
nacos.core.protocol.raft.data.sync_meta=false
### Internal disruptor buffer size. For applications with high write throughput, you need to increase this value. The default value is 16384.
nacos.core.protocol.raft.data.disruptor_buffer_size=16384
### Whether to enable replication of pipeline request optimization, which is enabled by default
nacos.core.protocol.raft.data.replicator_pipeline=true
### Maximum number of in-flight requests with pipeline requests enabled, default is 256
nacos.core.protocol.raft.data.max_replicator_inflight_msgs=256
### Whether to enable LogEntry checksum
nacos.core.protocol.raft.data.enable_log_entry_checksum=false

# About Distro

### Maximum interval between two data transmissions
nacos.core.protocol.distro.data.task_dispatch_period_ms=2000
### Number of keys per batch of tasks
nacos.core.protocol.distro.data.batch_sync_key_count=1000
### Task retry delay time
nacos.core.protocol.distro.data.sync_retry_delay_ms=5000
### Whether to enable the authoritative server mechanism
nacos.core.protocol.distro.data.distro_enabled=true
### Data synchronization retry strategy
nacos.core.protocol.distro.data.retry_policy=simple
```
Original file line number Diff line number Diff line change
@@ -17,13 +17,14 @@

import com.alibaba.nacos.api.config.listener.Listener;
import com.alibaba.nacos.api.exception.NacosException;
import com.alibaba.nacos.api.life.LifeCycle;

/**
* Config Service Interface
*
* @author Nacos
*/
public interface ConfigService {
public interface ConfigService extends LifeCycle {

/**
* Get config
35 changes: 35 additions & 0 deletions api/src/main/java/com/alibaba/nacos/api/life/LifeCycle.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
/*
* Copyright 1999-2018 Alibaba Group Holding Ltd.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.alibaba.nacos.api.life;

/**
* @author <a href="mailto:liaochuntao@live.com">liaochuntao</a>
*/
public interface LifeCycle {

/**
* Correlation initialization
* @throws Exception
*/
void init() throws Exception;

/**
* Destruction of the callback
* @throws Exception
*/
void destroy() throws Exception;

}
50 changes: 50 additions & 0 deletions api/src/main/java/com/alibaba/nacos/api/life/LifeCycleUtils.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
/*
* Copyright 1999-2018 Alibaba Group Holding Ltd.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.alibaba.nacos.api.life;

/**
* @author <a href="mailto:liaochuntao@live.com">liaochuntao</a>
*/
public final class LifeCycleUtils {

public static void invokeInit(LifeCycle lifeCycle) throws Exception {
if (lifeCycle != null) {
lifeCycle.init();
}
}

public static void invokeDestroy(LifeCycle lifeCycle) throws Exception {
if (lifeCycle != null) {
lifeCycle.destroy();
}
}

public static void registerShutdownHook(final LifeCycle lifeCycle) {
if (lifeCycle != null) {
Runtime.getRuntime().addShutdownHook(new Thread(new Runnable() {
@Override
public void run() {
try {
invokeDestroy(lifeCycle);
} catch (Exception e) {
e.printStackTrace();
}
}
}));
}
}

}
Original file line number Diff line number Diff line change
@@ -17,6 +17,7 @@
package com.alibaba.nacos.api.naming;

import com.alibaba.nacos.api.exception.NacosException;
import com.alibaba.nacos.api.life.LifeCycle;
import com.alibaba.nacos.api.naming.pojo.Instance;
import com.alibaba.nacos.api.naming.pojo.Service;
import com.alibaba.nacos.api.selector.AbstractSelector;
@@ -29,7 +30,7 @@
* @author liaochuntao
* @since 1.0.1
*/
public interface NamingMaintainService {
public interface NamingMaintainService extends LifeCycle {

/**
* update instance info
Original file line number Diff line number Diff line change
@@ -16,6 +16,7 @@
package com.alibaba.nacos.api.naming;

import com.alibaba.nacos.api.exception.NacosException;
import com.alibaba.nacos.api.life.LifeCycle;
import com.alibaba.nacos.api.naming.listener.EventListener;
import com.alibaba.nacos.api.naming.pojo.Instance;
import com.alibaba.nacos.api.naming.pojo.ListView;
@@ -29,7 +30,7 @@
*
* @author nkorange
*/
public interface NamingService {
public interface NamingService extends LifeCycle {

/**
* register a instance to service
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
/*
* Copyright 1999-2018 Alibaba Group Holding Ltd.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package com.alibaba.nacos.api.naming.listener;

import com.alibaba.nacos.api.naming.pojo.Instance;

import java.util.List;

/**
* @author zhixiang.yuan
*/
public class NamingBaseEvent implements Event {

private String serviceName;

private String groupName;

private String clusters;

private List<Instance> instances;

public NamingBaseEvent(String serviceName, List<Instance> instances) {
this.serviceName = serviceName;
this.instances = instances;
}

public NamingBaseEvent(String serviceName, String groupName, String clusters, List<Instance> instances) {
this.serviceName = serviceName;
this.groupName = groupName;
this.clusters = clusters;
this.instances = instances;
}

public String getServiceName() {
return serviceName;
}

public void setServiceName(String serviceName) {
this.serviceName = serviceName;
}

public List<Instance> getInstances() {
return instances;
}

public void setInstances(List<Instance> instances) {
this.instances = instances;
}

public String getGroupName() {
return groupName;
}

public void setGroupName(String groupName) {
this.groupName = groupName;
}

public String getClusters() {
return clusters;
}

public void setClusters(String clusters) {
this.clusters = clusters;
}
}
Loading