Result of FeaturePreservingSmoothing is corrupted for GeoTIFFs created with PREDICTOR=3

Result of FeaturePreservingSmoothing is corrupted for GeoTIFFs created with -co PREDICTOR=3:
gdalwarp -overwrite -of GTiff -ot Float32 -co TILED=YES -co BIGTIFF=YES -co COMPRESS=LZW -co PREDICTOR=3 -co NUM_THREADS=ALL_CPUS -wo NUM_THREADS=ALL_CPUS -multi -r cubicspline -order 3 -tr 19.109257071294062687 19.109257071294062687 -tap -t_srs "+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +no_defs +over" N47E020.HGT N47E020.tif

whitebox_tools -r=FeaturePreservingSmoothing -v --wd="." --dem=N47E020.tif -o=test.tif --filter=15 --norm_diff=20.0 --num_iter=4

Result:

Without providing -co PREDICTOR=3 the result is fine.

The WBT GeoTIFF reader does not support float-point predictors. I've just made a commit that will result in a more graceful error message when predictor 3 is used. Predictor 2 is however supported.

I just encountered the same issue but with DEFLATE (predictor 2 or 3) when reading GeoTiff. Predictor 1 is fine however. Is predictor 2 with LZW only supported in the next release? Everything tested on v1.3.0.

The fix will come in the next release, sometime this week. Predictor 3 will not be supported, but the library will issue a human-readable error message indicating this fact so users at least have some guidance. Predictor 2 should work; at least it is working on the development branch on my tests.


OneSignalNotificationServiceExtension fails to build in XCode with SDK 3.4.1

Description:

I'm getting the following error after trying to upgrade my SDK version to 3.4.1 (to support iOS 13).

Environment
SDK version 3.4.1
React Native version 59.9
Note: This builds in SDK version 3.3.3
Steps to Reproduce Issue:

Set up OneSignal SDK with OneSignalNotificationServiceExtension
Upgrade to 3.4.1
Attempt to build in XCode

Duplicate of #856


Added FILELINE and it's test file

Guido can you put this in so I can put out Flog and Final please?

Hi Brian, can you do the changes we talked about here https://github.com/FStarLang/FStar/pull/3574#issuecomment-2427810488. It should return only the basename of the file (no leading directory components) and then the test would work without anything fancy (string_of_list, etc)

All good, rebuild and retested just to be sure.
But it still needs the list_of_string/string_of_list.
Good enough?

I tried to move the string code changes into FStarC_Parser_LexFStar with
| "FILELINE"   -> STRING (
let fl = ((L.source_file lexbuf) ^ "(" ^ (string_of_int (L.current_line lexbuf)) ^ ")") in
let lfl = BatList.init (BatUTF8.length fl) (fun i -> BatUChar.code (BatUTF8.get fl i)) in
let sfl = BatUTF8.init (BatList.length lfl) (fun i -> BatUChar.chr (BatList.at lfl i)) in
sfl)
but when I
let fl1 = FILELINE in
print_string("|" ^ FILELINE ^ "|" ^ (strlen fl1) ^ "\n");
strlen says F* is typing fl1 as a nat.
Any idea how a STRING () lexeme can end up gettting typed as a nat?

Yet print as a string directly?

Hi Brian, I updated this PR to take only the basename, and the test to not use string_of_list/list_of_string. I didn't understand your comment on why it's still needed, can you say more if I misunderstood?
If this looks good to you I can merge it.

Looks good. Pls merge.


Use PyPi version to install lm-eval

Some evaluation integrations require additional packages to be installed. The following instructions detail how to install the necessary packages for lm-eval integration
lm-eval
lm-eval is a package that provides a harness for evaluating language models. It is used by the lm-evaluation-harness integration.
This package has installs transformers and torch versions that are incompatible with the versions required by SparseML . To use this integration kindly follow the following steps to install the correct versions of the packages.
Installation Instructions
Step 1: Make a new virtual environment + activate it
python3.11 -m venv sparseml-venv
source sparseml-venv/bin/activate

Step 2: Install lm-eval package
pip install lm-eval==0.4.1

Step 3: Uninstall transformers package, we will install the correct version later
pip uninstall transformers

Step 4: Install sparseml package with [llm,torch] extras
This will install the correct version of transformers package
pip install "sparseml[llm, torch]"

Step 5: Verify correct versions
pip show lm-eval nm-transformers

Note: If editable mode is used to install sparseml package, then the nm-transformers package will not be installed. In that case, the nm-transformers-nightly package is installed. Verify the correct version using the following command in that case.
pip show lm-eval nm-transformers-nightly

Output when installing in non-editable mode:
$ pip show lm-eval nm-transformers
Name: lm_eval
Version: 0.4.1
Summary: A framework for evaluating language models
Home-page:
Author:
Author-email: EleutherAI<EMAIL_ADDRESS>License: MIT
Location: /home/ubuntu/venvs/sparseml-venv/lib/python3.11/site-packages
Requires: accelerate, datasets, evaluate, evaluate, jsonlines, numexpr, peft, pybind11, pytablewriter, rouge-score, sacrebleu, scikit-learn, sqlitedict, torch, tqdm-multiprocess, transformers, zstandard
Required-by:
---
Name: nm-transformers
Version: 1.7.0
Summary: State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow
Home-page: https://github.com/neuralmagic/transformers
Author: The Hugging Face team (past and future) with the help of all our contributors (https://github.com/huggingface/transformers/graphs/contributors)
Author-email<EMAIL_ADDRESS>License: Apache 2.0 License
Location: /home/ubuntu/venvs/sparseml-venv/lib/python3.11/site-packages
Requires: filelock, huggingface-hub, numpy, packaging, pyyaml, regex, requests, safetensors, tokenizers, tqdm
Required-by:

Output when installing in editable mode:
$ pip show lm-eval nm-transformers-nightly
Name: lm_eval
Version: 0.4.1
Summary: A framework for evaluating language models
Home-page:
Author:
Author-email: EleutherAI<EMAIL_ADDRESS>License: MIT
Location: /home/ubuntu/venvs/sparseml-venv/lib/python3.11/site-packages
Requires: accelerate, datasets, evaluate, evaluate, jsonlines, numexpr, peft, pybind11, pytablewriter, rouge-score, sacrebleu, scikit-learn, sqlitedict, torch, tqdm-multiprocess, transformers, zstandard
Required-by:
---
Name: nm-transformers-nightly
Version: <IP_ADDRESS>40131
Summary: State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow
Home-page: https://github.com/neuralmagic/transformers
Author: The Hugging Face team (past and future) with the help of all our contributors (https://github.com/huggingface/transformers/graphs/contributors)
Author-email<EMAIL_ADDRESS>License: Apache 2.0 License
Location: /home/ubuntu/venvs/sparseml-venv/lib/python3.11/site-packages
Requires: filelock, huggingface-hub, numpy, packaging, pyyaml, regex, requests, safetensors, tokenizers, tqdm
Required-by:

Test Command(s)
We will evaluate the model mgoin/llama2.c-stories15M-quant-pt on the hellaswag dataset using the lm-evaluation-harness integration and limit the number of samples to 1 for a quicker test. The following command can be used to do so:
Using the CLI
sparseml.evaluate \
    "mgoin/llama2.c-stories15M-quant-pt" \
    --dataset hellaswag \
    --integration lm-evaluation-harness \
    --limit 1 # for a quicker test

Example Output (Truncated):
sparseml.evaluate \                                                                                (update-lm-eval-to-0.4.1|✚1…4⚑5)
    "mgoin/llama2.c-stories15M-quant-pt" \
    --dataset hellaswag \
    --integration lm-evaluation-harness \
    --limit 1 # for a quicker test
2024-02-23 12:05:55 sparseml.evaluation.cli INFO     Datasets to evaluate on: hellaswag
Batch size: 1
Additional integration arguments supplied: {}
2024-02-23 12:05:57 sparseml.evaluation.registry INFO     Auto collected lm-evaluation-harness integration for eval
2024-02-23:12:05:57,189 INFO     [registry.py:100] Auto collected lm-evaluation-harness integration for eval
2024-02-23:12:05:57,974 WARNING  [logging.py:61] Detected kernel version 5.4.0, which is below the recommended minimum of 5.5.0; this can cause the process to hang. It is recommended to upgrade the kernel to the minimum version or higher.
.
.
.
2024-02-23 12:05:58 sparseml.pytorch.model_load.helpers INFO     Applied an unstaged recipe to the model at mgoin/llama2.c-stories15M-quant-pt
.
.
2024-02-23:12:06:03,787 INFO     [lm_evaluation_harness.py:84] Selected Tasks: ['hellaswag']
2024-02-23:12:06:07,687 INFO     [task.py:363] Building contexts for task on rank 0...
2024-02-23:12:06:07,688 INFO     [evaluator.py:324] Running loglikelihood requests
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00,  5.28it/s]
2024-02-23 12:06:08 sparseml.evaluation.cli INFO     Evaluation done. Results:
formatted=[Evaluation(task='lm-evaluation-harness', dataset=Dataset(type=None, name='hellaswag', config={'model': 'mgoin/llama2.c-stories15M-quant-pt', 'model_args': None, 'batch_size': 1, 'batch_sizes': [], 'device': None,
.
.
.
2024-02-23 12:06:08 sparseml.evaluation.cli INFO     Saving the evaluation results to /home/ubuntu/projects/sparseml/results.json
2024-02-23:12:06:08,494 INFO     [cli.py:172] Saving the evaluation results to /home/ubuntu/projects/sparseml/results.json

Using python API
from sparseml import evaluate

results = evaluate(
    "mgoin/llama2.c-stories15M-quant-pt",
    datasets="hellaswag",
    integration="lm-evaluation-harness",
    limit=1, # for a quicker test
)
print(results)

Output (Truncated):
2024-02-23 12:12:43 sparseml.evaluation.registry INFO     Auto collected lm-evaluation-harness integration for eval
2024-02-23:12:12:43,968 INFO     [registry.py:100] Auto collected lm-evaluation-harness integration for eval
2024-02-23:12:12:44,782 WARNING  [logging.py:61] Detected kernel version 5.4.0, which is below the recommended minimum of 5.5.0; this can cause the process to hang. It is recommended to upgrade the kernel to the minimum version or higher.
2024-02-23:12:12:44,783 INFO     [huggingface.py:148] Using device 'cuda'
2024-02-23 12:12:45 sparseml.transformers.utils.helpers INFO     model_path is a huggingface model id. Attempting to download recipe from https://huggingface.co/
2024-02-23:12:12:45,398 INFO     [helpers.py:357] model_path is a huggingface model id. Attempting to download recipe from https://huggingface.co/
2024-02-23 12:12:45 sparseml.transformers.utils.helpers INFO     Found recipe: recipe.yaml for model id: mgoin/llama2.c-stories15M-quant-pt. Downloading...
2024-02-23:12:12:45,398 INFO     [helpers.py:363] Found recipe: recipe.yaml for model id: mgoin/llama2.c-stories15M-quant-pt. Downloading...
Logging all SparseML modifier-level logs to sparse_logs/23-02-2024_12.12.45.log
2024-02-23 12:12:45 sparseml.core.logger.logger INFO     Logging all SparseML modifier-level logs to sparse_logs/23-02-2024_12.12.45.log
2024-02-23:12:12:45,450 INFO     [logger.py:400] Logging all SparseML modifier-level logs to sparse_logs/23-02-2024_12.12.45.log
2024-02-23 12:12:45 sparseml.core.recipe.recipe INFO     Loading recipe from file /home/ubuntu/.cache/huggingface/hub/models--mgoin--llama2.c-stories15M-quant-pt/snapshots/863d525c4424fb315d1baaabf895320e9fcddae3/recipe.yaml
2024-02-23:12:12:45,451 INFO     [recipe.py:90] Loading recipe from file /home/ubuntu/.cache/huggingface/hub/models--mgoin--llama2.c-stories15M-quant-pt/snapshots/863d525c4424fb315d1baaabf895320e9fcddae3/recipe.yaml
manager stage: Model structure initialized
2024-02-23 12:12:45 sparseml.pytorch.model_load.helpers INFO     Applied an unstaged recipe to the model at mgoin/llama2.c-stories15M-quant-pt
2024-02-23:12:12:45,655 INFO     [helpers.py:120] Applied an unstaged recipe to the model at mgoin/llama2.c-stories15M-quant-pt
2024-02-23 12:12:45 sparseml.pytorch.model_load.helpers WARNING  Model state was not reloaded for SparseML: could not find model weights for mgoin/llama2.c-stories15M-quant-pt
2024-02-23:12:12:45,656 WARNING  [helpers.py:149] Model state was not reloaded for SparseML: could not find model weights for mgoin/llama2.c-stories15M-quant-pt
2024-02-23:12:12:46,777 WARNING  [lm_evaluation_harness.py:74] WARNING: --limit SHOULD ONLY BE USED FOR TESTING. REAL METRICS SHOULD NOT BE COMPUTED USING LIMIT.
2024-02-23:12:12:48,789 WARNING  [__init__.py:194] Some tasks could not be loaded due to missing dependencies. Run with `--verbosity DEBUG` for full details.
2024-02-23:12:12:50,643 WARNING  [__init__.py:194] Some tasks could not be loaded due to missing dependencies.   He', ' is using wrap to wrap a pair of skis.'), ('Roof shingle removal: A man is sitting on a roof.   He', ' starts pulling up roofing on a roof.')], 'resps': [[(-114.10840606689453, False)], [(-82.98793029785156, False)], [(-93.36141967773438, False)], [(-93.36141967773438, False)]], 'filtered_resps': [(-114.10840606689453, False), (-82.98793029785156, False), (-93.36141967773438, False), (-93.36141967773438, False)], 'acc': 0.0, 'acc_norm': 1.0}]}, 'config': {'model': 'mgoin/llama2.c-stories15M-quant-pt', 'model_args': None, 'batch_size': 1, 'batch_sizes': [], 'device': None, 'use_cache': None, 'limit': 1, 'bootstrap_iters': 100000, 'gen_kwargs': None}, 'git_hash': 'ba5071079a'}

To see the specific tasks where the Asana app for GitHub is being used, see below:

https://app.asana.com/0/0/1206616363971440

@rahul-tuli could you put this detailed PR description in this evaluation folder as a README? currently there is no description of this integration in the directory


Made it possible to have refs that include URI-special characters

Turns out that before:

if you had a ref that included a colon, the text before the colon was
treated as the scheme and the text after the colon as a path
the unescape_uri method was actually only parsing the path of the
uri (therefore anything before a colon was being stripped when passed
to unescape_uri)

Fixes #319

🎉

:+1:


Apr 29 10:32:37 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12254, index: 153355393] sent vote request to 134af6e5d3e35861 at term 12255
Apr 29 10:32:37 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12254, index: 153355393] sent vote request to 2a0965170762fd09 at term 12255
Apr 29 10:32:37 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 lost leader 2a0965170762fd09 at term 12255
Apr 29 10:32:37 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12255] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12254]
Apr 29 10:32:37 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12255] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12254]
Apr 29 10:32:37 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12255] ignored a MsgHeartbeat message with lower term from 2a0965170762fd09 [term: 12254]
Apr 29 10:32:37 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12255] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12254]
Apr 29 10:32:37 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12255] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12254]
Apr 29 10:32:37 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12255] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12254]
Apr 29 10:32:37 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12255] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12254]
Apr 29 10:32:37 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12255] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12254]
Apr 29 10:32:37 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12254, index: 153355393, vote: 91d25258d70ad868] rejected vote from 134af6e5d3e35861 [logterm: 12254, index: 153355393] at term 2fdf
Apr 29 10:32:37 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote rejection from 134af6e5d3e35861 at term 12255
Apr 29 10:32:37 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [q:2] has received 1 votes and 1 vote rejections
Apr 29 10:32:37 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote rejection from 2a0965170762fd09 at term 12255
Apr 29 10:32:37 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [q:2] has received 1 votes and 2 vote rejections
Apr 29 10:32:37 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12255
Apr 29 10:32:39 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12255] received a MsgVote message with higher term from 134af6e5d3e35861 [term: 12256]
Apr 29 10:32:39 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12256
Apr 29 10:32:39 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12254, index: 153355393, vote: 0] voted for 134af6e5d3e35861 [logterm: 12254, index: 153355393] at term 12256
Apr 29 10:32:39 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 elected leader 134af6e5d3e35861 at term 12256
Apr 29 10:32:43 k8s-master-3688E46E-2 etcd[1188]: got unexpected response error (etcdserver: request timed out)
Apr 29 10:35:10 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153357436, lastsnap: 153347435)
Apr 29 10:35:10 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153357436
Apr 29 10:35:10 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153352436
Apr 29 10:35:14 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002fd6-0000000009234927.snap successfully
Apr 29 10:37:34 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 is starting a new election at term 12256
Apr 29 10:37:34 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became candidate at term 12257
Apr 29 10:37:34 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 91d25258d70ad868 at term 12257
Apr 29 10:37:34 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12256, index: 153359453] sent vote request to 134af6e5d3e35861 at term 12257
Apr 29 10:37:34 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12256, index: 153359453] sent vote request to 2a0965170762fd09 at term 12257
Apr 29 10:37:34 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 lost leader 134af6e5d3e35861 at term 12257
Apr 29 10:37:34 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12257] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12256]
Apr 29 10:37:34 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 2a0965170762fd09 at term 12257
Apr 29 10:37:34 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [q:2] has received 2 votes and 0 vote rejections
Apr 29 10:37:34 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became leader at term 12257
Apr 29 10:37:34 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 elected leader 91d25258d70ad868 at term 12257
Apr 29 10:37:34 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12257] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12256]
Apr 29 10:37:34 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12257] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12256]
Apr 29 10:37:34 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12257] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12256]
Apr 29 10:37:34 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12257] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12256]
Apr 29 10:37:34 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12257] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12256]
Apr 29 10:37:34 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12257] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12256]
Apr 29 10:37:34 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12257] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12256]
Apr 29 10:37:34 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12257] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12256]
Apr 29 10:37:34 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12257] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12256]
Apr 29 10:37:34 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12257] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12256]
Apr 29 10:37:39 k8s-master-3688E46E-2 etcd[1188]: etcdserver: request timed out, possibly due to previous leader failure
Apr 29 10:37:41 k8s-master-3688E46E-2 etcd[1188]: etcdserver: request timed out, possibly due to previous leader failure
Apr 29 10:44:04 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12257] received a MsgVote message with higher term from 2a0965170762fd09 [term: 12258]
Apr 29 10:44:04 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12258
Apr 29 10:44:04 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12257, index: 153364884, vote: 0] rejected vote from 2a0965170762fd09 [logterm: 12257, index: 153364879] at term 12258
Apr 29 10:44:04 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 lost leader 91d25258d70ad868 at term 12258
Apr 29 10:44:04 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12258] ignored a MsgAppResp message with lower term from 134af6e5d3e35861 [term: 12257]
Apr 29 10:44:04 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12258] ignored a MsgHeartbeatResp message with lower term from 134af6e5d3e35861 [term: 12257]
Apr 29 10:44:04 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12258] ignored a MsgAppResp message with lower term from 134af6e5d3e35861 [term: 12257]
Apr 29 10:44:04 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12258] ignored a MsgAppResp message with lower term from 134af6e5d3e35861 [term: 12257]
Apr 29 10:44:04 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12258] ignored a MsgAppResp message with lower term from 134af6e5d3e35861 [term: 12257]
Apr 29 10:44:05 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12258] received a MsgVote message with higher term from 134af6e5d3e35861 [term: 12259]
Apr 29 10:44:05 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12259
Apr 29 10:44:05 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12257, index: 153364884, vote: 0] rejected vote from 134af6e5d3e35861 [logterm: 12257, index: 153364883] at term 12259
Apr 29 10:44:05 k8s-master-3688E46E-2 etcd[1188]: found conflict at index 153364884 [existing term: 12257, conflicting term: 12259]
Apr 29 10:44:05 k8s-master-3688E46E-2 etcd[1188]: replace the unstable entries from index 153364884
Apr 29 10:44:05 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 elected leader 134af6e5d3e35861 at term 12259
Apr 29 10:44:15 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 is starting a new election at term 12259
Apr 29 10:44:15 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became candidate at term 12260
Apr 29 10:44:15 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 91d25258d70ad868 at term 12260
Apr 29 10:44:15 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12259, index: 153365060] sent vote request to 134af6e5d3e35861 at term 12260
Apr 29 10:44:15 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12259, index: 153365060] sent vote request to 2a0965170762fd09 at term 12260
Apr 29 10:44:15 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 lost leader 134af6e5d3e35861 at term 12260
Apr 29 10:44:15 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 2a0965170762fd09 at term 12260
Apr 29 10:44:15 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [q:2] has received 2 votes and 0 vote rejections
Apr 29 10:44:15 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became leader at term 12260
Apr 29 10:44:15 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 elected leader 91d25258d70ad868 at term 12260
Apr 29 10:44:16 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12260] ignored a MsgHeartbeat message with lower term from 134af6e5d3e35861 [term: 12259]
Apr 29 10:44:16 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12260] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12259]
Apr 29 10:44:16 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12260] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12259]
Apr 29 10:44:16 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12260] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12259]
Apr 29 10:44:16 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12260] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12259]
Apr 29 10:44:16 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12260] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12259]
Apr 29 10:44:16 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12260] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12259]
Apr 29 10:47:10 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153367437, lastsnap: 153357436)
Apr 29 10:47:10 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153367437
Apr 29 10:47:10 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153362437
Apr 29 10:47:14 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002fd7-0000000009237038.snap successfully
Apr 29 10:59:05 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153377438, lastsnap: 153367437)
Apr 29 10:59:05 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153377438
Apr 29 10:59:05 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153372438
Apr 29 10:59:14 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002fd7-0000000009239749.snap successfully
Apr 29 11:11:08 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153387439, lastsnap: 153377438)
Apr 29 11:11:08 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153387439
Apr 29 11:11:08 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153382439
Apr 29 11:11:14 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002fdb-000000000923be5a.snap successfully
Apr 29 11:15:35 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12260] received a MsgVote message with higher term from 2a0965170762fd09 [term: 12261]
Apr 29 11:15:35 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12261
Apr 29 11:15:35 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12260, index: 153391157, vote: 0] rejected vote from 2a0965170762fd09 [logterm: 12260, index: 153391149] at term 12261
Apr 29 11:15:35 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 lost leader 91d25258d70ad868 at term 12261
Apr 29 11:15:35 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12261] ignored a MsgAppResp message with lower term from 134af6e5d3e35861 [term: 12260]
Apr 29 11:15:36 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 is starting a new election at term 12261
Apr 29 11:15:36 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became candidate at term 12262
Apr 29 11:15:36 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 91d25258d70ad868 at term 12262
Apr 29 11:15:36 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12260, index: 153391157] sent vote request to 134af6e5d3e35861 at term 12262
Apr 29 11:15:36 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12260, index: 153391157] sent vote request to 2a0965170762fd09 at term 12262
Apr 29 11:15:36 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 134af6e5d3e35861 at term 12262
Apr 29 11:15:36 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [q:2] has received 2 votes and 0 vote rejections
Apr 29 11:15:36 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became leader at term 12262
Apr 29 11:15:36 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 elected leader 91d25258d70ad868 at term 12262
Apr 29 11:22:25 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12262] received a MsgVote message with higher term from 134af6e5d3e35861 [term: 12263]
Apr 29 11:22:25 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12263
Apr 29 11:22:25 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12262, index: 153396848, vote: 0] rejected vote from 134af6e5d3e35861 [logterm: 12262, index: 153396832] at term 12263
Apr 29 11:22:25 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 lost leader 91d25258d70ad868 at term 12263
Apr 29 11:22:25 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12263] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12262]
Apr 29 11:22:25 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12263] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12262]
Apr 29 11:22:25 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12263] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12262]
Apr 29 11:22:25 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12263] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12262]
Apr 29 11:22:25 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12263] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12262]
Apr 29 11:22:25 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12263] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12262]
Apr 29 11:22:25 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12263] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12262]
Apr 29 11:22:25 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12263] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12262]
Apr 29 11:22:25 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12263] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12262]
Apr 29 11:22:25 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12263] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12262]
Apr 29 11:22:25 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12263] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12262]
Apr 29 11:22:25 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12263] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12262]
Apr 29 11:22:25 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12263] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12262]
Apr 29 11:22:25 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12263] ignored a MsgHeartbeatResp message with lower term from 2a0965170762fd09 [term: 12262]
Apr 29 11:22:25 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12263] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12262]
Apr 29 11:22:25 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12263] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12262]
Apr 29 11:22:25 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12263] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12262]
Apr 29 11:22:26 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12263] received a MsgVote message with higher term from 2a0965170762fd09 [term: 12264]
Apr 29 11:22:26 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12264
Apr 29 11:22:26 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12262, index: 153396848, vote: 0] rejected vote from 2a0965170762fd09 [logterm: 12262, index: 153396847] at term 12264
Apr 29 11:22:26 k8s-master-3688E46E-2 etcd[1188]: found conflict at index 153396848 [existing term: 12262, conflicting term: 12264]
Apr 29 11:22:26 k8s-master-3688E46E-2 etcd[1188]: replace the unstable entries from index 153396848
Apr 29 11:22:26 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 elected leader 2a0965170762fd09 at term 12264
Apr 29 11:23:10 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153397440, lastsnap: 153387439)
Apr 29 11:23:10 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153397440
Apr 29 11:23:10 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153392440
Apr 29 11:23:14 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002fde-000000000923e56b.snap successfully
Apr 29 11:26:26 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 is starting a new election at term 12264
Apr 29 11:26:26 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became candidate at term 12265
Apr 29 11:26:26 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 91d25258d70ad868 at term 12265
Apr 29 11:26:26 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12264, index: 153400109] sent vote request to 134af6e5d3e35861 at term 12265
Apr 29 11:26:26 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12264, index: 153400109] sent vote request to 2a0965170762fd09 at term 12265
Apr 29 11:26:26 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 lost leader 2a0965170762fd09 at term 12265
Apr 29 11:26:26 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12265] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12264]
Apr 29 11:26:26 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12265] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12264]
Apr 29 11:26:26 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12265] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12264]
Apr 29 11:26:26 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12265] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12264]
Apr 29 11:26:26 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12265] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12264]
Apr 29 11:26:26 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12265] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12264]
Apr 29 11:26:26 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote rejection from 2a0965170762fd09 at term 12265
Apr 29 11:26:26 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [q:2] has received 1 votes and 1 vote rejections
Apr 29 11:26:26 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote rejection from 134af6e5d3e35861 at term 12265
Apr 29 11:26:26 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [q:2] has received 1 votes and 2 vote rejections
Apr 29 11:26:26 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12265
Apr 29 11:26:27 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 is starting a new election at term 12265
Apr 29 11:26:27 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became candidate at term 12266
Apr 29 11:26:27 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 91d25258d70ad868 at term 12266
Apr 29 11:26:27 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12264, index: 153400109] sent vote request to 2a0965170762fd09 at term 12266
Apr 29 11:26:27 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12264, index: 153400109] sent vote request to 134af6e5d3e35861 at term 12266
Apr 29 11:26:27 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote rejection from 2a0965170762fd09 at term 12266
Apr 29 11:26:27 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [q:2] has received 1 votes and 1 vote rejections
Apr 29 11:26:27 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote rejection from 134af6e5d3e35861 at term 12266
Apr 29 11:26:27 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [q:2] has received 1 votes and 2 vote rejections
Apr 29 11:26:27 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12266
Apr 29 11:26:28 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12266] received a MsgVote message with higher term from 134af6e5d3e35861 [term: 12267]
Apr 29 11:26:28 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12267
Apr 29 11:26:28 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12264, index: 153400109, vote: 0] voted for 134af6e5d3e35861 [logterm: 12264, index: 153400110] at term 12267
Apr 29 11:26:28 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 elected leader 134af6e5d3e35861 at term 12267
Apr 29 11:30:44 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12267] received a MsgVote message with higher term from 2a0965170762fd09 [term: 12268]
Apr 29 11:30:44 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12268
Apr 29 11:30:44 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12267, index: 153403606, vote: 0] voted for 2a0965170762fd09 [logterm: 12267, index: 153403606] at term 12268
Apr 29 11:30:44 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 lost leader 134af6e5d3e35861 at term 12268
Apr 29 11:30:44 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 elected leader 2a0965170762fd09 at term 12268
Apr 29 11:30:44 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12268] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12267]
Apr 29 11:30:44 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12268] ignored a MsgHeartbeat message with lower term from 134af6e5d3e35861 [term: 12267]
Apr 29 11:30:45 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12268] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12267]
Apr 29 11:30:45 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12268] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12267]
Apr 29 11:35:21 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153407441, lastsnap: 153397440)
Apr 29 11:35:21 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153407441
Apr 29 11:35:21 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153402441
Apr 29 11:35:44 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002fe0-0000000009240c7c.snap successfully
Apr 29 11:47:24 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153417442, lastsnap: 153407441)
Apr 29 11:47:25 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153417442
Apr 29 11:47:25 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153412442
Apr 29 11:47:44 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002fe4-000000000924338d.snap successfully
Apr 29 11:59:34 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153427443, lastsnap: 153417442)
Apr 29 11:59:35 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153427443
Apr 29 11:59:35 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153422443
Apr 29 11:59:44 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002fe4-0000000009245a9e.snap successfully
Apr 29 12:11:51 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153437444, lastsnap: 153427443)
Apr 29 12:11:51 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153437444
Apr 29 12:11:51 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153432444
Apr 29 12:12:14 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002fe4-00000000092481af.snap successfully
Apr 29 12:18:40 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 is starting a new election at term 12268
Apr 29 12:18:40 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became candidate at term 12269
Apr 29 12:18:40 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 91d25258d70ad868 at term 12269
Apr 29 12:18:40 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12268, index: 153443145] sent vote request to 134af6e5d3e35861 at term 12269
Apr 29 12:18:40 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12268, index: 153443145] sent vote request to 2a0965170762fd09 at term 12269
Apr 29 12:18:40 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 lost leader 2a0965170762fd09 at term 12269
Apr 29 12:18:40 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 134af6e5d3e35861 at term 12269
Apr 29 12:18:40 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [q:2] has received 2 votes and 0 vote rejections
Apr 29 12:18:40 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became leader at term 12269
Apr 29 12:18:40 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 elected leader 91d25258d70ad868 at term 12269
Apr 29 12:18:40 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12269] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12268]
Apr 29 12:18:41 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12269] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12268]
Apr 29 12:18:41 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12269] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12268]
Apr 29 12:18:41 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12269] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12268]
Apr 29 12:18:41 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12269] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12268]
Apr 29 12:18:41 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12269] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12268]
Apr 29 12:18:46 k8s-master-3688E46E-2 etcd[1188]: etcdserver: request timed out, possibly due to previous leader failure
Apr 29 12:23:55 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153447445, lastsnap: 153437444)
Apr 29 12:23:55 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153447445
Apr 29 12:23:55 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153442445
Apr 29 12:24:14 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002fe8-000000000924a8c0.snap successfully
Apr 29 12:35:59 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153457446, lastsnap: 153447445)
Apr 29 12:35:59 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153457446
Apr 29 12:35:59 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153452446
Apr 29 12:36:14 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002fec-000000000924cfd1.snap successfully
Apr 29 12:42:28 k8s-master-3688E46E-2 etcd[1188]: segmented wal file /var/lib/etcddisk/member/wal/00000000000004c3-000000000925a860.wal is created
Apr 29 12:42:46 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/wal/00000000000004be-00000000091c47e8.wal successfully
Apr 29 12:47:42 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153467447, lastsnap: 153457446)
Apr 29 12:47:42 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153467447
Apr 29 12:47:42 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153462447
Apr 29 12:47:44 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002fec-000000000924f6e2.snap successfully
Apr 29 12:59:44 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153477448, lastsnap: 153467447)
Apr 29 12:59:44 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153477448
Apr 29 12:59:44 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153472448
Apr 29 13:00:14 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002fec-0000000009251df3.snap successfully
Apr 29 13:11:51 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153487449, lastsnap: 153477448)
Apr 29 13:11:51 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153487449
Apr 29 13:11:51 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153482449
Apr 29 13:12:14 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002fec-0000000009254504.snap successfully
Apr 29 13:23:52 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153497450, lastsnap: 153487449)
Apr 29 13:23:52 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153497450
Apr 29 13:23:52 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153492450
Apr 29 13:24:14 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002fed-0000000009256c15.snap successfully
Apr 29 13:27:27 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12269] received a MsgVote message with higher term from 2a0965170762fd09 [term: 12270]
Apr 29 13:27:27 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12270
Apr 29 13:27:27 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12269, index: 153500498, vote: 0] rejected vote from 2a0965170762fd09 [logterm: 12269, index: 153500494] at term 12270
Apr 29 13:27:27 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 lost leader 91d25258d70ad868 at term 12270
Apr 29 13:27:27 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12270] ignored a MsgAppResp message with lower term from 134af6e5d3e35861 [term: 12269]
Apr 29 13:27:28 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12270] received a MsgVote message with higher term from 2a0965170762fd09 [term: 12271]
Apr 29 13:27:28 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12271
Apr 29 13:27:28 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12269, index: 153500498, vote: 0] rejected vote from 2a0965170762fd09 [logterm: 12269, index: 153500494] at term 12271
Apr 29 13:27:29 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 is starting a new election at term 12271
Apr 29 13:27:29 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became candidate at term 12272
Apr 29 13:27:29 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 91d25258d70ad868 at term 12272
Apr 29 13:27:29 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12269, index: 153500498] sent vote request to 134af6e5d3e35861 at term 12272
Apr 29 13:27:29 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12269, index: 153500498] sent vote request to 2a0965170762fd09 at term 12272
Apr 29 13:27:29 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12269, index: 153500498, vote: 91d25258d70ad868] rejected vote from 134af6e5d3e35861 [logterm: 12269, index: 153500495] at term 2ff0
Apr 29 13:27:29 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote rejection from 134af6e5d3e35861 at term 12272
Apr 29 13:27:29 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [q:2] has received 1 votes and 1 vote rejections
Apr 29 13:27:29 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote rejection from 2a0965170762fd09 at term 12272
Apr 29 13:27:29 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [q:2] has received 1 votes and 2 vote rejections
Apr 29 13:27:29 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12272
Apr 29 13:27:29 k8s-master-3688E46E-2 etcd[1188]: found conflict at index 153500496 [existing term: 12269, conflicting term: 12272]
Apr 29 13:27:29 k8s-master-3688E46E-2 etcd[1188]: replace the unstable entries from index 153500496
Apr 29 13:27:29 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 elected leader 134af6e5d3e35861 at term 12272
Apr 29 13:27:33 k8s-master-3688E46E-2 etcd[1188]: got unexpected response error (etcdserver: request timed out)
Apr 29 13:35:58 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153507451, lastsnap: 153497450)
Apr 29 13:35:58 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153507451
Apr 29 13:35:58 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153502451
Apr 29 13:36:14 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002fed-0000000009259326.snap successfully
Apr 29 13:48:02 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153517452, lastsnap: 153507451)
Apr 29 13:48:02 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153517452
Apr 29 13:48:02 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153512452
Apr 29 13:48:14 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002fed-000000000925ba37.snap successfully
Apr 29 13:59:15 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 is starting a new election at term 12272
Apr 29 13:59:15 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became candidate at term 12273
Apr 29 13:59:15 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 91d25258d70ad868 at term 12273
Apr 29 13:59:15 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12272, index: 153526713] sent vote request to 134af6e5d3e35861 at term 12273
Apr 29 13:59:15 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12272, index: 153526713] sent vote request to 2a0965170762fd09 at term 12273
Apr 29 13:59:15 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 lost leader 134af6e5d3e35861 at term 12273
Apr 29 13:59:15 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 2a0965170762fd09 at term 12273
Apr 29 13:59:15 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [q:2] has received 2 votes and 0 vote rejections
Apr 29 13:59:15 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became leader at term 12273
Apr 29 13:59:15 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 elected leader 91d25258d70ad868 at term 12273
Apr 29 13:59:15 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12273] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12272]
Apr 29 13:59:15 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12273] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12272]
Apr 29 13:59:15 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12273] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12272]
Apr 29 13:59:15 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12273] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12272]
Apr 29 13:59:15 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12273] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12272]
Apr 29 13:59:15 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12273] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12272]
Apr 29 13:59:21 k8s-master-3688E46E-2 etcd[1188]: etcdserver: request timed out, possibly due to previous leader failure
Apr 29 13:59:21 k8s-master-3688E46E-2 etcd[1188]: etcdserver: request timed out, possibly due to previous leader failure
Apr 29 14:00:09 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153527453, lastsnap: 153517452)
Apr 29 14:00:09 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153527453
Apr 29 14:00:09 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153522453
Apr 29 14:00:14 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002fed-000000000925e148.snap successfully
Apr 29 14:12:11 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153537454, lastsnap: 153527453)
Apr 29 14:12:11 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153537454
Apr 29 14:12:11 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153532454
Apr 29 14:12:14 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002fed-0000000009260859.snap successfully
Apr 29 14:15:09 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12273] received a MsgVote message with higher term from 134af6e5d3e35861 [term: 12274]
Apr 29 14:15:09 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12274
Apr 29 14:15:09 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12273, index: 153539864, vote: 0] rejected vote from 134af6e5d3e35861 [logterm: 12273, index: 153539851] at term 12274
Apr 29 14:15:09 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 lost leader 91d25258d70ad868 at term 12274
Apr 29 14:15:09 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12274] ignored a MsgHeartbeatResp message with lower term from 2a0965170762fd09 [term: 12273]
Apr 29 14:15:09 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12274] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12273]
Apr 29 14:15:09 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12274] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12273]
Apr 29 14:15:09 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12274] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12273]
Apr 29 14:15:09 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12274] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12273]
Apr 29 14:15:09 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12274] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12273]
Apr 29 14:15:09 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12274] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12273]
Apr 29 14:15:09 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12274] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12273]
Apr 29 14:15:09 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12274] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12273]
Apr 29 14:15:09 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12274] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12273]
Apr 29 14:15:09 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12274] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12273]
Apr 29 14:15:09 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12274] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12273]
Apr 29 14:15:09 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12274] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12273]
Apr 29 14:15:10 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 is starting a new election at term 12274
Apr 29 14:15:10 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became candidate at term 12275
Apr 29 14:15:10 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 91d25258d70ad868 at term 12275
Apr 29 14:15:10 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12273, index: 153539864] sent vote request to 134af6e5d3e35861 at term 12275
Apr 29 14:15:10 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12273, index: 153539864] sent vote request to 2a0965170762fd09 at term 12275
Apr 29 14:15:10 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 2a0965170762fd09 at term 12275
Apr 29 14:15:10 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [q:2] has received 2 votes and 0 vote rejections
Apr 29 14:15:10 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became leader at term 12275
Apr 29 14:15:10 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 elected leader 91d25258d70ad868 at term 12275
Apr 29 14:24:18 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153547455, lastsnap: 153537454)
Apr 29 14:24:18 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153547455
Apr 29 14:24:18 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153542455
Apr 29 14:24:44 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002fed-0000000009262f6a.snap successfully
Apr 29 14:36:21 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153557456, lastsnap: 153547455)
Apr 29 14:36:22 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153557456
Apr 29 14:36:22 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153552456
Apr 29 14:36:44 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002ff0-000000000926567b.snap successfully
Apr 29 14:48:21 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153567457, lastsnap: 153557456)
Apr 29 14:48:21 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153567457
Apr 29 14:48:21 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153562457
Apr 29 14:48:44 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002ff0-0000000009267d8c.snap successfully
Apr 29 14:52:49 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12275] received a MsgVote message with higher term from 2a0965170762fd09 [term: 12276]
Apr 29 14:52:49 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12276
Apr 29 14:52:49 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12275, index: 153571068, vote: 0] rejected vote from 2a0965170762fd09 [logterm: 12275, index: 153571059] at term 12276
Apr 29 14:52:49 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 lost leader 91d25258d70ad868 at term 12276
Apr 29 14:52:49 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12276] ignored a MsgHeartbeatResp message with lower term from 134af6e5d3e35861 [term: 12275]
Apr 29 14:52:49 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12276] ignored a MsgAppResp message with lower term from 134af6e5d3e35861 [term: 12275]
Apr 29 14:52:49 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12276] ignored a MsgAppResp message with lower term from 134af6e5d3e35861 [term: 12275]
Apr 29 14:52:49 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12276] ignored a MsgAppResp message with lower term from 134af6e5d3e35861 [term: 12275]
Apr 29 14:52:49 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12276] ignored a MsgAppResp message with lower term from 134af6e5d3e35861 [term: 12275]
Apr 29 14:52:49 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12276] ignored a MsgAppResp message with lower term from 134af6e5d3e35861 [term: 12275]
Apr 29 14:52:49 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12276] ignored a MsgAppResp message with lower term from 134af6e5d3e35861 [term: 12275]
Apr 29 14:52:50 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 is starting a new election at term 12276
Apr 29 14:52:50 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became candidate at term 12277
Apr 29 14:52:50 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 91d25258d70ad868 at term 12277
Apr 29 14:52:50 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12275, index: 153571068] sent vote request to 134af6e5d3e35861 at term 12277
Apr 29 14:52:50 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12275, index: 153571068] sent vote request to 2a0965170762fd09 at term 12277
Apr 29 14:52:51 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 134af6e5d3e35861 at term 12277
Apr 29 14:52:51 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [q:2] has received 2 votes and 0 vote rejections
Apr 29 14:52:51 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became leader at term 12277
Apr 29 14:52:51 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 elected leader 91d25258d70ad868 at term 12277
Apr 29 15:00:25 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153577458, lastsnap: 153567457)
Apr 29 15:00:25 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153577458
Apr 29 15:00:25 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153572458
Apr 29 15:00:45 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002ff1-000000000926a49d.snap successfully
Apr 29 15:03:40 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12277] received a MsgVote message with higher term from 134af6e5d3e35861 [term: 12278]
Apr 29 15:03:40 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12278
Apr 29 15:03:40 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12277, index: 153580095, vote: 0] rejected vote from 134af6e5d3e35861 [logterm: 12277, index: 153580089] at term 12278
Apr 29 15:03:40 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 lost leader 91d25258d70ad868 at term 12278
Apr 29 15:03:40 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12278] ignored a MsgAppResp message with lower term from 2a0965170762fd09 [term: 12277]
Apr 29 15:03:41 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 is starting a new election at term 12278
Apr 29 15:03:41 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became candidate at term 12279
Apr 29 15:03:41 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 91d25258d70ad868 at term 12279
Apr 29 15:03:41 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12277, index: 153580095] sent vote request to 134af6e5d3e35861 at term 12279
Apr 29 15:03:41 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12277, index: 153580095] sent vote request to 2a0965170762fd09 at term 12279
Apr 29 15:03:41 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 2a0965170762fd09 at term 12279
Apr 29 15:03:41 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [q:2] has received 2 votes and 0 vote rejections
Apr 29 15:03:41 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became leader at term 12279
Apr 29 15:03:41 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 elected leader 91d25258d70ad868 at term 12279
Apr 29 15:06:24 k8s-master-3688E46E-2 etcd[1188]: segmented wal file /var/lib/etcddisk/member/wal/00000000000004c4-0000000009277b1d.wal is created
Apr 29 15:06:46 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/wal/00000000000004bf-00000000091e4098.wal successfully
Apr 29 15:12:32 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153587459, lastsnap: 153577458)
Apr 29 15:12:32 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153587459
Apr 29 15:12:32 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153582459
Apr 29 15:12:45 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002ff1-000000000926cbae.snap successfully
Apr 29 15:24:38 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153597460, lastsnap: 153587459)
Apr 29 15:24:38 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153597460
Apr 29 15:24:38 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153592460
Apr 29 15:24:45 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002ff3-000000000926f2bf.snap successfully
Apr 29 15:35:29 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12279] received a MsgVote message with higher term from 134af6e5d3e35861 [term: 12280]
Apr 29 15:35:29 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12280
Apr 29 15:35:29 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12279, index: 153606432, vote: 0] rejected vote from 134af6e5d3e35861 [logterm: 12279, index: 153606425] at term 12280
Apr 29 15:35:29 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 lost leader 91d25258d70ad868 at term 12280
Apr 29 15:35:29 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12279, index: 153606432, vote: 0] rejected vote from 2a0965170762fd09 [logterm: 12279, index: 153606425] at term 12280
Apr 29 15:35:30 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 is starting a new election at term 12280
Apr 29 15:35:30 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became candidate at term 12281
Apr 29 15:35:30 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 91d25258d70ad868 at term 12281
Apr 29 15:35:30 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12279, index: 153606432] sent vote request to 134af6e5d3e35861 at term 12281
Apr 29 15:35:30 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12279, index: 153606432] sent vote request to 2a0965170762fd09 at term 12281
Apr 29 15:35:30 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 134af6e5d3e35861 at term 12281
Apr 29 15:35:30 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [q:2] has received 2 votes and 0 vote rejections
Apr 29 15:35:30 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became leader at term 12281
Apr 29 15:35:30 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 elected leader 91d25258d70ad868 at term 12281
Apr 29 15:36:49 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153607461, lastsnap: 153597460)
Apr 29 15:36:49 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153607461
Apr 29 15:36:49 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153602461
Apr 29 15:37:15 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002ff3-00000000092719d0.snap successfully
Apr 29 15:44:07 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12281] received a MsgVote message with higher term from 134af6e5d3e35861 [term: 12282]
Apr 29 15:44:07 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12282
Apr 29 15:44:07 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12281, index: 153613549, vote: 0] rejected vote from 134af6e5d3e35861 [logterm: 12281, index: 153613545] at term 12282
Apr 29 15:44:07 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 lost leader 91d25258d70ad868 at term 12282
Apr 29 15:44:07 k8s-master-3688E46E-2 etcd[1188]: found conflict at index 153613546 [existing term: 12281, conflicting term: 12282]
Apr 29 15:44:07 k8s-master-3688E46E-2 etcd[1188]: replace the unstable entries from index 153613546
Apr 29 15:44:07 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 elected leader 134af6e5d3e35861 at term 12282
Apr 29 15:44:13 k8s-master-3688E46E-2 etcd[1188]: etcdserver: request timed out, possibly due to previous leader failure
Apr 29 15:48:52 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153617462, lastsnap: 153607461)
Apr 29 15:48:52 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153617462
Apr 29 15:48:52 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153612462
Apr 29 15:49:15 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002ff3-00000000092740e1.snap successfully
Apr 29 15:54:38 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12282] received a MsgVote message with higher term from 2a0965170762fd09 [term: 12283]
Apr 29 15:54:38 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12283
Apr 29 15:54:38 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12282, index: 153622253, vote: 0] voted for 2a0965170762fd09 [logterm: 12282, index: 153622253] at term 12283
Apr 29 15:54:38 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 lost leader 134af6e5d3e35861 at term 12283
Apr 29 15:54:38 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12283] ignored a MsgHeartbeat message with lower term from 134af6e5d3e35861 [term: 12282]
Apr 29 15:54:38 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12283] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12282]
Apr 29 15:54:38 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 elected leader 2a0965170762fd09 at term 12283
Apr 29 15:54:38 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12283] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12282]
Apr 29 15:54:38 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12283] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12282]
Apr 29 15:54:38 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12283] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12282]
Apr 29 15:54:38 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12283] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12282]
Apr 29 15:54:46 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12283] received a MsgVote message with higher term from 134af6e5d3e35861 [term: 12284]
Apr 29 15:54:46 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12284
Apr 29 15:54:46 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12283, index: 153622308, vote: 0] rejected vote from 134af6e5d3e35861 [logterm: 12283, index: 153622307] at term 12284
Apr 29 15:54:46 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 lost leader 2a0965170762fd09 at term 12284
Apr 29 15:54:46 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12284] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12283]
Apr 29 15:54:46 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12284] ignored a MsgHeartbeat message with lower term from 2a0965170762fd09 [term: 12283]
Apr 29 15:54:46 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12284] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12283]
Apr 29 15:54:46 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12284] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12283]
Apr 29 15:54:46 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12284] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12283]
Apr 29 15:54:46 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12284] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12283]
Apr 29 15:54:46 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12284] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12283]
Apr 29 15:54:46 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12284] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12283]
Apr 29 15:54:46 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12284] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12283]
Apr 29 15:54:46 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12284] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12283]
Apr 29 15:54:47 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 is starting a new election at term 12284
Apr 29 15:54:47 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became candidate at term 12285
Apr 29 15:54:47 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 91d25258d70ad868 at term 12285
Apr 29 15:54:47 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12283, index: 153622308] sent vote request to 134af6e5d3e35861 at term 12285
Apr 29 15:54:47 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12283, index: 153622308] sent vote request to 2a0965170762fd09 at term 12285
Apr 29 15:54:47 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12283, index: 153622308, vote: 91d25258d70ad868] rejected vote from 134af6e5d3e35861 [logterm: 12283, index: 153622307] at term 2ffd
Apr 29 15:54:47 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote rejection from 134af6e5d3e35861 at term 12285
Apr 29 15:54:47 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [q:2] has received 1 votes and 1 vote rejections
Apr 29 15:54:47 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote rejection from 2a0965170762fd09 at term 12285
Apr 29 15:54:47 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [q:2] has received 1 votes and 2 vote rejections
Apr 29 15:54:47 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12285
Apr 29 15:54:48 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12285] received a MsgVote message with higher term from 134af6e5d3e35861 [term: 12286]
Apr 29 15:54:48 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12286
Apr 29 15:54:48 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12283, index: 153622308, vote: 0] rejected vote from 134af6e5d3e35861 [logterm: 12283, index: 153622307] at term 12286
Apr 29 15:54:50 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 is starting a new election at term 12286
Apr 29 15:54:50 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became candidate at term 12287
Apr 29 15:54:50 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 91d25258d70ad868 at term 12287
Apr 29 15:54:50 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12283, index: 153622308] sent vote request to 134af6e5d3e35861 at term 12287
Apr 29 15:54:50 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12283, index: 153622308] sent vote request to 2a0965170762fd09 at term 12287
Apr 29 15:54:50 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote rejection from 2a0965170762fd09 at term 12287
Apr 29 15:54:50 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [q:2] has received 1 votes and 1 vote rejections
Apr 29 15:54:50 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 134af6e5d3e35861 at term 12287
Apr 29 15:54:50 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [q:2] has received 2 votes and 1 vote rejections
Apr 29 15:54:50 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became leader at term 12287
Apr 29 15:54:50 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 elected leader 91d25258d70ad868 at term 12287
Apr 29 15:54:52 k8s-master-3688E46E-2 etcd[1188]: got unexpected response error (etcdserver: request timed out)
Apr 29 15:54:53 k8s-master-3688E46E-2 etcd[1188]: got unexpected response error (etcdserver: request timed out)
Apr 29 15:54:53 k8s-master-3688E46E-2 etcd[1188]: got unexpected response error (etcdserver: request timed out)
Apr 29 16:00:48 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12287] received a MsgVote message with higher term from 134af6e5d3e35861 [term: 12288]
Apr 29 16:00:48 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12288
Apr 29 16:00:48 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12287, index: 153627368, vote: 0] rejected vote from 134af6e5d3e35861 [logterm: 12287, index: 153627362] at term 12288
Apr 29 16:00:48 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 lost leader 91d25258d70ad868 at term 12288
Apr 29 16:00:48 k8s-master-3688E46E-2 etcd[1188]: found conflict at index 153627363 [existing term: 12287, conflicting term: 12288]
Apr 29 16:00:48 k8s-master-3688E46E-2 etcd[1188]: replace the unstable entries from index 153627363
Apr 29 16:00:48 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 elected leader 134af6e5d3e35861 at term 12288
Apr 29 16:00:55 k8s-master-3688E46E-2 etcd[1188]: etcdserver: request timed out, possibly due to previous leader failure
Apr 29 16:00:55 k8s-master-3688E46E-2 etcd[1188]: etcdserver: request timed out, possibly due to previous leader failure
Apr 29 16:00:57 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153627463, lastsnap: 153617462)
Apr 29 16:00:57 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153627463
Apr 29 16:00:57 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153622463
Apr 29 16:01:15 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002ff5-00000000092767f2.snap successfully
Apr 29 16:13:02 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153637464, lastsnap: 153627463)
Apr 29 16:13:02 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153637464
Apr 29 16:13:02 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153632464
Apr 29 16:13:15 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002ff7-0000000009278f03.snap successfully
Apr 29 16:21:03 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12288] received a MsgVote message with higher term from 2a0965170762fd09 [term: 12289]
Apr 29 16:21:03 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12289
Apr 29 16:21:03 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12288, index: 153644056, vote: 0] voted for 2a0965170762fd09 [logterm: 12288, index: 153644056] at term 12289
Apr 29 16:21:03 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 lost leader 134af6e5d3e35861 at term 12289
Apr 29 16:21:03 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 elected leader 2a0965170762fd09 at term 12289
Apr 29 16:21:03 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12289] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12288]
Apr 29 16:21:04 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12289] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12288]
Apr 29 16:21:04 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12289] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12288]
Apr 29 16:21:04 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12289] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12288]
Apr 29 16:21:04 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12289] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12288]
Apr 29 16:21:04 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12289] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12288]
Apr 29 16:21:04 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12289] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12288]
Apr 29 16:21:04 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12289] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12288]
Apr 29 16:21:04 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12289] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12288]
Apr 29 16:21:04 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12289] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12288]
Apr 29 16:21:09 k8s-master-3688E46E-2 etcd[1188]: etcdserver: request timed out, possibly due to previous leader failure
Apr 29 16:21:10 k8s-master-3688E46E-2 etcd[1188]: etcdserver: request timed out, possibly due to previous leader failure
Apr 29 16:21:10 k8s-master-3688E46E-2 etcd[1188]: etcdserver: request timed out, possibly due to previous leader failure
Apr 29 16:21:10 k8s-master-3688E46E-2 etcd[1188]: etcdserver: request timed out, possibly due to previous leader failure
Apr 29 16:21:10 k8s-master-3688E46E-2 etcd[1188]: etcdserver: request timed out, possibly due to previous leader failure
Apr 29 16:21:10 k8s-master-3688E46E-2 etcd[1188]: etcdserver: request timed out, possibly due to previous leader failure
Apr 29 16:21:10 k8s-master-3688E46E-2 etcd[1188]: etcdserver: request timed out, possibly due to previous leader failure
Apr 29 16:25:08 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153647465, lastsnap: 153637464)
Apr 29 16:25:09 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153647465
Apr 29 16:25:09 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153642465
Apr 29 16:25:15 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002ff7-000000000927b614.snap successfully
Apr 29 16:37:16 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153657466, lastsnap: 153647465)
Apr 29 16:37:16 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153657466
Apr 29 16:37:16 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153652466
Apr 29 16:37:45 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002ff9-000000000927dd25.snap successfully
Apr 29 16:39:38 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 is starting a new election at term 12289
Apr 29 16:39:38 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became candidate at term 12290
Apr 29 16:39:38 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 91d25258d70ad868 at term 12290
Apr 29 16:39:38 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12289, index: 153659417] sent vote request to 134af6e5d3e35861 at term 12290
Apr 29 16:39:38 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12289, index: 153659417] sent vote request to 2a0965170762fd09 at term 12290
Apr 29 16:39:38 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 lost leader 2a0965170762fd09 at term 12290
Apr 29 16:39:38 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 134af6e5d3e35861 at term 12290
Apr 29 16:39:38 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [q:2] has received 2 votes and 0 vote rejections
Apr 29 16:39:38 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became leader at term 12290
Apr 29 16:39:38 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 elected leader 91d25258d70ad868 at term 12290
Apr 29 16:39:39 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12290] ignored a MsgHeartbeat message with lower term from 2a0965170762fd09 [term: 12289]
Apr 29 16:39:39 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12290] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12289]
Apr 29 16:39:39 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12290] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12289]
Apr 29 16:39:45 k8s-master-3688E46E-2 etcd[1188]: etcdserver: request timed out, possibly due to previous leader failure
Apr 29 16:49:18 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153667467, lastsnap: 153657466)
Apr 29 16:49:18 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153667467
Apr 29 16:49:18 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153662467
Apr 29 16:49:45 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000002ffa-0000000009280436.snap successfully
Apr 29 16:54:09 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12290] received a MsgVote message with higher term from 134af6e5d3e35861 [term: 12291]
Apr 29 16:54:09 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12291
Apr 29 16:54:09 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12290, index: 153671478, vote: 0] rejected vote from 134af6e5d3e35861 [logterm: 12290, index: 153671470] at term 12291
Apr 29 16:54:09 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 lost leader 91d25258d70ad868 at term 12291
Apr 29 16:54:09 k8s-master-3688E46E-2 etcd[1188]: found conflict at index 153671471 [existing term: 12290, conflicting term: 12291]
Apr 29 16:54:09 k8s-master-3688E46E-2 etcd[1188]: replace the unstable entries from index 153671471
Apr 29 16:54:09 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 elected leader 134af6e5d3e35861 at term 12291
Apr 29 16:54:16 k8s-master-3688E46E-2 etcd[1188]: etcdserver: request timed out, possibly due to previous leader failure
Apr 29 16:54:16 k8s-master-3688E46E-2 etcd[1188]: etcdserver: request timed out, possibly due to previous leader failure
Apr 29 17:01:27 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153677468, lastsnap: 153667467)
Apr 29 17:01:27 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153677468
Apr 29 17:01:27 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153672468
Apr 29 17:01:45 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000003000-0000000009282b47.snap successfully
Apr 29 17:13:32 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153687469, lastsnap: 153677468)
Apr 29 17:13:32 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153687469
Apr 29 17:13:32 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153682469
Apr 29 17:13:45 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000003000-0000000009285258.snap successfully
Apr 29 17:25:44 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153697470, lastsnap: 153687469)
Apr 29 17:25:45 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153697470
Apr 29 17:25:45 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153692470
Apr 29 17:25:45 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000003001-0000000009287969.snap successfully
Apr 29 17:26:55 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12291] received a MsgVote message with higher term from 2a0965170762fd09 [term: 12292]
Apr 29 17:26:55 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12292
Apr 29 17:26:55 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12291, index: 153698381, vote: 0] voted for 2a0965170762fd09 [logterm: 12291, index: 153698381] at term 12292
Apr 29 17:26:55 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 lost leader 134af6e5d3e35861 at term 12292
Apr 29 17:26:55 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12292] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12291]
Apr 29 17:26:55 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 elected leader 2a0965170762fd09 at term 12292
Apr 29 17:26:55 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12292] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12291]
Apr 29 17:26:55 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12292] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12291]
Apr 29 17:26:55 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12292] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12291]
Apr 29 17:26:55 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12292] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12291]
Apr 29 17:26:55 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12292] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12291]
Apr 29 17:26:55 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12292] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12291]
Apr 29 17:27:01 k8s-master-3688E46E-2 etcd[1188]: etcdserver: request timed out, possibly due to previous leader failure
Apr 29 17:32:07 k8s-master-3688E46E-2 etcd[1188]: segmented wal file /var/lib/etcddisk/member/wal/00000000000004c5-0000000009295143.wal is created
Apr 29 17:32:16 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/wal/00000000000004c0-0000000009201fd9.wal successfully
Apr 29 17:37:57 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153707471, lastsnap: 153697470)
Apr 29 17:37:57 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153707471
Apr 29 17:37:57 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153702471
Apr 29 17:38:15 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000003001-000000000928a07a.snap successfully
Apr 29 17:45:52 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12292] received a MsgVote message with higher term from 134af6e5d3e35861 [term: 12293]
Apr 29 17:45:52 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12293
Apr 29 17:45:52 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12292, index: 153713987, vote: 0] voted for 134af6e5d3e35861 [logterm: 12292, index: 153713987] at term 12293
Apr 29 17:45:52 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 lost leader 2a0965170762fd09 at term 12293
Apr 29 17:45:52 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 elected leader 134af6e5d3e35861 at term 12293
Apr 29 17:45:52 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12293] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12292]
Apr 29 17:45:52 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12293] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12292]
Apr 29 17:45:52 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12293] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12292]
Apr 29 17:45:52 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12293] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12292]
Apr 29 17:45:52 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12293] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12292]
Apr 29 17:45:52 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12293] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12292]
Apr 29 17:45:52 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12293] ignored a MsgApp message with lower term from 2a0965170762fd09 [term: 12292]
Apr 29 17:45:58 k8s-master-3688E46E-2 etcd[1188]: etcdserver: request timed out, possibly due to previous leader failure
Apr 29 17:45:58 k8s-master-3688E46E-2 etcd[1188]: etcdserver: request timed out, possibly due to previous leader failure
Apr 29 17:50:03 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153717472, lastsnap: 153707471)
Apr 29 17:50:03 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153717472
Apr 29 17:50:03 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153712472
Apr 29 17:50:15 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000003002-000000000928c78b.snap successfully
Apr 29 18:02:08 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153727473, lastsnap: 153717472)
Apr 29 18:02:08 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153727473
Apr 29 18:02:08 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153722473
Apr 29 18:02:15 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000003003-000000000928ee9c.snap successfully
Apr 29 18:05:16 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 is starting a new election at term 12293
Apr 29 18:05:16 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became candidate at term 12294
Apr 29 18:05:16 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 91d25258d70ad868 at term 12294
Apr 29 18:05:16 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12293, index: 153730049] sent vote request to 134af6e5d3e35861 at term 12294
Apr 29 18:05:16 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12293, index: 153730049] sent vote request to 2a0965170762fd09 at term 12294
Apr 29 18:05:16 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 lost leader 134af6e5d3e35861 at term 12294
Apr 29 18:05:16 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 2a0965170762fd09 at term 12294
Apr 29 18:05:16 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [q:2] has received 2 votes and 0 vote rejections
Apr 29 18:05:16 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became leader at term 12294
Apr 29 18:05:16 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 elected leader 91d25258d70ad868 at term 12294
Apr 29 18:05:17 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12294] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12293]
Apr 29 18:05:17 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12294] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12293]
Apr 29 18:05:17 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12294] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12293]
Apr 29 18:05:17 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12294] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12293]
Apr 29 18:05:17 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12294] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12293]
Apr 29 18:05:17 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12294] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12293]
Apr 29 18:05:17 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12294] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12293]
Apr 29 18:05:22 k8s-master-3688E46E-2 etcd[1188]: etcdserver: request timed out, possibly due to previous leader failure
Apr 29 18:05:23 k8s-master-3688E46E-2 etcd[1188]: etcdserver: request timed out, possibly due to previous leader failure
Apr 29 18:05:23 k8s-master-3688E46E-2 etcd[1188]: etcdserver: request timed out, possibly due to previous leader failure
Apr 29 18:11:56 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12294] received a MsgVote message with higher term from 134af6e5d3e35861 [term: 12295]
Apr 29 18:11:56 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12295
Apr 29 18:11:56 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12294, index: 153735477, vote: 0] rejected vote from 134af6e5d3e35861 [logterm: 12294, index: 153735472] at term 12295
Apr 29 18:11:56 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 lost leader 91d25258d70ad868 at term 12295
Apr 29 18:11:56 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12294, index: 153735477, vote: 0] rejected vote from 2a0965170762fd09 [logterm: 12294, index: 153735472] at term 12295
Apr 29 18:11:57 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 is starting a new election at term 12295
Apr 29 18:11:57 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became candidate at term 12296
Apr 29 18:11:57 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 91d25258d70ad868 at term 12296
Apr 29 18:11:57 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12294, index: 153735477] sent vote request to 134af6e5d3e35861 at term 12296
Apr 29 18:11:57 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12294, index: 153735477] sent vote request to 2a0965170762fd09 at term 12296
Apr 29 18:11:57 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 received vote from 2a0965170762fd09 at term 12296
Apr 29 18:11:57 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [q:2] has received 2 votes and 0 vote rejections
Apr 29 18:11:57 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became leader at term 12296
Apr 29 18:11:57 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 elected leader 91d25258d70ad868 at term 12296
Apr 29 18:13:37 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12296] received a MsgVote message with higher term from 134af6e5d3e35861 [term: 12297]
Apr 29 18:13:37 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12297
Apr 29 18:13:37 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12296, index: 153736990, vote: 0] rejected vote from 134af6e5d3e35861 [logterm: 12296, index: 153736987] at term 12297
Apr 29 18:13:37 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 lost leader 91d25258d70ad868 at term 12297
Apr 29 18:13:37 k8s-master-3688E46E-2 etcd[1188]: found conflict at index 153736988 [existing term: 12296, conflicting term: 12297]
Apr 29 18:13:37 k8s-master-3688E46E-2 etcd[1188]: replace the unstable entries from index 153736988
Apr 29 18:13:37 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 elected leader 134af6e5d3e35861 at term 12297
Apr 29 18:13:43 k8s-master-3688E46E-2 etcd[1188]: etcdserver: request timed out, possibly due to previous leader failure
Apr 29 18:14:14 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153737474, lastsnap: 153727473)
Apr 29 18:14:14 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153737474
Apr 29 18:14:14 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153732474
Apr 29 18:14:15 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000003003-00000000092915ad.snap successfully
Apr 29 18:25:18 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12297] received a MsgVote message with higher term from 2a0965170762fd09 [term: 12298]
Apr 29 18:25:18 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 became follower at term 12298
Apr 29 18:25:18 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [logterm: 12297, index: 153746578, vote: 0] voted for 2a0965170762fd09 [logterm: 12297, index: 153746578] at term 12298
Apr 29 18:25:18 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 lost leader 134af6e5d3e35861 at term 12298
Apr 29 18:25:18 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12298] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12297]
Apr 29 18:25:18 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12298] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12297]
Apr 29 18:25:18 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12298] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12297]
Apr 29 18:25:18 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12298] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12297]
Apr 29 18:25:18 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12298] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12297]
Apr 29 18:25:18 k8s-master-3688E46E-2 etcd[1188]: 91d25258d70ad868 [term: 12298] ignored a MsgApp message with lower term from 134af6e5d3e35861 [term: 12297]
Apr 29 18:25:18 k8s-master-3688E46E-2 etcd[1188]: raft.node: 91d25258d70ad868 elected leader 2a0965170762fd09 at term 12298
Apr 29 18:25:23 k8s-master-3688E46E-2 etcd[1188]: etcdserver: request timed out, possibly due to previous leader failure
Apr 29 18:25:25 k8s-master-3688E46E-2 etcd[1188]: etcdserver: request timed out, possibly due to previous leader failure
Apr 29 18:26:22 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153747475, lastsnap: 153737474)
Apr 29 18:26:22 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153747475
Apr 29 18:26:22 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153742475
Apr 29 18:26:45 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000003003-0000000009293cbe.snap successfully
Apr 29 18:38:21 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153757476, lastsnap: 153747475)
Apr 29 18:38:21 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153757476
Apr 29 18:38:21 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153752476
Apr 29 18:38:45 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000003004-00000000092963cf.snap successfully
Apr 29 18:50:29 k8s-master-3688E46E-2 etcd[1188]: start to snapshot (applied: 153767477, lastsnap: 153757476)
Apr 29 18:50:29 k8s-master-3688E46E-2 etcd[1188]: saved snapshot at index 153767477
Apr 29 18:50:29 k8s-master-3688E46E-2 etcd[1188]: compacted raft log at 153762477
Apr 29 18:50:45 k8s-master-3688E46E-2 etcd[1188]: purged file /var/lib/etcddisk/member/snap/0000000000003005-0000000009298ae0.snap successfully

@gyuho  Any comments on this?