Adding a New Language

First of all, many thanks for a great repo. I'm kind of new to this stuff, please forgive me. Can we train a new speaker and language using this repo, for example Turkish. I would be very grateful if you could provide information on what the structure of the data set should be and how it was prepared.

Hi! Yes, enabling training models for new languages is one of the core goals of this repo. I tried my best to write detailed instructions in the README on how to train a new model on a new language.
Generally you don't need to train an aligner and a vocoder, those two components can just be downloaded from the release page and used as they are. For the TTS model that goes from text to spectrogram frames, you need to train a model. You can either do that from scratch, or you can download the meta-learning checkpoint from the releases and finetune this model to new data. The second option is generally recommended, because it's faster and can get by with fewer data.
The dataset can be in any format you have available, you just need to write a function that returns a dictionary with the keys being the path to a wav file and the value for that key being the transcription of what is said in that audio. For Turkish you'd also have to add handling in the text processing, because Turkish is not yet supported. (This will change in a few versions, we will add support for a lot more languages.) Simply looking up the ID of Turkish in the espeak phonemizer (https://github.com/espeak-ng/espeak-ng/blob/master/docs/languages.md) and adding it to the TextFrontend should be enough, like in the following:
https://github.com/DigitalPhonetics/IMS-Toucan/blob/4af4c7c04dc5bffae225dc507727261f702738fb/Preprocessing/TextFrontend.py#L128
You will then also need to add a sentence for testing the learning-progress in the train loop, since it's a new language, like in the following:
https://github.com/DigitalPhonetics/IMS-Toucan/blob/4af4c7c04dc5bffae225dc507727261f702738fb/TrainingInterfaces/Text_to_Spectrogram/FastSpeech2/fastspeech2_train_loop.py#L53
To train a model with the new data, have a look at the finetuning example:
https://github.com/DigitalPhonetics/IMS-Toucan/blob/4af4c7c04dc5bffae225dc507727261f702738fb/TrainingInterfaces/TrainingPipelines/FastSpeech2_finetuning_example.py#L31
There are a bunch of comments that guide you through the changes you need to make.

Hello, I am trying to fine-tune with Quechua, but got this error:
python run_training_pipeline.py fine_qu --gpu_id 0,1 --resume_checkpoint /Models/FastSpeech2_Meta --finetune --model_save_dir /home/luis/Documentos/VCQuechua/IMS-Toucan/Models/Quechua Preparing Prepared an Aligner dataset with 0 datapoints in Corpora/Quechua. Traceback (most recent call last):   File "run_training_pipeline.py", line 77, in <module>     pipeline_dict[args.pipeline](gpu_id=args.gpu_id,   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/TrainingInterfaces/TrainingPipelines/FastSpeech2_finetuning_example.py", line 36, in run     datasets.append(prepare_fastspeech_corpus(transcript_dict=build_path_to_transcript_dict_quechua(),   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/Utility/corpus_preparation.py", line 38, in prepare_fastspeech_corpus     train_aligner(train_dataset=aligner_datapoints,   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/TrainingInterfaces/Text_to_Spectrogram/AutoAligner/autoaligner_train_loop.py", line 48, in train_loop     train_loader = DataLoader(batch_size=batch_size,   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 344, in __init__     sampler = RandomSampler(dataset, generator=generator)  # type: ignore[arg-type]   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/torch/utils/data/sampler.py", line 107, in __init__     raise ValueError("num_samples should be a positive integer " ValueError: num_samples should be a positive integer value, but got num_samples=0
Any ideas? I made all the changes you mention here.
I had trouble with espeak since installing it with apt-get installed an older version which didn't support Quechua, so I had to install it from source and the first archive from the release was incomplete, I uninstalled the apt-get installation and installed from source the release source code and now I got this error...

It says in your output Prepared an Aligner dataset with 0 datapoints in Corpora/Quechua. So apparently something went wrong with the creation of the data cache. If this is from an earlier attempt, you can just delete the cache and try again.  If there is no other error-message that comes before this output telling you that there are no datapoints, it might mean that the path to transcript dictionary you input to the dataset creation method has a problem. You could print the dict and make sure that it actually has some content, that the paths are absolute and that the values are not empty.

Hi, just some minutes ago I came across the solution too (I didn't notice you had answered), the cache file was created from the first attempt, but in that moment I had de espeak 1.50 which didn't support quechua, so it didn't work, I deleted it, so it could create the real cache, but now I get another error!
python run_training_pipeline.py fine_qu --gpu_id 0,1 --resume_checkpoint /Models/FastSpeech2_Meta --finetune --model_save_dir /home/luis/Documentos/VCQuechua/IMS-Toucan/Models/Quechua Preparing ... building dataset cache ... Process Process-4: Traceback (most recent call last):   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap     self.run()   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/multiprocessing/process.py", line 108, in run     self._target(*self._args, **self._kwargs)   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/TrainingInterfaces/Text_to_Spectrogram/AutoAligner/AlignerDataset.py", line 123, in cache_builder_process     _, sr = sf.read(path_list[0])   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 282, in read     with SoundFile(file, 'r', samplerate, channels,   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 655, in __init__     self._file = self._open(file, mode_int, closefd)   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 1213, in _open     raise LibsndfileError(err, prefix="Error opening {0!r}: ".format(self.name)) soundfile.LibsndfileError: Error opening 'Corpora/Quechua/cus_quz_gui_202078.wav': Format not recognised. Process Process-13: Traceback (most recent call last):   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap     self.run()   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/multiprocessing/process.py", line 108, in run     self._target(*self._args, **self._kwargs)   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/TrainingInterfaces/Text_to_Spectrogram/AutoAligner/AlignerDataset.py", line 123, in cache_builder_process     _, sr = sf.read(path_list[0])   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 282, in read     with SoundFile(file, 'r', samplerate, channels,   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 655, in __init__     self._file = self._open(file, mode_int, closefd)   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 1213, in _open     raise LibsndfileError(err, prefix="Error opening {0!r}: ".format(self.name)) soundfile.LibsndfileError: Error opening 'Corpora/Quechua/cus_quz_gui_202074.wav': Format not recognised. Process Process-17: Traceback (most recent call last):   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap     self.run()   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/multiprocessing/process.py", line 108, in run     self._target(*self._args, **self._kwargs)   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/TrainingInterfaces/Text_to_Spectrogram/AutoAligner/AlignerDataset.py", line 123, in cache_builder_process     _, sr = sf.read(path_list[0])   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 282, in read     with SoundFile(file, 'r', samplerate, channels,   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 655, in __init__     self._file = self._open(file, mode_int, closefd)   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 1213, in _open     raise LibsndfileError(err, prefix="Error opening {0!r}: ".format(self.name)) soundfile.LibsndfileError: Error opening 'Corpora/Quechua/cus_quz_gui_202081.wav': Format not recognised. Process Process-18: Traceback (most recent call last):   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap     self.run()   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/multiprocessing/process.py", line 108, in run     self._target(*self._args, **self._kwargs)   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/TrainingInterfaces/Text_to_Spectrogram/AutoAligner/AlignerDataset.py", line 123, in cache_builder_process     _, sr = sf.read(path_list[0])   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 282, in read     with SoundFile(file, 'r', samplerate, channels,   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 655, in __init__     self._file = self._open(file, mode_int, closefd)   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 1213, in _open     raise LibsndfileError(err, prefix="Error opening {0!r}: ".format(self.name)) soundfile.LibsndfileError: Error opening 'Corpora/Quechua/cus_quz_gui_202080.wav': Format not recognised.   0%|                                                                                | 0/3 [00:00<?, ?it/s]unknown phoneme:
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  2.87it/s]
0%|                                                                                | 0/3 [00:00<?, ?it/s]unknown phoneme:  67%|████████████████████████████████████████████████████████████████████                                  | 2/3 [00:00<00:00,  2.91it/s]   0%|                                                                                | 0/3 [00:00<?, ?it/s]Process Process-6: Traceback (most recent call last):   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap     self.run()   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/multiprocessing/process.py", line 108, in run     self._target(*self._args, **self._kwargs)   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/TrainingInterfaces/Text_to_Spectrogram/AutoAligner/AlignerDataset.py", line 130, in cache_builder_process     wave, sr = sf.read(path)   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 282, in read     with SoundFile(file, 'r', samplerate, channels,   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 655, in __init__     self._file = self._open(file, mode_int, closefd)   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 1213, in _open     raise LibsndfileError(err, prefix="Error opening {0!r}: ".format(self.name)) soundfile.LibsndfileError: Error opening 'Corpora/Quechua/cus_quz_gui_202072.wav': Format not recognised.  67%|████████████████████████████████████████████████████████████████████                                  | 2/3 [00:00<00:00,  2.80it/s]unknown phoneme:
unknown phoneme:  67%|████████████████████████████████████████████████████████████████████                                  | 2/3 [00:00<00:00,  2.60it/s] unknown phoneme:
unknown phoneme:  33%|██████████████████████████████████                                                                    | 1/3 [00:00<00:01,  1.45it/s]Process Process-12: Traceback (most recent call last):   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap     self.run()   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/multiprocessing/process.py", line 108, in run     self._target(*self._args, **self._kwargs)   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/TrainingInterfaces/Text_to_Spectrogram/AutoAligner/AlignerDataset.py", line 130, in cache_builder_process     wave, sr = sf.read(path)   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 282, in read     with SoundFile(file, 'r', samplerate, channels,   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 655, in __init__     self._file = self._open(file, mode_int, closefd)   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 1213, in _open     raise LibsndfileError(err, prefix="Error opening {0!r}: ".format(self.name)) soundfile.LibsndfileError: Error opening 'Corpora/Quechua/cus_quz_gui_202076.wav': Format not recognised. unknown phoneme:
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00,  3.32it/s]
unknown phoneme:   0%|                                                                                | 0/3 [00:00<?, ?it/s]unknown phoneme:
67%|████████████████████████████████████████████████████████████████████                                  | 2/3 [00:00<00:00,  2.36it/s]unknown phoneme: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:01<00:00,  2.87it/s] unknown phoneme:
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00,  3.29it/s]
unknown phoneme: unknown phoneme:
33%|██████████████████████████████████                                                                    | 1/3 [00:00<00:00,  2.01it/s]
50%|███████████████████████████████████████████████████                                                   | 1/2 [00:00<00:00,  1.61it/s]Process Process-23:
Traceback (most recent call last):
File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/home/luis/Documentos/VCQuechua/IMS-Toucan/TrainingInterfaces/Text_to_Spectrogram/AutoAligner/AlignerDataset.py", line 130, in cache_builder_process
wave, sr = sf.read(path)
File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 282, in read
with SoundFile(file, 'r', samplerate, channels,
File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 655, in init
self._file = self._open(file, mode_int, closefd)
File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 1213, in _open
raise LibsndfileError(err, prefix="Error opening {0!r}: ".format(self.name))
soundfile.LibsndfileError: Error opening 'Corpora/Quechua/cus_quz_gui_202077.wav': Format not recognised.
0%|                                                                                | 0/3 [00:00<?, ?it/s]unknown phoneme:  33%|██████████████████████████████████                                                                    | 1/3 [00:00<00:01,  1.72it/s] unknown phoneme:
33%|██████████████████████████████████                                                                    | 1/3 [00:00<00:01,  1.51it/s]Process Process-27:
Traceback (most recent call last):
File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/home/luis/Documentos/VCQuechua/IMS-Toucan/TrainingInterfaces/Text_to_Spectrogram/AutoAligner/AlignerDataset.py", line 130, in cache_builder_process
wave, sr = sf.read(path)
File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 282, in read
with SoundFile(file, 'r', samplerate, channels,
File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 655, in init
self._file = self._open(file, mode_int, closefd)
File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 1213, in _open
raise LibsndfileError(err, prefix="Error opening {0!r}: ".format(self.name))
soundfile.LibsndfileError: Error opening 'Corpora/Quechua/cus_quz_gui_202075.wav': Format not recognised.
unknown phoneme: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:01<00:00,  2.14it/s] 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:01<00:00,  2.39it/s]  67%|████████████████████████████████████████████████████████████████████                                  | 2/3 [00:00<00:00,  2.61it/s]unknown phoneme:
0%|                                                                                | 0/3 [00:00<?, ?it/s]unknown phoneme:  33%|██████████████████████████████████                                                                    | 1/3 [00:00<00:01,  1.58it/s] Process Process-25: Traceback (most recent call last):   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap     self.run()   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/multiprocessing/process.py", line 108, in run     self._target(*self._args, **self._kwargs)   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/TrainingInterfaces/Text_to_Spectrogram/AutoAligner/AlignerDataset.py", line 130, in cache_builder_process     wave, sr = sf.read(path)   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 282, in read     with SoundFile(file, 'r', samplerate, channels,   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 655, in __init__     self._file = self._open(file, mode_int, closefd)   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 1213, in _open     raise LibsndfileError(err, prefix="Error opening {0!r}: ".format(self.name)) soundfile.LibsndfileError: Error opening 'Corpora/Quechua/cus_quz_gui_202079.wav': Format not recognised.  33%|██████████████████████████████████                                                                    | 1/3 [00:00<00:01,  1.34it/s]unknown phoneme:
67%|████████████████████████████████████████████████████████████████████                                  | 2/3 [00:01<00:00,  2.13it/s]unknown phoneme: unknown phoneme:
67%|████████████████████████████████████████████████████████████████████                                  | 2/3 [00:00<00:00,  2.77it/s]unknown phoneme: unknown phoneme:
67%|████████████████████████████████████████████████████████████████████                                  | 2/3 [00:01<00:00,  1.74it/s]
Process Process-29:
Traceback (most recent call last):
File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/home/luis/Documentos/VCQuechua/IMS-Toucan/TrainingInterfaces/Text_to_Spectrogram/AutoAligner/AlignerDataset.py", line 130, in cache_builder_process
wave, sr = sf.read(path)
File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 282, in read
with SoundFile(file, 'r', samplerate, channels,
File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 655, in init
self._file = self._open(file, mode_int, closefd)
File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/soundfile.py", line 1213, in _open
raise LibsndfileError(err, prefix="Error opening {0!r}: ".format(self.name))
soundfile.LibsndfileError: Error opening 'Corpora/Quechua/cus_quz_gui_202071.wav': Format not recognised.
unknown phoneme: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:01<00:00,  2.23it/s] unknown phoneme:
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:01<00:00,  1.65it/s]
unknown phoneme:  33%|██████████████████████████████████                                                                    | 1/3 [00:00<00:01,  1.29it/s]unknown phoneme:
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:01<00:00,  2.42it/s]
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:01<00:00,  2.34it/s]
unknown phoneme:  33%|██████████████████████████████████                                                                    | 1/3 [00:00<00:01,  1.37it/s]unknown phoneme:
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:01<00:00,  2.76it/s]
unknown phoneme: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:01<00:00,  2.83it/s] unknown phoneme:
67%|████████████████████████████████████████████████████████████████████                                  | 2/3 [00:01<00:00,  2.05it/s]unknown phoneme: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:01<00:00,  2.71it/s] unknown phoneme:
unknown phoneme:  33%|██████████████████████████████████                                                                    | 1/3 [00:00<00:01,  1.50it/s]unknown phoneme:
unknown phoneme:  100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:01<00:00,  2.39it/s] 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:01<00:00,  2.28it/s] 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:01<00:00,  2.34it/s] 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:01<00:00,  2.57it/s] 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:01<00:00,  2.73it/s] Converting into convenient format... 100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:00<00:00, 40150.60it/s] 100%|████████████████████████████████████████████████████████████████████████████████████████████████████| 28/28 [00:01<00:00, 19.50it/s] Prepared an Aligner dataset with 28 datapoints in Corpora/Quechua. 0it [00:00, ?it/s] Traceback (most recent call last):   File "run_training_pipeline.py", line 77, in <module>     pipeline_dict[args.pipeline](gpu_id=args.gpu_id,   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/TrainingInterfaces/TrainingPipelines/FastSpeech2_finetuning_example.py", line 36, in run     datasets.append(prepare_fastspeech_corpus(transcript_dict=build_path_to_transcript_dict_quechua(),   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/Utility/corpus_preparation.py", line 38, in prepare_fastspeech_corpus     train_aligner(train_dataset=aligner_datapoints,   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/TrainingInterfaces/Text_to_Spectrogram/AutoAligner/autoaligner_train_loop.py", line 129, in train_loop     loss_this_epoch = sum(loss_sum) / len(loss_sum) ZeroDivisionError: division by zero
I think this last division by zero error is due to the first 'soundfile.LibsndfileError: Error opening 'Corpora/Quechua/cus_quz_gui_202074.wav': Format not recognised.' errors, I don't know why it doesn't recognise the format...

After 1 hour of training I got this new error which has something to do with the espeak installation I did I think:
Epoch:              1945 Spectrogram Loss:   0.20996291935443878 Time elapsed:       74 Minutes Steps:              3890 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  5.41it/s] Traceback (most recent call last):   File "run_training_pipeline.py", line 77, in <module>     pipeline_dict[args.pipeline](gpu_id=args.gpu_id,   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/TrainingInterfaces/TrainingPipelines/FastSpeech2_finetuning_example.py", line 52, in run     train_loop(net=model,   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/TrainingInterfaces/Text_to_Spectrogram/FastSpeech2/fastspeech2_train_loop.py", line 271, in train_loop     path_to_most_recent_plot = plot_progress_spec(net,   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context     return func(*args, **kwargs)   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/TrainingInterfaces/Text_to_Spectrogram/FastSpeech2/fastspeech2_train_loop.py", line 27, in plot_progress_spec     tf = ArticulatoryCombinedTextFrontend(language=lang)   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/Preprocessing/TextFrontend.py", line 168, in __init__     self.phonemizer_backend = EspeakBackend(language=self.g2p_lang,   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/phonemizer/backend/espeak/espeak.py", line 45, in __init__     super().__init__(   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/phonemizer/backend/espeak/base.py", line 45, in __init__     self._espeak = EspeakWrapper()   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/phonemizer/backend/espeak/wrapper.py", line 60, in __init__     self._espeak = EspeakAPI(self.library())   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/phonemizer/backend/espeak/api.py", line 84, in __init__     self._library = ctypes.cdll.LoadLibrary(str(espeak_copy))   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/ctypes/__init__.py", line 451, in LoadLibrary     return self._dlltype(name)   File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/ctypes/__init__.py", line 373, in __init__     self._handle = _dlopen(self._name, mode) OSError: /tmp/tmpswtj5jb9/libespeak-ng.so.1.1.51: failed to map segment from shared object
Any ideas on this? I had trouble at the beginning with installing this version, I followed the instructions here to build and install it from source...

By the way, for all who will see this issue in the future, the only command needed to finetune on your data is:
python run_training_pipeline.py  --gpu_id <the ID(s) of the GPU(s) you are going to use>

Not fully correct, this is usually for training a model from scratch, starting with random initialization. Finetuning would mean that you don't start from scratch, but instead you start from a different model and adapt it to fit the new data. It's just that in the FastSpeech2_finetuning_example.py this is already hardcoded, so you don't need to specify the other arguments that you would usually need (--resume_checkpoint <path to pretrained model> --finetune). I think at some point I will write a script that automates this even further, so that finetuning on a new language becomes super easy. I will try to increase the supported languages to many many more soon.
Regarding your new error: Whenever a checkpoint is saved, a spectrogram is generated and saved as an image to visualize the training progress. Generating this spectrogram is treated the same way as if you were doing inference normally, so the text goes in, is phonemized, then it is synthesized. Your error occurs at this point, where the text is phonemized during inference. The error has however nothing to do with anything that the toolkit does and it also doesn't seem to be an error in espeak either, it probably has something to do with your system. I found a few suggestions here: https://stackoverflow.com/questions/13502156/what-are-possible-causes-of-failed-to-map-segment-from-shared-object-operation
As a sidenote: If you only have enough datapoints to fill two batches per epoch, you have to be careful to not train for too long, because otherwise the model might forget things if it only sees the same few datapoints over and over. In TTS, a lower loss does not necessarily mean a better model. It's best to save checkpoints every few epochs and just try them out to see at which point it stops improving and gets worse again.

I just resume the training later and it worked again, I don't really know why that error ocurred. I would like to know how long should I train it. In your paper it says you trained with only five minutes for new languages with good results, I want to do something similar with Quechua, I found this little dataset on internet and I am trying on it first, but then I will build my own with only 5 minutes as you did in your work...You say in the instructions in the Readme that 200 000 steps are enough for FastSpeech2, but maybe that's too much in this case...I would like to know also if the 5 min datasets you used were of just 1 audio of 5 min or many audios that in total lasted 5 min...

yes, the 200,000 steps are meant for training a model from scratch. Training from scratch however needs a bit more data than just 5 minutes. I found that 30 minutes can already be enough for that.
For finetuning, you only need a few thousand steps, but it might vary depending on the data itself. The data I used were 50 samples that added up to a duration of 301 seconds and I stopped noticing improvements after ~4,000 steps. the performance then got worse at around 60,000 steps because it started overfitting too much.

OK, so my model is finetuned for 7786 steps by now. I tried it with the run_interactive_demo.py script and the results are:

The language seems to be learnt correctly (I can't validate it myself since I don't speak it, but it sounds pretty good).
It's conditioned to the speaker of the dataset (and to the language too). When I tried to clone my own voice with my model, it reproduces the voice of the speaker of the dataset and when I tried to clone in english, it does it too, and it sounds with the Quechua accent! By the way, when I try on the Meta model given by default with my voice, it doesn't clone my voice either.

Sounds great!
Yes, when you only finetune of the 5 minutes, it forgets about multispeaker and multilingual properties. To keep those, you can just jointly train on multispeaker datasets in high-resource languages and mix your 5 minute corpus with low resource data in. Because of the way we sample batches, the small dataset will considered equally as much as the large dataset(s). This way it learns the new language, but keeps the other properties. And about the voice cloning in general: is is not very exact. We're currently working on making it better, the models in the recent release will be updated soon, but it's still far from perfect. It will always just procude a similar voice, not an exact replication, which is also not our goal, as that would make impersonation a bit too easy for my liking. If you train on data created with your own voice, then it will adapt much better, but zero-shot adaptation is a bit limited.

So, I am using the train_loop from meta_train_loop to train on LibriTTS + Quechua (Single Speaker) dataset with the finetuning example script, and I got the following error:
python run_training_pipeline.py fine_qu --gpu_id 0,1
Preparing
Prepared a FastSpeech dataset with 33 datapoints in Corpora/Quechua.
Prepared a FastSpeech dataset with 23854 datapoints in Corpora/LibriTTS.
Training model
0%|                                                                                | 0/100000 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/home/luis/Documentos/VCQuechua/IMS-Toucan/TrainingInterfaces/Text_to_Spectrogram/FastSpeech2/meta_train_loop.py", line 189, in train_loop
batch = next(train_iters[index])
File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 628, in next
data = self._next_data()
File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1306, in _next_data
raise StopIteration
StopIteration
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "run_training_pipeline.py", line 77, in
pipeline_dict[args.pipeline](gpu_id=args.gpu_id,
File "/home/luis/Documentos/VCQuechua/IMS-Toucan/TrainingInterfaces/TrainingPipelines/FastSpeech2_finetuning_example.py", line 52, in run
train_loop(net=model,
File "/home/luis/Documentos/VCQuechua/IMS-Toucan/TrainingInterfaces/Text_to_Spectrogram/FastSpeech2/meta_train_loop.py", line 193, in train_loop
batch = next(train_iters[index])
File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 628, in next
data = self._next_data()
File "/home/luis/Documentos/VCQuechua/IMS-Toucan/toucan_conda_venv/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1306, in _next_data
raise StopIteration
StopIteration
I am using batch size >=34, less than that it throws me CUDA out of memory.
What  should I do?


Legalize b{and,or,xor}_not into component instructions

I was curious to see the effect of #5701 on the compilation of spidermonkey.wasm and much to my dismay I saw that the andn instruction was never generated. It turns out, however, that WebAssembly does not have a "bnot" equivalent instruction for i32 or i64, nor a "band_not" instruction. To trigger the optimization added there I added a few egraph simplification rules to pattern match xor against -1 to bnot.
I then additionally noticed that the lowering added in #5701 was for the clif band_not instruction, but this was never actually used by wasm. Additionally egraphs didn't fuse band-of-bnot into one instruction, so the lowering still never fired. To address this issue I've added legalizations now from band_not into band-of-bnot in addition to bor_not and bxor_not. This involved updating all backends with fused lowerings to do more pattern-matching on the top-level operation instead of matching on b*_not since those now never reach the backends.
After all this I've added a *.wat test to assert that bnot shows up and at least for x86_64 the specialized lowering of band_not is triggered.
Along the way I noticed a few things:

I cleaned up some riscv64 bits where band_not unconditionally used gen_andn but internally gen_andn tested for codegen features. Instead the lowering of (band ... (bnot ..)) now explicitly tests for codegen features and falls back to normal rules if the feature for andn isn't available.
I think the bxor_not instruction was buggy on s390x int that it was calculating (bnot (bxor x y)) instead of (bxor x (bnot y)). The tests got updated as a result. I'm not so certain that the remaining lowerings are correct since the instructions are called NotXor32, for example, but I haven't tested.

Alternatively instead of modifying clif one possibility would be to split apart band_not during either legalization or during egraph optimizations and then have the backend pattern match both band_not and (band (bnot ..))

The general tack we've taken is to decompose ops, so yes, removing the _not variants and making backends match when able to use andn instructions seems like the better approach here. In general the more "normalized" our IR is, the easier it'll be to write rules that apply appropriately. cc @elliottt -- something we can think about next week...

Ok I've pushed up an alternate strategy of reaching my goal:

Legalization has been added to "break apart" the band_not and other related instructions
All backend lowerings for band_not and related instructions are moved up to pattern-matching

A single egraph-rule is retained for x^-1 being translated to ~x, which is enough to trigger wasm modules to use andn on x86_64, for example.
One shortcoming remaining, however, is that the backends currently only pattern match (band x (bnot y)), not (band (bnot y) x), for example. This means that if the order of the operands in the *.wat test I added are swapped then the andn instruction isn't generated. Is this something that should be fixed with an egraph rule? I thought I remembered reading there's a "move constants to the right" rule so should there similarly be a "move bnot to the right" rule? Or should the backends all have duplicate patterns for each order?

One shortcoming remaining, however, is that the backends currently only pattern match (band x (bnot y)), not (band (bnot y) x), for example. This means that if the order of the operands in the *.wat test I added are swapped then the andn instruction isn't generated. Is this something that should be fixed with an egraph rule? I thought I remembered reading there's a "move constants to the right" rule so should there similarly be a "move bnot to the right" rule?

That seems like a reasonable thing to try, yup!

Ok I've pushed up a commit which does that and it seems to work well for at least the small unit tests.

I wouldn't expect a "move bnot to the right" egraph rule to work reliably for backend patterns. It correctly declares that they're equivalent expressions, but then during elaboration we pick one expression as our final choice to hand to the backend, and there's no guarantee we'll pick the one with bnot on the right, since the two alternatives have equal cost. I assume it's working now because elaboration happens to be picking the last-added option in the e-class in case of ties, or something like that.
I think the backends need to match both patterns instead. Which is annoying, since the patterns overlap; we have to give them different priorities.
I also think we need to work on helping ISLE understand commutative operators. It could generate every unique version of a rule with operands swapped, given a commutative flag on terms or something. We'd have to put some thought into how that interacts with overlap checking though.

Ok! I've removed the egraph bits for that and added more lowering rules

I think the bxor_not instruction was buggy on s390x int that it was calculating (bnot (bxor x y)) instead of (bxor x (bnot y)).

Isn't this the same, given that bnot is XOR with -1, and XOR is associative?

Ah yes indeed!


cmake: Add configuration option for FORTIFY_SOURCE

This allows the fortify source option to be disabled if it isn't
desired. Using fortify source against a C library which supports it
increases code size and stack usage significantly. As the minimal C library
doesn't have support, this effect is not visible when using that library.
Signed-off-by: Keith Packard<EMAIL_ADDRESS>

Just noting for future reference that, while it would be nice to add depends on !MINIMAL_LIBC here, minimal libc isn't the only libc that does not support this feature (in fact, the "fortify source" feature is almost exclusive to the GNU toolchains).

Good point. Would it be useful for the toolchain to advertise support and then only enable this selection in that case?

Would it be useful for the toolchain to advertise support and then only enable this selection in that case?

We could add TOOLCHAIN_SUPPORTS_FORTIFY_SOURCE, which gets selected by the relevant toolchain configs; but, all these different toolchain properties are currently work in progress and I would hold off adding one until we have more bird's eye view on the matter.
FYI @tejlmand

A fine plan. We could add the FORTIFY_SOURCE config option ahead of that and then make it conditional based on toolchain support; it's harmless to include the option to the compiler in any case.

For now, I've added depends on !MINIMAL_LIBC to avoid enabling this in that case. Note that this causes a change to the minimal libc builds -- it used to always pass -D_FORTIFY_SOURCE=2 for every build, and that is now removed. It "should" have no effect as there's nothing which should be using that symbol. But it is a change.
I'd like to get this merged at some point as it has a pretty big impact on stack usage and code size when compiling with picolibc or newlib.

So, it sounds like this patch isn't actually useful or necessary?

From the description of this PR and current implementation, I agree.

I'd have to say that allowing developers to enable FORTIFY_SOURCE on production builds would be really nice

There are many compiler flags that can be nice, and Zephyr should for sure support most common schemes for convenience of users, while still consider how to handle differences between toolchain and C libraries.
Options not directly supported through a Kconfig can always be applied using CONFIG_COMPILER_OPT if users has the need.
Up until now I haven't noticed requests for having FORTIFY_SOURCE directly available as setting, so I assume current scheme is satisfactory, and thus I prefer to focus on areas where there are requests for enhancements or features.

Up until now I haven't noticed requests for having FORTIFY_SOURCE directly available as setting when building with optimization, so I assume current scheme is satisfactory to a large majority of users, and thus I prefer to focus on areas where there are requests for enhancements or features.

Hrm. In cmake/compiler/gcc/compiler_flags.cmake:
if(NOT CONFIG_NO_OPTIMIZATIONS)
  # _FORTIFY_SOURCE: Detect common-case buffer overflows for certain functions
  # _FORTIFY_SOURCE=1 : Compile-time checks (requires -O1 at least)
  # _FORTIFY_SOURCE=2 : Additional lightweight run-time checks
  set_compiler_property(PROPERTY security_fortify _FORTIFY_SOURCE=2)
endif()

Unless I'm mis-reading this, it looks like _FORTIFY_SOURCE=2 is getting set whenever optimizations are enabled, which makes sense because the builtins used by that code don't work when optimization is disabled. That means that this patch is still necessary to allow apps to select whether they want to pay the runtime cost of enabling checks.
What we probably want is to always use at least _FORTIFY_SOURCE=1 and make _FORTIFY_SOURCE=2 optional. This would add compile-time checks all the time and then add run-time checks only when desired.

I've rewritten this patch to allow selection of _FORTIFY_SOURCE mode, with the default being compile-time instead of
both compile-time and run-time. _FORTIFY_SOURCE depends on compiler optimization being enabled and not using minimal libc. Mapping of the selection to the correct compiler option is still delegated to the compiler cmake fragment, but that code has been extended to check for the configured mode.

What's going on with the posix native build today? This patch shouldn't have any impact there...

Rebased


Real time web log not work after update to 1.5.2 SSL

Hi developer,I have this problem, please help.
I have two servers, before I upgrade, everything is alright, after both servers upgrade to 1.5.2, the websocket real-time web log not work anymore, I didn't touch anything, just upgrade the version, the page left corner green light turns off,

and the page become to static page,I need to reload the page update the data. let me describe the settings in details.
I use this option to compile 1.5.2
./configure --enable-utf8 --enable-geoip=mmdb --enable-debug --with-getline --with-openssl
two servers os also use ubuntu 21.04
I make goaccess as systemd service run in background：
root@ubuntu:/etc/nginx# cat /etc/systemd/system/goaccess.service
[Unit]
Description=Goaccess Web log report.
After=network.target
StartLimitIntervalSec=0

[Service]
Type=simple
User=root
Group=root
Restart=always
RestartSec=1800
ExecStart=/usr/local/bin/goaccessreport
StandardInput=tty
StandardOutput=tty
StandardError=tty
TTYPath=/dev/tty7

[Install]
WantedBy=multi-user.target

here is the /usr/local/bin/goaccessreport
#!/bin/bash
zcat -f /var/log/nginx/*access.log.* | /usr/local/bin/goaccess /var/log/nginx/*access.log - -o /var/www/goaccess.mysite.com/index.html --ws-url=wss://goaccess.mysite.com --ssl-cert=/root/cert/ecc384.crt --ssl-key=/root/cert/ecc384.key --log-format='%h %^[%d:%t %^] %K/%k "%r" %s %b "%R" "%u"' --date-format=%d/%b/%Y --time-format=%T --real-time-html

nginx.conf log settings：
         log_format proxy '$proxy_protocol_addr - $remote_user [$time_local] '
                     '$ssl_protocol/$ssl_cipher '
                     '"$request" $status $body_bytes_sent '
                     '"$http_referer" "$http_user_agent"';

        access_log /var/log/nginx/access.log proxy;
        error_log /var/log/nginx/error.log;

goaccess service can run without error, and it also LISTEN 7890

When I stop the goaccess service, run it in command:
root@ubuntu:/usr/local/bin# ./goaccessreport
==Using TLS/SSL==
 [PARSING -] {43650} @ {3357/s}.log] {5320} @ {5320/s}
WebSocket server ready to accept new client connections

after I access my sites, and check back the goaccess status, it show the following errors
root@ubuntu:/usr/local/bin# ./goaccessreport
==Using TLS/SSL==
 [PARSING -] {43650} @ {3357/s}.log] {5320} @ {5320/s}
WebSocket server ready to accept new client connections
Accepted: 14 <IP_ADDRESS>
SSL: SSL_accept -> SSL_ERROR_WANT_READ
SSL: incomplete, data available for reading
SSL: SSL_accept -> failed fatal error code: 1
SSL: error:14094416:SSL routines:ssl3_read_bytes:sslv3 alert certificate unknown
SSL: SSL_read -> SSL_ERROR_ZERO_RETURN
SSL: TLS/SSL connection has been closed
SSL: SSL_write -> SSL_ERROR_ZERO_RETURN
SSL: TLS/SSL connection has been closed
SSL: SSL_write -> SSL_ERROR_ZERO_RETURN
SSL: TLS/SSL connection has been closed
SSL: SSL_shutdown -> failed fatal error code: 1
SSL: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
Active: 0

SSL Error？so I check my certificate, ecc384.crt / ecc384.key, It don't have any problem

I don’t know why this happened. There was no problem before the update, but the problem appeared after the update.
Thanks for help first.

Thanks for reporting this. If you are using Lets Encrypt, then it may be relayed to the September 30 issue. You will need to update the certificate chain. Let me know if that helps.

Thanks for reporting this. If you are using Lets Encrypt, then it may be relayed to the September 30 issue. You will need to update the certificate chain. Let me know if that helps.

thanks for your quick reply, I just updated the certificate, actually I really use letsencrypt cert.

and then, restart goaccess and nginx
service goaccess restart
service nginx restart
but the problem still here.

Have you updated CA-Certificates' Server?

 Have you updated CA-Certificates' Server?

Excuse me, I don’t know how to update CA certificate,when I search google, I found that use this command apt update-ca-certificates ?
It didn’t update any certifcate

If I use wrong command, can you tell me how to update？

It only updates when I kill goaccess and start it again and refresh the report page...

I have the same problem

Hi @cwyin7788
Do you check LANG variable enviroment? -- i.e. echo $LANG
Maybe you can set it. --  i.e. export LANG en_US.UTF-8
You must be set encoding to UTF-8, but the language is your choice.

One more thing, note that when you run it via ask, you need to make sure you are using an https connection through the browser and your web socket should be wss instead of ws.

I Have the same or similar problem when trying to use a secure connection.

This is a new install and so unlike the OP, it has not worked for me previously

I too have to stop and restart the goaccess process to update the static report.

@markthompson1 are you able to telnet to that specific endpoint e.g., 209.... using port 7890? please make sure you are accessing the report via https://your_server.com/report.html.
also, does the connection work without wss/ssl? and last, I assume you are only specifying a valid ipv4 address under ws-url... Thanks

Hi @allinurl ,
Yes, the port is open...

And it works for when I disable https and use http only.

I'm trying to reproduce this, not sure yet exactly what's trigerring it, rt.goaccess.io is running v1.5.2 and seems to work fine using a Let's Encrypt certificate. A few questions for everyone:

Can anyone confirm if this was working fine on a previous version, e.g., v1.5 (please build from source)?
If using Let's Encrypt, can I ask you to please try renewing the certificate with --preferred-chain "ISRG Root X1" to exclude expired root from the trust chains.
Please try if the following works (If you see a similar issue then it is Let's Encrypt related and you need to remove expired Root CA)

     $ curl https://curl.se

Please take a look at this post and let me know if that helps solve the issue.

Thanks everyone!

I'm trying to reproduce this, not sure yet exactly what's triggering it, rt.goaccess.io is running v1.5.2 and seems to work fine using a Let's Encrypt certificate. A few questions for everyone:

Can anyone confirm if this was working fine on a previous version, e.g., v1.5 (please build from source)?
If using Let's Encrypt, can I ask you to please try renewing the certificate with --preferred-chain "ISRG Root X1" to exclude expired root from the trust chains.
Please try if the following works (If you see a similar issue then it is Let's Encrypt related and you need to remove expired Root CA)

     $ curl https://curl.se

Please take a look at this post and let me know if that helps solve the issue.

Thanks everyone!

Hi Developer,
We also need to say thank you, thanks for your patiently help,

Can anyone confirm if this was working fine on a previous version, e.g., v1.5 (please build from source)?

I've tried downgrade to 1.5, it's working fine, please see the screen capture.

===============================
Very strange, I just tried to test your suggestion 1.), after I finished the test, I upgrade to 1.5.2 again, and restart the service, It unexpectedly works again (the screen capture below is 1.5.2 and the green light on again)

Here is what I did:
1.)delete /usr/local/bin/goaccess, (1.5.2)，and download 1.5 from source and compile it
2.)restart  goaccess
3.)visit my goaccess site and take a screen capture
4.)delete /usr/local/bin/goaccess, (1.5) and download 1.5.2 from source and re-compile it
5.)restart goaccess
1.5.2 real time web log work again 😳
I don't know why, I didn't touch any certificate stuff.
I will continue to pay attention to see the problem is really gone or not.
I didn't test 2, 3, 4 because it seems solved my problem.

@markthompson1 are you still seeing this issue on your end? If yes, please let me know if following the steps on my previous post help. Thanks!

@cwyin7788 Thanks for the update. Glad it's working for you.

I followed the, downgrade/upgrade steps that worked for the other poster, this didn’t work for me. I’ll try your additional steps about renewing and revoking the “lets encrypt” old certs and report back when I get chance.
Thanks for your support.
Mark
From: "Gerardo O." @.>
Reply to: allinurl/goaccess @.>
Date: Tuesday, 23 November 2021 at 00:13
To: allinurl/goaccess @.>
Cc: Mark Thompson @.>, Mention @.***>
Subject: Re: [allinurl/goaccess] Real time web log not work after update to 1.5.2 SSL (#2202)
@markthompson1 are you still seeing this issue on your end? If yes, please let me know if following the steps on my previous post help. Thanks!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.

Also no livew update here....on two fresh Debian 11.1 installs with Apache2....
goaccess example.net/logs/access.log -o example.net/html_public/report.html --log-format=COMBINED --real-time-html --ssl-cert=/etc/letsencrypt/archive/example.net/cert1.pem --ssl-key=/etc/letsencrypt/archive/example.net/privkey1.pem
No warning/errors reported when starting goaccess...
It only updates when I kill goaccess and start it again and refresh the report page... WS in Chromes inspector jsut shows "pending" as status....

Confirm having the same issue

@tmlnv, which version of GoAccess are you currently running?

@allinurl, 1.8.1
I am on Ubuntu 20 and I've tried a few options:

apt-get install goaccess
Generating report, the report is served via nginx, it is accessible, but not updating.
Official Debian/Ubuntu repo stated here
Same result as 1
Building with ./configure --enable-utf8 --enable-geoip=mmdb --with-openssl
Got this config and the same result

Your build configuration:

  Prefix         : /usr/local
  Package        : goaccess
  Version        : 1.8.1
  Compiler flags :  -pthread
  Linker flags   : -lnsl -lncursesw -lmaxminddb -lcrypto -lssl -lpthread
  UTF-8 support  : yes
  Dynamic buffer : no
  Geolocation    : GeoIP2
  Storage method : In-Memory with On-Disk Persistent Storage
  TLS/SSL        : yes
  Bugs<EMAIL_ADDRESS>
To run used
./goaccess /var/log/nginx/access.log -o /var/www/my_web/report.html --log-format=COMBINED --real-time-html --ssl-cert=/etc/ssl/my_cert.crt --ssl-key=/etc/ssl/my_private_key.txt

or
goaccess /var/log/nginx/access.log -o /var/www/my_web/report.html --log-format=COMBINED --real-time-html --ssl-cert=/etc/ssl/my_cert.crt --ssl-key=/etc/ssl/my_private_key.txt

No green light ever appeared on the dashboard near the gear.

@tmlnv, I'm curious, can you run it without SSL and access the report using a non-HTTPS URL?
# goaccess /var/log/nginx/access.log -o /var/www/my_web/report.html --log-format=COMBINED --real-time-html

@allinurl Yes, I can.
I get:
WebSocket server ready to accept new client connections

However, the report stays static and does not update.
P.S.
I have set up a cron job to execute a script for the creation of static reports from both .log and .gz log files regularly. This configuration fulfills all my needs. Therefore, no additional support is needed.
Thank you for your efforts and responses!

It seems that this issue maybe isolated to your environment. My guess is it could be the network or some missing configuration. Either way, thanks for giving me the heads up, and good to know you found a workaround. Feel free to reopen it if you need to.


Revert #166 and add explanation why the formula is in fact correct.

Reverts prometheus/docs#166
This is frequently confused so it should be explained, but the original formula is correct. The reason is that the histogram buckets are cumulative; the le="0.3" bucket is also contained in the le="1.2" bucket; dividing by 2 corrects for that.
Add an explanation.

:+1: thanks for walking through the old stuff and catching this.


add tuning template

Hello @rasbt! Thanks for opening this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file templates/pytorch_lightning/tune_classification_basic.py:

Line 17:80: E501 line too long (88 > 79 characters)
Line 22:80: E501 line too long (86 > 79 characters)
Line 205:80: E501 line too long (89 > 79 characters)


Wizard discardProgress

Hi,
On MacOSX 10.11 bèta (15A278b) safari emulating IE 10
As well as on Windows with IE11
(On Chrome it works 100% of the time)
The function discardProgress of sap.m.Wizard is not working correctly
    this._wizard.discardProgress(this.getView().byId("wiz1"));
    this.getView().byId("SettingsWizard").discardProgress(this.getView().byId("wiz1"));

This is throwing an error on _getStepScrollOffset with:
    TypeError: undefined is not an object (evaluating 's.$().position().top')
_getStepScrollOffset
goToStep
_handleStepChanged
(anonieme functie)
fireEventsap-ui-core.js:142:70157
fireEventsap-ui-core.js:151:133004
(anonieme functie)sap-ui-core.js:151:39662
_updateCurrentStep
discardProgress
discardProgress
wizardCompletedHandler
fireEventsap-ui-core.js:142:70157
fireEventsap-ui-core.js:151:133004
(anonieme functie)sap-ui-core.js:151:39662
_handleNextButtonPress
(anonieme functie)
fireEventsap-ui-core.js:142:70157
fireEventsap-ui-core.js:151:133004
(anonieme functie)sap-ui-core.js:151:39662
ontap
_callEventHandlessap-ui-core.js:151:131004
_handleEventsap-ui-core.js:151:130686
_handleEventsap-ui-core.js:151:176640
psap-ui-core.js:59:4274
dispatchsap-ui-core.js:70:39408
g
p
dispatchsap-ui-core.js:70:39408
handlesap-ui-core.js:70:35830
triggersap-ui-core.js:70:38467
(anonieme functie)sap-ui-core.js:70:46747
eachsap-ui-core.js:59:3191
eachsap-ui-core.js:59:902
triggersap-ui-core.js:70:46720
P
R
dispatchsap-ui-core.js:70:39408
handlesap-ui-core.js:70:35830

If more information is required let me know.
Kind regards,
Max

Hello,
Can you tell us a bit more about your setup since I tested it on WIN 8.1 and IE11 but could not reproduce the issue.

Hi,
I have the following:
In the view i have about 7 Wizard steps. Each with their own content. Which I have removed here since I believe they are not part of the issue.
<mvc:View class="viewPadding" controllerName="controller.Wizard" xmlns:core="sap.ui.core" xmlns:u="sap.ui.unified" xmlns:l="sap.ui.layout" xmlns:f="sap.ui.layout.form" xmlns:m="sap.m" xmlns:mvc="sap.ui.core.mvc" xmlns:vh="sap.ui.commons.ValueHelpField" >
    <m:Page id="pageid" showNavButton="false" title="{i18n>Wizard_Title}">
		<m:content>
			<m:Wizard complete="wizardCompletedHandler" id="SettingsWizard" stepIcons="sap-icon://form" >
				<m:WizardStep id="wiz1" title="{i18n>Wizard_Wiz1Title}">
                    <f:SimpleForm class="editableForm" columnsL="1" columnsM="1" editable="true" emptySpanL="1" emptySpanM="1" id="Wiz1Form" labelSpanL="4"
						labelSpanM="4" layout="ResponsiveGridLayout" maxContainerCols="2" minWidth="1024">
						<f:content>
                   <!--- content here --->
                        </f:content>
					</f:SimpleForm>
				</m:WizardStep>
            <!--- more WizardSteps --->
			</m:Wizard>
		</m:content>
	</m:Page>
</mvc:View>

In the controller I'm using the function wizardCompletedHandler. This is the same function I've declared in the view to be in charge of the complete event.
	onInit: function() {
				this._wizard = this.getView().byId("SettingsWizard");
				this.getView().setModel(Appconfig.getModel(), "Config");
				var oRouter = sap.ui.core.UIComponent.getRouterFor(this);
				oRouter.getRoute("Wizard").attachMatched(this._onRouteMatched, this);

			},
			wizardCompletedHandler: function() {
				Appconfig.updatePersistent();
				sap.ui.core.UIComponent.getRouterFor(this).navTo("Tiles");
				// this._wizard.goToStep(this.getView().byId("wiz1"));
				// this._wizard.discardProgress(this.getView().byId("wiz1"));
				this.getView().byId("SettingsWizard").discardProgress(this.getView().byId("wiz1"));
			},

Kind regards,
Max

Thank you Max,
opened internal issue<PHONE_NUMBER> for this finding.
Best Regards,
Martin

Issue confirmed.
What I think could be the problem is this :
sap.ui.core.UIComponent.getRouterFor(this).navTo("Tiles"):
You are navigating to another place in your page, and this hides the wizard, and sets the visibility internally to false
Then when you call  the goToStep(internally used by discardProgress) method, it is not working, because it can't get the top position of the invisible step element.
Issue is also reproducible when you call wizard.setVisible(false), wizard.goToStep(wizard.getSteps()[0]).
What you could try for this time is first call discard, and then move away from the page.
We are going to provide fix for this case as well.
BR,
Shtiliyan

Hi Shtiliyan,
What you stated is correct. This works now. But I assume it needs to work the other way around too?
I've not tested the invisible part. But the progress is reset and the steps are back to 1.
Thanks!
Kind regards,
Max

Hi,
I think this old issue was not maintained correctly, as the internal issue is closed with fix.
Best Regards,
Nikolay


CAPZ certificate management

Epic
CAPZ workload clusters certificate management.
- [ ] https://github.com/giantswarm/roadmap/issues/2474
- [ ] https://github.com/giantswarm/roadmap/issues/2476
- [ ] https://github.com/giantswarm/roadmap/issues/2502
- [ ] https://github.com/giantswarm/giantswarm/issues/15981
- [ ] https://github.com/giantswarm/giantswarm/issues/15976
- [ ] Deploy certificate management changes to all CAPZ clusters

https://github.com/giantswarm/roadmap/issues/2474


[Lake][ETL] ETL.md - Explain architecture & ux end-to-end

Background / motivation
To integrate duckDB (#685)[https://github.com/oceanprotocol/pdr-backend/issues/685] the user should be able to follow the README.md, get their lake setup, and complete all the work end-to-end without any issues.
Further, if the user should experience any issues, they should be able to resolve/heal the process effortlessly.
DoD
Write README.md for updated lake, and verify that it's working end-to-end
Tasks:

[ ] Write new lake README.md
[ ] Test README.md end-to-end
[ ] Documentation is working as expected

Reviewed the README and have some feedback:

[ ] Add information about where the data comes from and what kind of data we are ingesting into the lake
[ ] I suggest to move the 'use analytics' part into a different README, keeping the lake-and-etl strictly about describing and using the lake + etl
[ ] There is no information about the data available, tables etc. I would expect to visually see the data that I'm going to get before going trough the lake setup process
[ ] Add a simple example containing the command the users has to execute to go trough the etl end to end
[ ] The README is long and we don't have any legend on the top to make it easy to see the content and navigate trough different sections. Add a legend on the top of the page, example:

Readme has been concluded in PR for issue #1001 -> https://github.com/oceanprotocol/pdr-backend/pull/1100


Update System requirement guidelines section in README

Is your feature request related to a problem? Please describe.
Our first alpha release should be able to run on a wide range of systems. We want to make this an explicit requirement for our first alpha release. However we still need to provide some guidelines on what the minimum system requirements should be for someone running zebra.
Describe the solution you'd like
We should define what the minimum system requirements should be for our first alpha.

let's remove this from the release criteria and just add to the README


After adding my own firebase file app starts crashing at workoutNow.getDays?

This is because your database on firebase is probably wrong and doesnt fit the scheme of my functions.
Please make your database and adjust the functions accordingly, start with a small DB there is no reason for you to handle the same amount if levels as in my app.
Here is an example of a basic json file that will work
{"exercises": [ {
      "id" : 1,
      "name" : "Downword Dog"
    }, {
      "id" : 2,
      "name" : "Warrior Stance"
    } ],
    "id" : 100,
    "name" : "Yoga"
  } ]

the above json is invalid to start

kindly give basic correct json in order to work more on the app
thanks .

The above json is an example of a basic json to start with, this is exactly how part of the DB looks like, the other parts are the User DB and the User Programs DB.
I don't want to sound rude, but it sounds like you just trying to copy paste the entire app without understanding how it works, and the app is 99% available here, the only missing part is the json which you need to build yourself from the documentation I listed in your other issue.
You can start by copying the DB I showed above, and build the correct functions to get the data from the DB, this is exactly 1 line of code.


Add device info for LIFX

Description:
If new models are released, this will return 'model': None until aiolifx is updated, is that okay?
Checklist:

[X] The code change is tested and works locally.
[X] Local tests pass with tox. Your PR cannot be merged unless tests pass

If the code communicates with devices, web services, or third-party tools:

[X] New dependencies have been added to the REQUIREMENTS variable (example).
[X] New dependencies are only imported inside functions that use them (example).
[X] New or updated dependencies have been added to requirements_all.txt by running script/gen_requirements_all.py.
[X] New files were added to .coveragerc.

That's not okay, setting model: None will override the device registry and reset model

So what is the correct thing to do for an unknown/new model?

Oh, I think that I was unclear.
If new models do come out, we are unable to map their product ID to a model name until aiolifx is updated. Thus, None will be returned for these as of yet unknown models.
Existing entities, however, will keep their current model value.

Oh, I guess in that case it's fine actually, as the model wouldn't have any info otherwise.

The best practice is if you don't know a piece of information, don't send it. Because if you send None, we will remove any data that is currently there.


"Lexical with name '$a' does not exist in this frame"

See https://stackoverflow.com/questions/50221466/assignment-to-nested-lists-in-perl-6/50222378?noredirect=1#comment87463338_50222378

The Jupyter kernel uses the internal REPL class -- so the behavior of it will be the same as the built-in REPL.  As a workaround (for either one), you could put the declarations and assignment on the same line (or in the same cell), e.g.
In [1]: my ($a,$b,$c); :(($a,$b), $c) := ((1,2), 3);
Out[1]: ($ ($a, $b), $c)

In [2]: $a, $b, $c
Out[2]: (1 2 3)


Config keeps restarting.

Every time I restart my server my config keeps restarting. This started happening when I added a new class. If anyone knows what source this starts from or anyone who has had this before, please any help I would be grateful for.

I recommend trying with a fresh install of the latest Clockwork release if you're experiencing this.


Unable to use SSLKEYLOGFILE option

As i wanted to look at the decrypted packets using wireshark, i tried using the below:
export SSLKEYLOGFILE=<quic_keylog_file>, which should work for boring ssl
https://github.com/google/boringssl/search?q=SSLKEYLOGFILE&type=code
But it didn't work and i didn't get the keylog file.
Should we change the cmake file to include boringssl tool/client.cc to make it work and obtain SSLKEYLOGFILE?

You can use SSL_CTX_set_keylog_callback to generate keylog. For example:

Thank you @MacroLau  for the above. It works well! I'll go ahead and and give an option to specify the filename to write into as well.

@MacroLau This works well when we haven't specified the server's quick version. Once I use the --quic_versions="h3-Q043" no logs are generated. Do you know why?


Add API Server support.

Add API Server support to test framework.
Add a basic example under integration2 folder.
Minor refactoring/code-move.

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: ozevren
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers: chxchx, geeknoid
If they are not already assigned, you can assign the PR to them by writing /assign @chxchx @geeknoid in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

pkg/OWNERS
tests/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


Error: The XMLHttpRequest compatibility library was not found.

Hi guys
I've recently updated my angularfire2 to version RC6, which stopped my webapp from being served as SSR webapp. I've provided information below. I hope this will suffice. :)
Version info
"dependencies": {
    "@angular/animations": "^5.0.0",
    "@angular/common": "^5.0.0",
    "@angular/compiler": "^5.0.0",
    "@angular/core": "^5.0.0",
    "@angular/forms": "^5.0.0",
    "@angular/http": "^5.0.0",
    "@angular/platform-browser": "^5.0.0",
    "@angular/platform-browser-dynamic": "^5.0.0",
    "@angular/router": "^5.0.0",
    "@firebase/app": "^0.1.8",
    "angularfire2": "5.0.0-rc.6",
    "core-js": "^2.4.1",
    "firebase": "4.9.1",
    "path": "^0.12.7",
    "rxjs": "^5.5.1",
    "zone.js": "^0.8.14"
}

How to reproduce these conditions
Install angularfire2 with firebase as per my dependency versions.
Steps to set up and reproduce
I've setup a firebase function which serves my application like shown in the video from firebase -> Link to firebase tutorial
When trying to access my site online, it comes out with a 503 - first byte timeout
This did not appear until i updated angularfire2 to version rc6.
Since then i've had this error.
Note that i've tried with multiple versions of "firebase" this includes 4.8.0, 4.8.2 and 4.9.0 without luck. Everything compiles perfectly when running ng serve.
Debug output
I've checked the functions logs and this error comes out.
 Unhandled Promise rejection: The XMLHttpRequest compatibility library was not found. ; Zone: <root> ;
Errors in the JavaScript console
GET https://path-to-site 503 ()
Screenshot
Screenshot 1

Screenshot 2

Screenshot 3

Expected behavior
Should serve the webapp
Actual behavior
Comes out with service unavailable and function log errors
I hope i've provided sufficient data for you guys to take a look at. I can provide more information if need be.
Kind regards Chris

Hey @chriseenberg! AngularFire currently doesn't work with Angular Universal but we are very close!
I'm not sure what you were doing to get it working before, but it shouldn't have been. I hope we are only around a week away from getting Universal support released to a next tag. I'm closing this issue to keep the conversation confined the the PR I linked above. Feel free to follow that and post any comments or feedback there.

@davideast  I am having the same issue too. Eagerly waiting for the universal addon

having the same issue to use angularfire with SSR. Does this seem to be the current case still? does the angularfire library intend to support SSR support or still pending PR?

Check out angularfire2@next for SSR
On Tue, Apr 17, 2018, 3:31 AM Erick Ballesteros<EMAIL_ADDRESS>wrote:

having the same issue to use angularfire with SSR. Does this seem to be
the current case still? does the angularfire library intend to support SSR
support or still pending PR?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/angular/angularfire2/issues/1461#issuecomment-381920455,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AACvr33gVi3XCWb7F5hy2kBvSZmOmhUZks5tpbZugaJpZM4R_5nb
.

I have the same issue. There's any solution for running AngularFire2 with Angular5 and Universal?

@davidmartinezros See the comment above yours about installing angularfire2@next. It should solve this problem.


failed to load external entity "http://docbook.sourceforge.net/release/xsl/current/manpages/docbook.xsl"

Before you submit an issue, please review the guidelines for this repository.
10:44:40 ┌ tesseract git ............................... [Updates found]
10:44:44 ├ Running autogen...
10:46:41 ├ Running configure...
10:51:18 ├ Running make...
Likely error:
warning: failed to load external entity "http://docbook.sourceforge.net/release/xsl/current/manpages/docbook.xsl"
cannot parse http://docbook.sourceforge.net/release/xsl/current/manpages/docbook.xsl
close failed in file object destructor:
sys.excepthook is missing
lost sys.stderr
make[2]: *** [Makefile:618: combine_lang_model.1] Error 4
make[2]: Leaving directory '/build/tesseract-git/build-64bit/doc'
make[1]: *** [Makefile:494: all-recursive] Error 1
make[1]: Leaving directory '/build/tesseract-git/build-64bit'
make: *** [Makefile:403: all] Error 2
make failed. Check C:/Users/kuh3h/Downloads/media-autobuild_suite/build/tesseract-git/build-64bit/ab-suite.make.log
This is required for other packages, so this script will exit.
11:11:42   Creating diagnostics file...
All relevant logs have been anonymously uploaded to https://0x0.st/zsPU.zip
Copy and paste logs.zip in the GitHub issue.

fixed by install docbook-xsl


use timings to time some stuff

how to time: small wrapper api ontop of aikairs timings, implement in modules.
stuff we can time:

command execute
command registering
event registering
module loading
game load


Flyway prompts for a license key, the next day after install

flyway clean
The system cannot find the file specified.
Error occurred while processing: 03-30-2023_.
The system cannot find the file specified.
Error occurred while processing: 8-34-46.81.
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
ERROR: Missing license key. Ensure flyway.licenseKey is set to a valid Flyway license key ("FL01" followed by 512 hex chars)

This is supposed to be a free, community version

From the output I can see here, it looks like you are running the paid edition, which usually will drop down to the community edition in the absence of a license key (there is another download that only has community artefacts).
Unfortunately there is currently a current bug with this fallback logic which will be resolved in the next release. If you use the pure community artefacts from here, this issue will not impact you.

Thank you


Implements Student Topic Link Adaptation

Implements Link Adaptation at a Topic Level

Task linked: CU-3e69xk2 Implement Student Topic Navigation Link Adaptation


Unable to accept cookies in mobile view

Describe the bug
When browsing a page with cookies in mobile view, it is not possible to tap the buttons in the consent dialog to accept or decline cookies.
To Reproduce
Steps to reproduce the behavior:

Go to 'https://energyvision.app.radix.equinor.com/careers'
Click on the 'Cookie Setting' button
Scroll down to 'Try taping any of the buttons'
Nothing happens

Expected behavior
Consent is given and dialog is closed
Screenshots

Desktop (please complete the following information):

OS: ChromeOS
Browser chrome
Version 97

Seems to work most of the time in Chrome on my iPhone 13. On a few occasions the buttons were not tab-able.


Generate meaningful names with from_list()

We could create more meaningful names instead of using the index as name:
For example similar to how Tensorflow handles that: It uses the type of node to create the names. For example:
The first instance of Dense will be called "dense", the second "dense_1" etc.

I see how it would make sense to have more meaningful names via the from_list constructor.
I'm not sure I'm a fan of the indexing, though. Doesn't that imply there is an ordering based on each node type? That dense comes before dense_1?
What about retaining the index across node types? Given the graph is constructed from a list, wouldn't it make sense to have a graph Linear -> Linear look like this? Input_0 -> Linear_1 -> Linear_2 -> Output_3?
Concretely, it could look something like this:
return NIRGraph(
    nodes={f"{type(n).__name__}_{str(i)}": n for i, n in enumerate(nodes)},
    edges=[(str(i), str(i + 1)) for i in range(len(nodes) - 1)],
)

Thank you for your proposal, that is of course another option. That has the nice benefit of not requiring a per-node-type counter.
However, it might also get confusing if you have a graph Dense->LIF->Dense. Then you would have Input_0->Dense_1->LIF_2->Dense_3->Output_4, with Dense_1 and Dense_3, but no Dense_2.
In the Tensorflow-like naming, that would be input->dense->lif->dense_1->output.
I do not see the ordering implication as an issue here. But maybe that is also because I am used to the way Tensorflow does the default naming. Also this default naming is only for the from_list which can only create a sequential graph, giving a deterministic naming.
Addendum: In your proposal the edges also need the changed names :)
return NIRGraph(
nodes={f"{type(n).name}{str(i)}": n for i, n in enumerate(nodes)},
edges=[(f"{type(nodes[i]).name}{str(i)}", f"{type(nodes[i+1]).name}_{str(i)}") for i in range(len(nodes) - 1)],
)

I honestly don't have strong opinions here. I can see both systems working. If you're happy with the Tensorflow style, I would be fine to sign off on that :-)


2016 Update?

Hey man,
Has this changed in the 2 years since you wrote it. I'm in the somewhat glorious position of only needing to support IE11+ and the last 2 versions of all the other browsers.
Would you recommend any changes to your query function?
Thanks,

@matt3224 The query function is so simple, it's future proof, so no updates are necessary. You may want to check out my other project sonic which is a context-aware and extentable CSS selector engine built on top of the native functions, similar to query.


Add option to edit the owner

Is it possible to add an option to edit the owner of tables/functions/views/etc? Just like the one used to rename the table, but for the owner instead?

This depends on database. But usually this is not supported because it is actually a move operation (move table from one schema to another).
In most cases you will need to create new table, recreate all constraints and properties, then move data from original table to a new one and then drop original table.
That's complicated, not safe and quite fragile operation for simple property value change.
So..

Whether this has been added to dbeaver and which databases support this ?
I tried PostgreSQL and Oracle, but I can't edit the owner of tables


Roll Flutter from 66ed64be4fd8 to 35b18ba2e902 (70 revisions)

Roll Flutter from 66ed64be4fd8 to 35b18ba2e902 (70 revisions)
https://github.com/flutter/flutter/compare/66ed64be4fd8...35b18ba2e902
2022-04-05<EMAIL_ADDRESS>Made flag for debugging build time of user created widgets (flutter/flutter#100926)
2022-04-05<EMAIL_ADDRESS>Add note to doctor validator if script is running Rosetta (flutter/flutter#101309)
2022-04-05<EMAIL_ADDRESS>[ci.yaml] Revert every target except roller back to luci scheduler (flutter/flutter#101323)
2022-04-05<EMAIL_ADDRESS>Use separate metric file names for same tests running on different platforms (flutter/flutter#101317)
2022-04-05<EMAIL_ADDRESS>Generate a demo for new icons when updating icons (flutter/flutter#101190)
2022-04-05<EMAIL_ADDRESS>Only run test harness tests once instead of every shard (flutter/flutter#101218)
2022-04-04<EMAIL_ADDRESS>Roll i10n localizations. (flutter/flutter#101310)
2022-04-04<EMAIL_ADDRESS>Roll Engine from be4ce8d88362 to e7e7ca12529f (5 revisions) (flutter/flutter#101319)
2022-04-04<EMAIL_ADDRESS>Remove gems dependency from tests that do not use it (flutter/flutter#101210)
2022-04-04<EMAIL_ADDRESS>Marks Mac_ios native_ui_tests_ios to be unflaky (flutter/flutter#97670)
2022-04-04<EMAIL_ADDRESS>Implements a PlatformMenuBar widget and associated data structures (flutter/flutter#100274)
2022-04-04<EMAIL_ADDRESS>Remove dead dartdocs.dart file (flutter/flutter#101215)
2022-04-04<EMAIL_ADDRESS>[Revert] Skip overall_experience_test.dart: flutter run writes and clears pidfile appropriately (flutter/flutter#101267)
2022-04-04<EMAIL_ADDRESS>Fix crashes when current route parsing transactions are discarded (flutter/flutter#100657)
2022-04-04<EMAIL_ADDRESS>Marks Windows module_custom_host_app_name_test to be unflaky (flutter/flutter#99407)
2022-04-04<EMAIL_ADDRESS>Marks Windows module_test to be unflaky (flutter/flutter#99408)
2022-04-04<EMAIL_ADDRESS>Reland "Allow unknown device kind to scroll scrollables (#100800)" (flutter/flutter#101301)
2022-04-04<EMAIL_ADDRESS>Roll Engine from 801d8c530fc5 to be4ce8d88362 (2 revisions) (flutter/flutter#101219)
2022-04-02<EMAIL_ADDRESS>801d8c530 Set iOS default frame rate to screen max. (flutter/engine#29797) (flutter/flutter#101216)
2022-04-01<EMAIL_ADDRESS>Roll Engine from 5f2b73e49eca to 69138cca1354 (4 revisions) (flutter/flutter#101213)
2022-04-01<EMAIL_ADDRESS>Roll Engine from 76b7da7a3d91 to 5f2b73e49eca (2 revisions) (flutter/flutter#101206)
2022-04-01<EMAIL_ADDRESS>Roll Engine from 045394787dfe to 76b7da7a3d91 (1 revision) (flutter/flutter#101202)
2022-04-01<EMAIL_ADDRESS>[web] reenable canvaskit benchmarks (flutter/flutter#101155)
2022-04-01<EMAIL_ADDRESS>Roll Engine from 0b20b9d14582 to 045394787dfe (2 revisions) (flutter/flutter#101201)
2022-04-01<EMAIL_ADDRESS>Roll Plugins from b3e37559f966 to 7d1923f4528e (2 revisions) (flutter/flutter#101193)
2022-04-01<EMAIL_ADDRESS>Roll Engine from 276261e5d8d2 to 0b20b9d14582 (1 revision) (flutter/flutter#101189)
2022-04-01<EMAIL_ADDRESS>Roll Engine from c3274494b0c8 to 276261e5d8d2 (4 revisions) (flutter/flutter#101188)
2022-04-01<EMAIL_ADDRESS>Roll Engine from 7acec05e9001 to c3274494b0c8 (1 revision) (flutter/flutter#101172)
2022-04-01<EMAIL_ADDRESS>Roll Engine from c2de44a59cf1 to 7acec05e9001 (1 revision) (flutter/flutter#101169)
2022-04-01<EMAIL_ADDRESS>Roll Engine from 3ffdd71530ee to c2de44a59cf1 (2 revisions) (flutter/flutter#101167)
2022-04-01<EMAIL_ADDRESS>Roll Engine from 503cc732413a to 3ffdd71530ee (2 revisions) (flutter/flutter#101165)
2022-04-01<EMAIL_ADDRESS>Roll Engine from 76f4c1b526fb to 503cc732413a (2 revisions) (flutter/flutter#101164)
2022-04-01<EMAIL_ADDRESS>Roll Engine from e8376c4ee31b to 76f4c1b526fb (1 revision) (flutter/flutter#101162)
2022-04-01<EMAIL_ADDRESS>Roll Engine from 848f6097e404 to e8376c4ee31b (3 revisions) (flutter/flutter#101159)
2022-04-01<EMAIL_ADDRESS>Roll Engine from a571b34fdb5b to 848f6097e404 (3 revisions) (flutter/flutter#101156)
2022-04-01<EMAIL_ADDRESS>Roll Engine from 7ebc7736f447 to a571b34fdb5b (3 revisions) (flutter/flutter#101153)
2022-03-31<EMAIL_ADDRESS>[web] fix CanvasKit benchmarks (was corrupted test PNG) (flutter/flutter#100918)
2022-03-31<EMAIL_ADDRESS>Roll Engine from 54b098fa03ec to 7ebc7736f447 (1 revision) (flutter/flutter#101148)
2022-03-31<EMAIL_ADDRESS>Roll Engine from 959dee969d5e to 54b098fa03ec (2 revisions) (flutter/flutter#101143)
2022-03-31<EMAIL_ADDRESS>Refactor ToggleButtons (remove RawMaterialButton) (flutter/flutter#99493)
2022-03-31<EMAIL_ADDRESS>Roll Engine from 93d6c705d050 to 959dee969d5e (1 revision) (flutter/flutter#101134)
2022-03-31<EMAIL_ADDRESS>Remove unintended comment (flutter/flutter#101130)
2022-03-31<EMAIL_ADDRESS>[ci.yaml] Revert all devicelab tests back to LUCI scheduler (flutter/flutter#101124)
2022-03-31<EMAIL_ADDRESS>Fix Local DataTableTheme not being inherited by DataTable Widget (flutter/flutter#101112)
2022-03-31<EMAIL_ADDRESS>migrate to super params (flutter/flutter#100509)
2022-03-31<EMAIL_ADDRESS>Revert "Allow unknown device kind to scroll scrollables (#100800)" (flutter/flutter#101129)
...

AutoRoller is stopped; closing the active roll.


Spelling corrections/ elongating a name

Hi there, My friend recently discovered this for there own use, and i being curious wanted to test it out for my self. My Name is Georges, which you might notice has an s at the end, its silent so it oft gets forgotten in correspondence. Interested in using your software for that purpose i tried to have Georges replaced with Georges. Ngl it was beautiful to watch the google results fill with "sssssssssssssssssssssssssssssssssssssssssss" when the software registered the "George" inside of Georges and replaced it repeatedly. I realize my desire is not really in the scope of your product base, but if someone wanted to change from Dan to Daniella I think they would run into the same problem. An idea might be to somehow frame the dead-name, so if it registered as part of another word it wouldn't be flagged and replaced.
This is an awesome extension I wish you guys the best of luck

Oh, I opened an issue that is the same as this one: #37. It's also related to this one #65

This should be resolved in the newest version (1.2)


Which Torch Version

Which Torch version did you use during developing a project?

Hi @Zekhire ,
I'm really sorry, but I don't have the access to the development environment yet, so I can't tell for sure. I assume it's something around torch 1.4 or 1.5, but well, I don't know :)
Best,
D


user logs add source beside tool

add source labels on logs
add source label on graphManagement

duplicate #728


Prototype using maven-resource-plugin filtering to avoid duplicating …

…module.xml files

WF Preview tests:
https://ci.wildfly.org/viewLog.html?buildId=267256&tab=buildResultsDiv&buildTypeId=WF_WildFlyPreviewLinuxJdk11
https://ci.wildfly.org/viewLog.html?buildId=267254&tab=buildResultsDiv&buildTypeId=WF_Ee9BootableJarLinuxJdk8

@jamezp or @ronsigal Is this approach to configuring this module ok with you?

@jamezp or @ronsigal Is this approach to configuring this module ok with you?

@bstansberry looks good to me.


Deprecation warning for using 'method_whitelist' with Retry from urllib3

Hi,
There is a deprecation warning for using 'method_whitelist' with Retry from urllib3.
/django_auth_adfs/config.py:192
DeprecationWarning: Using 'method_whitelist' with Retry is deprecated and will be removed in v2.0. Use 'allowed_methods' instead

For more info, have a look at their documentation / code:

https://pypi.org/project/urllib3/1.26.0/
https://github.com/urllib3/urllib3/blob/1.26.7/src/urllib3/util/retry.py#L241
https://github.com/urllib3/urllib3/blob/1.26.7/src/urllib3/util/retry.py#L252

Thanks!
Best regards
Koen de Leijer

Hello! Thank you so much for a well written issue report😊
Would you be interested in creating a pull request?

Hello! Thank you so much for a well written issue reportblush
Would you be interested in creating a pull request?

Yes I am. Never worked with poetry before, but I will give it a try

We're happy to help! A short introduction can be found here. 😊

Closed by #188


[iOS] can't get notification banner when app is in foreground

What version of RN and react-native-fcm are you running?
RN: 0.53.2 --- RN fcm: 13.3.1
What device are you using?
iOS 11 physical device
Is your app running in foreground, background or not running?
foreground

When I have the app running in the foreground, I get everything but the banner.  When in the background the banner will show up just fine.  However when in the foreground, I get the notification , vibration, and sound but no banner.  below is what I am sending in the local notification:
FCM.presentLocalNotification({               id: uniqueId,               title: notif.aps.title,               body: notif.aps.body,               sound: "default",               priority: "high",               click_action: "com.willcalled.Chat",               badge: 10,               number: 10,               show_in_foreground: true,             });
I have set FirebaseAppDelegateProxyEnabled to both Yes and No with no difference.
Not really sure where to go from here, and any help would be amazing.
thank you!

are you trying to build local notification for remote notification?

yes. if I don't build the local notification, then I get nothing but the raw notification data when the app is in the foreground.  So the notification comes in just fine, and I can log out the contents of it, but I don't get a vibration, sound or banner unless I do the local notification.  When I build the local notification, I do get the vibration, sound and data, but no banner.
Am I supposed to be getting the entire notification (banner and all) when the app is in the foreground without building a local notification?

you should get notification data when app is in foreground unless you return WillPresentNotificationResult.None in iOS 11

correct.  I do get the notification data.  when I build the local notification so that the banner is shown while using the app, I only get the vibration and sound.  The banner is never shown.

Ok that was the problem.  I was incorrectly setting notification result. Thank you for your help!

@n8stowell82 you were to show a banner when app is in foreground? How?


dashboard: node details: node resources

order: cru, mru, sru, hru (better to use Cores, RAM, SSD, HDD)
hovering for monitoring and replace with more appropriate wording

@rkhamis @amiraabouhadid any help with the wording?

@A-Harby CPU remained as it is and changed Monitoring url to Check Node Metrics.