Input data file only 24M, but cost about 90G memory while converting to unicode in the load_20newsgroups.py step

I replaced the 20newsgroups data file to my own data file which size is about 24M, when I ran the load_20newsgroups.py, it cost about 90G memory, why this implementation costs such big memory???

Wow that's a lot of memory. Are you talking about in memory usage or storage size?? That seems really high.
I will say, the implementation I've used in Spacy is computationally expensive. If we could figure out a smarter way to map Spacy hashes to embedding matrix index values, that would greatly reduce memory usage and preprocessing time. This is definitely the bottleneck in the preprocessing. I've found no way to do this online, so I hacked together my own way in the nlppipe.py file.

Wow that's a lot of memory. Are you talking about in memory usage or storage size?? That seems really high.
I will say, the implementation I've used in Spacy is computationally expensive. If we could figure out a smarter way to map Spacy hashes to embedding matrix index values, that would greatly reduce memory usage and preprocessing time. This is definitely the bottleneck in the preprocessing. I've found no way to do this online, so I hacked together my own way in the nlppipe.py file.

I found that you init the matrix with n*m size, where n is the document size and m is the hyperparameter, the matrix cost too much memory...
And I have a question about lda2vec, Is the preprocess step generating the lda and word2vec results, and then combine these result to train the lda2vec model in running process ? I read the paper, but did not understand where lda and word2vec results coming from.

Is there spark implementation of lda2vec?

No there is no spark implementation. Not sure what you mean by m. M is the embedding size (as far as I understand from what you said). No way to make this smaller. Thing that might affect lda2vec would be preprocessing. I don't know how to program in spark, but I am open to learning it if it applies

No there is no spark implementation. Not sure what you mean by m. M is the embedding size (as far as I understand from what you said). No way to make this smaller. Thing that might affect lda2vec would be preprocessing. I don't know how to program in spark, but I am open to learning it if it applies

yeah, m is the embedding size.
Is the preprocess step generating the lda and word2vec results, and then combine these result to train the lda2vec model in running process ? I read the paper, but did not understand where lda and word2vec results coming from.
Is the preprocess step generating the w2v and raw lda results?

Try reducing batch size from 10,000 to 500 in  nlppipe.py in the tokenizer() method on line #184:
E.g. -
for row, doc in enumerate(self.nlp.pipe(self.texts, n_threads=self.num_threads, batch_size=500)):

@dbl001 thank you!!! Totally forgot about that.

At batch_size= 500, my test file of news stories consume > 8gb of DRAM.
$ ls -l stories.txt
-rw-r--r--  1 davidlaxer  staff  68793151 Nov  8 08:48 stories.txt
$ wc stories.txt
5172 10169765 68793151 stories.txt

On Nov 9, 2018, at 10:43 AM, Nathan Raw<EMAIL_ADDRESS>wrote:
@dbl001 https://github.com/dbl001 thank you!!! Totally forgot about that.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/nateraw/Lda2vec-Tensorflow/issues/12#issuecomment-437455988, or mute the thread https://github.com/notifications/unsubscribe-auth/AC9i2znAucfn451RwB-uSv-yeVxJQGaMks5utczYgaJpZM4YKtIy.

@dbl001 Do you think we can (or should) make any changes to reduce the amount of memory consumed further? I can't tell if that amount (8GB) is an issue.

Converting to skip grams used ~18gb of DRAM.
converting to skipgrams
step 0 of 5172
step 500 of 5172
step 1000 of 5172
step 1500 of 5172
step 2000 of 5172
step 2500 of 5172
step 3000 of 5172
step 3500 of 5172
step 4000 of 5172
step 4500 of 5172
step 5000 of 5172

On Nov 9, 2018, at 2:24 PM, David Laxer<EMAIL_ADDRESS>wrote:
Investigating …
$ python load_stories.py
Using TensorFlow backend.
It took 13812.14863204956 seconds to run tokenizer method

On Nov 9, 2018, at 12:04 PM, Nathan Raw<EMAIL_ADDRESS><EMAIL_ADDRESS>wrote:
@dbl001 https://github.com/dbl001 Do you think we can (or should) make any changes to reduce the amount of memory consumed further? I can't tell if that amount (8GB) is an issue.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/nateraw/Lda2vec-Tensorflow/issues/12#issuecomment-437479346, or mute the thread https://github.com/notifications/unsubscribe-auth/AC9i20ILW3RkHOoE5KJ7mP_Or_217wJLks5utd-ygaJpZM4YKtIy.

@dbl001 @nateraw  I changed the batch size to 500 in file nlppipe.py  on line #184 and got a "KeyError". so I changed the batch size to 500 on line #282 too, but I still got the same error. the error message is shown blow:
<class 'KeyError'> nlppipe.py 199
Traceback (most recent call last):
File "E:/python/Lda2vec-Tensorflow/tests/twenty_newsgroups/load_20newsgroups.py", line 21, in
utils.run_preprocessing(texts, data_dir, run_name, bad=bad, max_length=10000)
File "E:\python\Lda2vec-Tensorflow\lda2vec\utils.py", line 149, in run_preprocessing
token_type=token_type, vectors=vectors, merge=merge)
File "E:\python\Lda2vec-Tensorflow\lda2vec\nlppipe.py", line 103, in init
self.tokenize()
File "E:\python\Lda2vec-Tensorflow\lda2vec\nlppipe.py", line 184, in tokenize
for row, doc in enumerate(self.nlp.pipe(self.texts, n_threads=self.num_threads, batch_size=2000)):
File "D:\Program Files\python\lib\site-packages\spacy\language.py", line 557, in pipe
recent_strings.add(word.text)
File "token.pyx", line 171, in spacy.tokens.token.Token.text.get
File "token.pyx", line 677, in spacy.tokens.token.Token.orth_.get
File "strings.pyx", line 116, in spacy.strings.StringStore.getitem
KeyError:<PHONE_NUMBER>5100257219.
how can I fix it?

@schneeLee the line you need to change is the batch size parameter of the nlp.pipe function within nlppipe.py. I need to update this to be more easily changed.

You might also consider adjusting ‘write_every’ (e.g. - write_every=100) when invoking utils.run_preprocessing() if you have a large input file.

On Nov 27, 2018, at 11:03 PM, Nathan Raw<EMAIL_ADDRESS>wrote:
@schneeLee https://github.com/schneeLee the line you need to change is the batch size parameter of the nlp.pipe function within nlppipe.py. I need to update this to be more easily changed.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/nateraw/Lda2vec-Tensorflow/issues/12#issuecomment-442341686, or mute the thread https://github.com/notifications/unsubscribe-auth/AC9i2zwPbr0emLLIyQCqjat4jbRln0qhks5uzjVHgaJpZM4YKtIy.

Collecting all the data into a list and outputting the entire list kept running out of memory (even replacing the data frame with file I/O:
E.g.
   import csv

    with open(file_out_path + "/skipgrams.txt", 'wb') as myfile:
            wr = csv.writer(myfile, delimiter="\t", quoting=csv.QUOTE_ALL)
            wr.writerow(data)

On Nov 28, 2018, at 9:26 AM, Nathan Raw<EMAIL_ADDRESS>wrote:
@dbl001 https://github.com/dbl001 you know my code better than me! I wonder if there is a better way to write that section too. The "write_every" fix was a quick and dirty solution, I feel like.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/nateraw/Lda2vec-Tensorflow/issues/12#issuecomment-442532727, or mute the thread https://github.com/notifications/unsubscribe-auth/AC9i25D43pzYbYukbkZ4ZnwhrP8Mcbz2ks5uzsdFgaJpZM4YKtIy.

Looks good to me, I'll test when I get the chance. Feel free to submit pull request if you want :)

The code below ran out of memory on a 5,000 line input file with 10 million rows.
So, the incremental write is probably best.  Is creating a data frame to output the CSV less efficient then doing python file I/O?
Probably, but it may not make a noticeable difference.
$wc data/stories.txt
5172 10128815 68793151 data/stories.txt
I’m currently running line profiling on utils.run_processing().
Stay tuned.

On Nov 28, 2018, at 10:13 AM, Nathan Raw<EMAIL_ADDRESS>wrote:
Looks good to me, I'll test when I get the chance. Feel free to submit pull request if you want :)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/nateraw/Lda2vec-Tensorflow/issues/12#issuecomment-442549183, or mute the thread https://github.com/notifications/unsubscribe-auth/AC9i2ym0p4qRyfrnTjanLNC8u8MTBQ7vks5uztJKgaJpZM4YKtIy.

(ai) David-Laxers-MacBook-Pro:twenty_newsgroups davidlaxer$ cat profile.out
Timer unit: 1e-06 s
Total time: 87014.2 s
File: /Users/davidlaxer/Lda2vec-Tensorflow/lda2vec/utils.py
Function: run_preprocessing at line 91
Line #      Hits         Time  Per Hit   % Time  Line Contents
91                                           @profile
92                                           def run_preprocessing(texts, data_dir, run_name, min_freq_threshold=10,
93                                                                 max_length=100, bad=[], vectors="en_core_web_lg",
94                                                                 num_threads=2, token_type="lemma", only_keep_alpha=False,
95                                                                 write_every=10000, merge=False):
96                                               """This function abstracts the rest of the preprocessing needed
97                                               to run Lda2Vec in conjunction with the NlpPipeline
98
99                                               Parameters

100                                               ----------
101                                               texts : TYPE
102                                                   Python list of text
103                                               data_dir : TYPE
104                                                   directory where your data is held
105                                               run_name : TYPE
106                                                   Name of sub-directory to be created that will hold preprocessed data
107                                               min_freq_threshold : int, optional
108                                                   If words occur less frequently than this threshold, then purge them from the docs
109                                               max_length : int, optional
110                                                   Length to pad/cut off sequences
111                                               bad : list, optional
112                                                   List or Set of words to filter out of dataset
113                                               vectors : str, optional
114                                                   Name of vectors to load from spacy (Ex. "en", "en_core_web_sm")
115                                               num_threads : int, optional
116                                                   Number of threads used in spacy pipeline
117                                               token_type : str, optional
118                                                   Type of tokens to keep (Options: "lemma", "lower", "orth")
119                                               only_keep_alpha : bool, optional
120                                                   Only keep alpha characters
121                                               write_every : int, optional
122                                                   Number of documents' data to store before writing cache to skipgrams file
123                                               merge : bool, optional
124                                                   Merge noun phrases or not
125                                               """
126
127         1      15572.0  15572.0      0.0      def clean(line):
128                                                   return ' '.join(w for w in line.split() if not any(t in w for t in bad))
129
130                                               # Location for preprocessed data to be stored
131         1          8.0      8.0      0.0      file_out_path = data_dir + "/" + run_name
132
133         1       2904.0   2904.0      0.0      if not os.path.exists(file_out_path):
134
135                                                   # Make directory to save data in
136         1        198.0    198.0      0.0          os.makedirs(file_out_path)
137
138                                                   # Remove tokens with these substrings
139         1          9.0      9.0      0.0          bad = set(bad)
140
141                                                   # Preprocess data
142
143                                                   # Convert to unicode (spaCy only works with unicode)
144         1   74906607.0 74906607.0      0.1          texts = [str(clean(d)) for d in texts]
145
146                                                   # Process the text, no file because we are passing in data directly
147         1         11.0     11.0      0.0          SP = NlpPipeline(None, max_length, texts=texts,
148         1          5.0      5.0      0.0                           num_threads=num_threads, only_keep_alpha=only_keep_alpha,
149         1<PHONE_NUMBER>4.0<PHONE_NUMBER>4.0     14.6                           token_type=token_type, vectors=vectors, merge=merge)
150
151                                                   # Computes the embed matrix along with other variables
152         1<PHONE_NUMBER>7.0<PHONE_NUMBER>7.0     70.2          SP._compute_embed_matrix()
153
154         1       6620.0   6620.0      0.0          print("converting data to w2v indexes")
155                                                   # Convert data to word2vec indexes
156         1  256696370.0 256696370.0      0.3          SP.convert_data_to_word2vec_indexes()
157
158         1       2465.0   2465.0      0.0          print("trimming 0's")
159                                                   # Trim zeros from idx data
160         1  293081773.0 293081773.0      0.3          SP.trim_zeros_from_idx_data()
161
162                                                   # This extracts the length of each document (needed for pyldaviz)
163         1      79761.0  79761.0      0.0          doc_lengths = [len(x) for x in SP.idx_data]
164
165                                                   # Find the cutoff idx
166     33365     162720.0      4.9      0.0          for i, freq in enumerate(SP.freqs):
167     33365     154817.0      4.6      0.0              if freq < min_freq_threshold:
168         1          3.0      3.0      0.0                  cutoff = i
169         1          5.0      5.0      0.0                  break
170                                                   # Then, cut off the embed matrix
171         1         43.0     43.0      0.0          embed_matrix = SP.embed_matrix[:cutoff]
172                                                   # Also, replace all tokens below cutoff in idx_data
173      5173      34349.0      6.6      0.0          for i in range(len(SP.idx_data)):
174      5172    2035591.0    393.6      0.0              SP.idx_data[i][SP.idx_data[i] > cutoff - 1] = 0
175                                                   # Next, cut off the frequencies
176         1       1562.0   1562.0      0.0          freqs = SP.freqs[:cutoff]
177
178         1         84.0     84.0      0.0          print("converting to skipgrams")
179
180         1          3.0      3.0      0.0          data = []
181         1         31.0     31.0      0.0          num_examples = SP.idx_data.shape[0]
182                                                   # Sometimes docs can be less than the required amount for
183                                                   # the skipgram function. So, we must manually make a counter
184                                                   # instead of relying on the enumerated index (i)
185         1          3.0      3.0      0.0          doc_id_counter = 0
186                                                   # Additionally, we will keep track of these lower level docs
187                                                   # and will purge them later
188         1          3.0      3.0      0.0          purged_docs = []
189      5173     679517.0    131.4      0.0          for i, t in enumerate(SP.idx_data):
190      5172      30736.0      5.9      0.0              pairs, _ = skipgrams(t,
191      5172      31826.0      6.2      0.0                                   vocabulary_size=SP.vocab_size,
192      5172      17912.0      3.5      0.0                                   window_size=5,
193      5172      15887.0      3.1      0.0                                   shuffle=True,
194      5172<PHONE_NUMBER>.0 975852.5      5.8                                   negative_samples=0)
195                                                       # Pairs will be 0 if document is less than 2 indexes
196      5172      76872.0     14.9      0.0              if len(pairs) > 2:
197 109260975  444886451.0      4.1      0.5                  for pair in pairs:
198 109255804  401964381.0      3.7      0.5                      temp_data = pair
199                                                               # Appends doc ID
200 109255804  543955438.0      5.0      0.6                      temp_data.append(doc_id_counter)
201                                                               # Appends document index
202 109255804  422388218.0      3.9      0.5                      temp_data.append(i)
203 109255804  436079510.0      4.0      0.5                      data.append(temp_data)
204      5171      27482.0      5.3      0.0                  doc_id_counter += 1
205                                                       else:
206         1          4.0      4.0      0.0                  purged_docs.append(i)
207      5172      22056.0      4.3      0.0              if i // write_every:
208                                                           temp_df = pd.DataFrame(data)
209                                                           temp_df.to_csv(file_out_path + "/skipgrams.txt", sep="\t", index=False, header=None, mode="a")
210                                                           del temp_df
211                                                           data = []
212      5172      22860.0      4.4      0.0              if i % 500 == 0:
213        11     355847.0  32349.7      0.0                  print("step", i, "of", num_examples)
214         1<PHONE_NUMBER>.0<PHONE_NUMBER>.0      5.0          temp_df = pd.DataFrame(data)
215         1  925359541.0 925359541.0      1.1          temp_df.to_csv(file_out_path + "/skipgrams.txt", sep="\t", index=False, header=None, mode="a")
216         1     494780.0 494780.0      0.0          del temp_df
217
218                                                   # Save embed matrix
219         1    8285844.0 8285844.0      0.0          np.save(file_out_path + "/embed_matrix", embed_matrix)
220                                                   # Save the doc lengths to be used later, also, purge those that didnt make it into skipgram function
221         1     124899.0 124899.0      0.0          np.save(file_out_path + "/doc_lengths", np.delete(doc_lengths, np.array(purged_docs)))
222                                                   # Save frequencies to file
223         1     197430.0 197430.0      0.0          np.save(file_out_path + "/freqs", freqs)
224                                                   # Save vocabulary dictionaries to file
225         1        229.0    229.0      0.0          idx_to_word_out = open(file_out_path + "/" + "idx_to_word.pickle", "wb")
226         1    1024778.0 1024778.0      0.0          pickle.dump(SP.idx_to_word, idx_to_word_out)
227         1      76339.0  76339.0      0.0          idx_to_word_out.close()
228         1        235.0    235.0      0.0          word_to_idx_out = open(file_out_path + "/" + "word_to_idx.pickle", "wb")
229         1     202652.0 202652.0      0.0          pickle.dump(SP.word_to_idx, word_to_idx_out)
230         1      92133.0  92133.0      0.0          word_to_idx_out.close()
(ai) David-Laxers-MacBook-Pro:twenty_newsgroups davidlaxer$

On Nov 28, 2018, at 10:41 AM, David Laxer<EMAIL_ADDRESS>wrote:
The code below ran out of memory on a 5,000 line input file with 10 million rows.
So, the incremental write is probably best.  Is creating a data frame to output the CSV less efficient then doing python file I/O?
Probably, but it may not make a noticeable difference.
$wc data/stories.txt
5172 10128815 68793151 data/stories.txt
I’m currently running line profiling on utils.run_processing().
Stay tuned.

On Nov 28, 2018, at 10:13 AM, Nathan Raw<EMAIL_ADDRESS><EMAIL_ADDRESS>wrote:
Looks good to me, I'll test when I get the chance. Feel free to submit pull request if you want :)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/nateraw/Lda2vec-Tensorflow/issues/12#issuecomment-442549183, or mute the thread https://github.com/notifications/unsubscribe-auth/AC9i2ym0p4qRyfrnTjanLNC8u8MTBQ7vks5uztJKgaJpZM4YKtIy.

Is this self.vocabulary, computed  in the function: compute_embed_matrix() in nlppipe.py being use
Append word onto unique vocabulary list
self.vocabulary = np.append(self.vocabulary, word)
It’s taking 98.5% of the time in compute_embed_matrix():
E.g. -
506<PHONE_NUMBER>7415647.0 451731.8     98.5              self.vocabulary = np.append(self.vocabulary, word)
numpy appends require a physical copy (because the use contiguous memory). If self.vocabulary is required to be a unique list, then appends to python lists are faster and more efficient.
Once the unique list is generated, you can generate a numpy array.

On Nov 28, 2018, at 5:25 PM, schneeLee<EMAIL_ADDRESS>wrote:
after I changeed the batch size parameter of the nlp.pipe function to 500, I still got the "KeyError" which I mentioned before. But when I cut down the 20_newsgroups corpus to a smaller size, it works. Is there anything special in corpus or in code?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/nateraw/Lda2vec-Tensorflow/issues/12#issuecomment-442670286, or mute the thread https://github.com/notifications/unsubscribe-auth/AC9i27yxowpahviiCEBbYw8JA4TZoLpWks5uzzeSgaJpZM4YKtIy.

@dbl001  thanks for the help. I forgot to adding the "texts" in the first line of the corpus. and there are some other questions: how can I get top N topics for each doc and the topN words for each topic after the train process is finished? and how to predict the topics of the unseen docs?

Working on fixing this here

@dbl001 @nateraw I changed the batch size to 500 in file nlppipe.py on line #184 and got a "KeyError". so I changed the batch size to 500 on line #282 too, but I still got the same error. the error message is shown blow:
Did you manage to find a solution? Same KeyError.

Pushing new preprocessing today. Looking back, I have no idea why I made it so complicated. Much easier to understand version coming shortly.

Updated. Closing this issue.


Test MLPerf Inference Llama2-7b model on Windows and add to Github action

Since we have cleaned up the MLPerf inference pipeline and made most of it Windows compatible we should check and support Llama2-7b model using the reference implementation on Windows and once successful add a github action for it on Windows. Scheduling it twice a month is enough.

We now have this working on Windows but the run is too slow on the CPUs.


Use django-taggit fork to test against Django main

Ref:

https://github.com/wagtail/wagtail/issues/10200#issuecomment-1513621347
https://github.com/jazzband/django-taggit/issues/851
https://github.com/jazzband/django-taggit/pull/852

@gasman Might want to get this in just so that we don't have CI failures after we create the stable/5.0.x branch. Can always cherry-pick later, though


Add support for raidroll

Usecase - MC's mats, AQ's scarabs and idols, ZG's bijous and coins, every raid's random greens/blues - all of these could be (and usually are) randomly distributed in a pug.
Technically - show loot window and a button next to each item that will raidroll item and then transfer that item to person who won.

Good call, we can add "Raid Roll" out items. Only annoying thing... if someone has full bags or isn't in the instance that makes more work for the Master Looter. Look for this in a future release.

Good call, we can add "Raid Roll" out items. Only annoying thing... if someone has full bags or isn't in the instance that makes more work for the Master Looter. Look for this in a future release.

Closing this... not a feature I want to pursue at this time.
Was going to add an option to the Loot Destinations drop down to send items to random people... but oof. Like it creates more work and a lot more hassle. At least if you control who gets items, you know who has been told to clear their bags.
You can give out greens to raid rolls at the end if you want, I don't see a good way to do this randomly that doesn't introduce more shit for the Master Looter to sort.
We won't have a "button" next to items because the point is to not see that loot window at all... just click on sparkles and have them distributed. Not have to pause and click more things as you go.

Closing this... not a feature I want to pursue at this time.
Was going to add an option to the Loot Destinations drop down to send items to random people... but oof. Like it creates more work and a lot more hassle. At least if you control who gets items, you know who has been told to clear their bags.
You can give out greens to raid rolls at the end if you want, I don't see a good way to do this randomly that doesn't introduce more shit for the Master Looter to sort.
We won't have a "button" next to items because the point is to not see that loot window at all... just click on sparkles and have them distributed. Not have to pause and click more things as you go.


Typescript definition file

Hi,
Where can I find the .d.ts file for aspectjs?
thanks
P.S: good job

If I'm not mistaken, they are shipped with npm.

Oh yes,
I have just seen it. I installed aspectjs instead of aspect.js.
My apologizes


Bump addon-tools dependencies to a more up-to-date kube-compare base module

Signed-off-by: Jim Ramsay<EMAIL_ADDRESS>

/approve


Impossible to start the Rancher Server on macOS

Rancher Versions: server:stable
Docker Version: 17.03.0-ce, build 60ccb22
OS and where are the hosts located? (cloud, bare metal, etc): Docker for macOS
Setup Details: (single node rancher vs. HA rancher, internal DB vs. external DB) Single node with external DB
Environment Type: (Cattle/Kubernetes/Swarm/Mesos) Cattle
Steps to Reproduce:
Docker-compose.yml
version: '2'

services:
  rancher_mysql:
    build: ./containers/rancher_mysql
    restart: always

  rancher_server:
    build: ./containers/rancher_server
    restart: always
    depends_on:
      - rancher_mysql

MySQL Dockerfile:
FROM mysql:latest

ENV MYSQL_ROOT_PASSWORD = qwerty
ENV MYSQL_DATABASE = cattle
ENV MYSQL_USER = cattle
ENV MYSQL_PASSWORD = qwerty

Rancher Server Dockerfile:
FROM rancher/server:stable

ENV CATTLE_DB_CATTLE_MYSQL_NAME = cattle
ENV CATTLE_DB_CATTLE_USERNAME = cattle
ENV CATTLE_DB_CATTLE_PASSWORD = qwerty

Results:
Error from the Rancher Server:
ERROR 1064 (42000) at line 1: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '= cattle COLLATE = 'utf8_general_ci' CHARACTER SET = 'utf8'' at line 1

ENV lines don't have an equal sign in them.  That's becoming part of the value, which makes all the statements that try to use them invalid.
FYI we do not support Docker for Mac.  It will start up but using it as an agent requires knowing secret IPs and publishing ports doesn't work because of the stuff they do to pretend that it is not just a VM running Linux.  Regular VMs or docker-machine with the xhyve driver are fine.

Oh I'm so sorry for this mistake... It works fine. Thanks.


Regenerate and re-release?

I've corrected the typo in satterthwaitetb/raw/sources.bib , regeneration and rerelease are pending.
Question: should we do before the final touches to CLICS3? The typo is already corrected in the paper.

Closing this -- I assume we'll release anything before lexibank 1.0


Marching cubes returns empty index buffer

A recent regression, marching cubes now returns zero length indices, though it returns positions.

This was corrupted by https://github.com/webaverse/app/commit/f5ccec2f4c7e48283d970bee7d09f64ab950d96e and apparently https://github.com/webaverse/app-wasm/pull/43/files
The build does not match what is in app-wasm.


Passing function as props inside JSX

Hi, I just wanted to check something about my own understanding, I've attached a screenshot of a lint error where I'm passing a function from props to a child element. Since the signatures are the same, I'm used to just passing it straight down without an anonymous function, however the linter claims that props won't be tracked, even though it is inside jsx. The lint doesn't appear if I do () => props.func()
Is this a linter bug or will it not actually be tracked?
Attached Screenshot:

Good question! This one is subtle, but the lint rule is correct here. From the (docs)[https://www.solidjs.com/docs/latest/api#on___]:

Events are never rebound and the bindings are not reactive, as it is expensive to attach and detach listeners. Since event handlers are called like any other function each time an event fires, there is no need for reactivity; simply shortcut your handler if desired.

So if the prop changes, the old value will still be set as the event handler, leading to potential bugs. Wrapping with a function ensures that the most up to date prop value is used.

Thank you for the explanation! And thank you for the eslint plugin, I would never have caught that from the docs without it.


Network, WebAssembly and Browser

I see that your are opening here also to a distributed perspective. What do you think of some emerging solution like http://arxiv.org/abs/1503.05743? Yes this rely on WebCL that it is not supported natively in any browser and actually propose only convolutional layers in a distributed fashion. But some discussion on WebAssembly, in which also Google is involved, are starting to think on GPU support of this forming standard. See https://github.com/WebAssembly/design/issues/273. Do you think that caffe2 design could be further proof to scale at the level where every node could run on a browser tab? Or don't you believe that training and networks design could evolve at large scale relying on users nodes with "standard" internet bandwidth interconnection?

Thanks - yeah, I think as long as we stick to the same protobuf definition (sort of like API) this would be possible. Of course one needs to write the WebCL implementation (Andrej actually wrote one before but with custom format) - other than that it should just be a formatting issue.

How do you think that a JavaScript back end could be introduced in the design? I.e. starting from the JavaScript matrix component https://github.com/mil-tokyo/sushi

Feel free to tinker around with it, but this is honestly not a top
priority...
Sent from Gmail mobile
On Aug 30, 2015 9:29 AM, "bhack"<EMAIL_ADDRESS>wrote:

How do you think that a JavaScript back end could be introduced in the
design? I.e. starting from the JavaScript matrix component
https://github.com/mil-tokyo/sushi
—
Reply to this email directly or view it on GitHub
https://github.com/Yangqing/caffe2/issues/11#issuecomment-136157348.

@Yangqing Yes I think that browser technology it is not enough mature actually so cannot be a priority for caffe2. But I think that if WebAssembly go through covering GPU computing could be interesting to have a "compatible" design . Can I ask you what are your design targets on distribute the training and parameter update? On caffe the multigpu it is oriented mainly on fast p2p communication between GPU on PCI bus. Are you targeting also multi node? On which kind of network infrastructures?

Some interesting Caffe experiments with Multicore/MultiGPU and Clusters nodes at http://arxiv.org/abs/1506.08272

See also  https://github.com/BVLC/caffe/pull/1148#issuecomment-117017412


Not receiving data messages on Android

I'm sending two types of firebase messages :

notifications and custom messages.
but I'm seeing that the only showing on Android are notifications.
Custom data messages are not showing.
Do I have to set some special fields on the data_payload to make them shown ?
moreover I read about the need to create a  notification channel... How can I create a channel using cordova? I know how to using native Java app, but we are in the cordova world.

@gvolpe1968 this is correct behavior. For data messages register onMessage/onBackgroundMessage callbacks and trigger some UI.

At present plugin does not have any methods for notification channel CRUD. This is in TODO list

Notification channel CRUD release in v4.2.1


labelColor attribute on FlatButton?

Why FlatButton has no labelColor attribute as the RaisedButton does? Is there any specific reason or it was simply forgot?

That's an inconsistency with our components. For some reason, it was only added on a single component.

We have been porting the component on the v1-alpha branch. We reimplemented it from the ground-up. We rely on CSS inheritance to propagate the color on the text. We also have some idiomatic color property values available.


npm run dev

In the laravel framework, I used the element-in-laravel-starter that you proposed last time, everything was configured correctly, but after NPM run dev, the page was open after the page was open, and nothing was wrong.
It's not a blank after build!
It's not a blank after build!
It's not a blank after build!
It's blank after NPM run dev. There is no error message.
How do I fix it?

NPM run prod is running normally, but NPM run dev is blank.

I am not the maintainer of element-in-laravel-starter, so I recommend you go to the official to ask questions.
By the way,there is nothing wrong with the terminal?

DONE, Compiled successfully in 1344ms ,and the console doesn't have any mistakes.

Sorry, I may not be able to help you. I don't know much about Laravel.

I really can't find the cause of the phenomenon. Do you know why these are "empty"?

This picture is the result of 'NPM run prod'
In the picture of the last reply，that picture of result happened by 'NPM run dev'.
In my main.js: ...xxxxxxx... new Vue({   router,   store,   i18n,   render: (h) => h(App) }).$mount('#app')
App.vue: `

And in my package.json:{
"scripts": {
"dev": "npm run development",
"development": "cross-env NODE_ENV=development node_modules/webpack/bin/webpack.js --progress --hide-modules --config=node_modules/laravel-mix/setup/webpack.config.js",
"watch": "npm run development -- --watch",
"watch-poll": "npm run watch -- --watch-poll",
"hot": "cross-env NODE_ENV=development node_modules/webpack-dev-server/bin/webpack-dev-server.js --inline --hot --config=node_modules/laravel-mix/setup/webpack.config.js",
"prod": "npm run production",
"production": "cross-env NODE_ENV=production node_modules/webpack/bin/webpack.js --no-progress --hide-modules --config=node_modules/laravel-mix/setup/webpack.config.js"
}
}`

群主超强啊，国内外通吃


DVT Eclipse crashing

Please provide a brief summary of the bug
A fatal error has been detected by the Java Runtime Environment:

SIGSEGV (0xb) at pc=0x00007fc12c7d66b8, pid=1481, tid=1482

JRE version: OpenJDK Runtime Environment AdoptOpenJDK (11.0.10+9) (build 11.0.10+9)
Java VM: OpenJDK 64-Bit Server VM AdoptOpenJDK (11.0.10+9, mixed mode, tiered, compressed oops, g1 gc, linux-amd64)
Problematic frame:
C  [libgtk-3.so.0+0x8e6b8]

Core dump will be written. Default location: Core dumps may be processed with "/usr/libexec/abrt-hook-ccpp %s %c %p %u %g %t e %P %I %h"

An error report file with more information is saved as:

If you would like to submit a bug report, please visit:
https://github.com/AdoptOpenJDK/openjdk-support/issues
The crash happened outside the Java Virtual Machine in native code.
See problematic frame for where to report the bug.

[1]    Done                          dvt.sh
Please provide steps to reproduce where possible
No response
Expected Results
Eclipse should  run properly
Actual Results
Eclipse  is crashing.
What Java Version are you using?
1.8.0_372
What is your operating system and platform?
No response
How did you install Java?
No response
Did it work before?
No response
Did you test with the latest update version?
No response
Did you test with other Java versions?
No response
Relevant log output
No response

There was a problem related to dvt eclipse version . Now the problem is resolved.


SPMT File Size Limit

I quite frequently get the following failure message when using SPMT:
Scan File   Failure:File size exceeds limit
Please could this be at least added and spelt of that the limit seems to still track the 15GB SharePoint limit and not the 100GB. Ideally as part of this someone could make it align with the browser upload limit, but if not it might also be worth specifying the workaround for this if a file exceeds 15GB. The workaround is to manually upload the file which is not ideal when trying to script this process end to end.
Thanks
Henry

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: cfa08336-690e-0d21-6491-3f14f6d13391
Version Independent ID: 8927975a-7956-fc72-055e-36603e4eddaf
Content: Troubleshoot SharePoint Migration Tool - Migrate to SharePoint and OneDrive
Content Source: migration/troubleshooting-common-spmt-issues.md
Service: sharepoint-online
GitHub Login: @JoanneHendrickson
Microsoft Alias: jhendr

@harp365  Thank you for submitting feedback and contributing to the docs. We are currently investigating this.

Hi @harp365
Thank you very much for posting this issue. Considering the nature of the issue, it is advised that please open a service ticket in your tenant and follow with them for the resolution. If you wish to know how to open the service ticket, please follow this link: https://docs.microsoft.com/office365/admin/contact-support-for-business-products We will be very happy if you can share the outcome or resolution with us. We will certainly update this documentation with that valuable information. Thanks again, for taking out some time to open the issue.

Thank you for submitting feedback. We understand that this issue has been resolved.
Please feel free to re-open this issue if there is a specific area of the docs that we can improve or make better. Thank you.

Hi, not sure what has been done here. Maybe I cannot see for looking but the documentation is the same and the issue still remains. Not sure what has happened for this to be closed?
Thanks

@harp365 Do you managed to have a solution for this? we had the same problem here


Can't run Gibber environment in master branch using localhost, external/script.min.js missing

Hi.
Don't know if this is really a bug, but , can't run Gibber environment in master branch using localhost (using node.js' http-server), external/script.min.js missing.
It does run at the gibber2 branch.
Thanks

This bug got lost... I have instructions for installing the server and populating the database here:
https://github.com/charlieroberts/gibber.server


Contact form styling

Tasks

Update contact form styling to match the rest of the forms on login page, signup page and registration page (update layout, form fields and labels)
Update the other forms with placeholder and styling as written in the style guide

Design
Border: 1px solid light gray (#CCCCCC)
Focus: Border changes to primary color (#4CAF50)
Padding: 8px
Labels: Secondary text color (#757575), small font size, and bold.
Validation Messages: Error messages in accent color (#FF5722) below the field.

I have styled the contact form with border, padding, labels and placeholders. I created a component of form fields and added it to all forms (contact, signup, login and registration) both on desktop and mobile.
For the contact error form, see issue #102 that I am currently working on.

Since the background color of the site is F5F5F5, using the color 757575 for the labels fail on contrast. I have now used the color 333333 witch is a neutral color under the main colors in the style guide. I added a comment on this in the Figma file.

@pauiws I added a comment to this on the Figma page !

Great work! Review completed


Add ability to cancel running tests

I wrote a test that accidentally never completed. The only way to halt the test run in VS Code was to restart the editor. We need a way to stop a test run. The little square stop button in the attached screenshot does nothing.

That used to work

A clarification, the Cancel button in the Test Explorer does work and stops a build/cancels running tests as you'd expect, but the Cancel button in the list panel to the right of the test results output (see screenshot) is the one that does nothing.

Verified on e7f81c930e752bb049fc7c5fae8e98684bd8b54d that a test can be properly cancelled.


Add Podrocket.ai to list of apps

I have a service called podrocket (https://podrocket.ai) that uses the Podcast Index APIs around episode and searching.  This application is a discover service to help users find podcasts that match their listening preferences/habits.  As part of this pull request I added a logo image to the assets folder and I updated the apps.json file.  Please let me know if you need anything else, thanks for creating and maintaining such a valuable service.

I believe I have completed what was needed for the requested change, please let me know if anything else if required.

I think is a matter of waiting for the P20 "staff" to have enough time to review.
Too much to do with work, life, family and Podcasting

Looks good to me. I approved the check to run and it passed. Although I can merge, I usually leave them for Dave since he is the only one that can update the website.


Lack of aarch64 builders is blocking nixos-unstable channel

Describe the bug
Lack of aarch64 builders is blocking nixos-unstable channel.
The aarch64 capacity has changed and cannot keep up with Linux/Darwin builders. The nixos-unstable channel is waiting 4 days already for certain aarch64 builds. This is an issue because it further reduces the speed at which we can distribute security updates.

Recently, Packet started provisioning and deprovisioning our c2.large.arm's in a tight loop:

This left us with an average of one machine at a time, but that one machine wasn't up long enough for it to actually do any useful work:

(data: https://status.nixos.org/grafana/d/27bz8SAWk/packet-provisioning-churn?orgId=1&from=1572123600000&to=1572192154115)
Once this problem resolved we were back up to a more normal capacity of ~4 spot instances. This brought our capacity up a good bit:

At the same time, I realized the permanent ARM hardware had fallen off the map too, due to a failed boot a few days ago. This compounded with new rebuilds of the ARM hardware images failing due to a bug in the tests for the JSON library for Nix. Because of these failing builds, no reboots were attempted. A reboot would have solved these problems.
I switched the builders from the 19.09-small channel to 19.09, as it has more tests for aarch64 and a working Nix build.
This update is currently working its way through the deploy process, having successfully built the images for the ARM machines: https://buildkite.com/grahamc/packet-nix-builder/builds/145
Already, capacity is up even more than after the spot market stabilized:

and once those remaining builders are rebooted, the capacity will rise further. (data: https://status.nixos.org/grafana/d/MJw9PcAiz/hydra-jobs?orgId=1&from=now-2d&to=now&fullscreen&panelId=15&refresh=30s)
Additionally, I canceled all queued non-current jobs in Hydra to reduce the queue and focus on getting the channel out.
I'm expecting and hoping this issue is solved, and that the remaining build tasks will be processed promptly.

@grahamc I appreciate you ;)

There seems to be enough capacity again so closing.


[v1.3.2] Update Sort to utilize new attributes

Not ready for review

Looks good, thanks for the contribution my boy keith


Random crash on quest finish

Describe the bug
The game randomly crashes when quest finished and the player is being loaded into the village. Investigate.
To Reproduce
?
Expected behavior
No crash.

Possibly fixed in de65b1500c2246ebe76beb3fc6af3bf5f82dbcfe and d9f5799fd2f531f3817462da8fc366c7f4b20b78.


SingularMatrixException on Error Calculation

Hi Scott,
It seems the following two lines both give a SingularMatrixException when run:
// error and geometry information
RealVector standardDeviation = optimum.getSigma(0);
RealMatrix covarianceMatrix = optimum.getCovariances(0);
I cannot get it to work with any input data. For example, here are some positions that throw the error (these are a Mercator projection of GPS coordinates):
Positions[0]:
0 = -4565919.193832541
1 = 1.6141672528603023E7
2 = 2200.0
Positions[1]:
0 = -4565920.313515583
1 = 1.6141679993156526E7
2 = 2200.0
Positions[2]:
0 = -4565927.778069112
1 = 1.6141678948119089E7
2 = 2200.0
Positions[3]:
0 = -4565926.658386086
1 = 1.6141671334274463E7
2 = 2200.0
Distances:
0 = 1.6
1 = 1.6
2 = 1.6
3 = 1.6
Would you have any idea as to why this might be the case? This is running on Android with the same (recently updated) Apache commons Math3 library you are using. It also doesn't work with the previous version of the library.
Calculating the point estimate works perfectly fine.
Regards,
Blair

Aha!
It seems to only happen when using a third position (altitude) in the Positions array. When you remove this it calculates both of these just fine.
Creating a separate function to calculate the error with only a two element position (lat/lng) allows you to calculate this just fine.

Hi Blair,
I have seen this exception before when requesting the error information.  It likely means that this information is meaningless.
When you request sigma, the covariance of the Jacobian matrix is determined in a QRDecomposition.  The double threshold argument you provide to getSigma is a way for the caller to specify that the result of this computation should be considered meaningless, and thus trigger an exception.  In your above example, change optimum.getSigma(0); and optimum.getCovariances(0); to optimum.getSigma(-1); and optimum.getCovariances(-1); respectively.  This will likely cause you to obtain NaN entries in your results.  Meaningless, right?
Do you think if the above information was better documented, this would avoid confusion in the future?  What do you think?
Thanks,
Scott

I think a brief explanation on the threshold in the readme and how to handle it should suffice. It just threw me because the optimum was calculated just fine.

@bleckers, I put a short comment in the README.  Take a look, let me know what you think.  Can I close this issue?

Maybe something about what input variables might constitute a SingularMatrixException? Most people will be using this library without going into detail about the maths involved and it would be useful to know when they might experience the error (like for example above when using a 3D coordinate space).

Anyone can tell me how to run this code with proper snapshot so that I can run this code and see the results. It will be very helpful if someone give me the idea to run this code

Hi wsingh23,
The test code (https://github.com/lemmingapex/Trilateration/blob/master/src/test/java/com/lemmingapex/trilateration/TrilaterationTest.java) gives a examples of how to use the code.
Take a look at that as a starting point.

Everything related to SingularMatrixException is well documented in apache's docs:
https://commons.apache.org/proper/commons-math/apidocs/org/apache/commons/math3/fitting/leastsquares/LeastSquaresProblem.Evaluation.html#getSigma(double)
Users looking to find out more about how these methods should look there.

how to run this code i mean to say on which platform i have to run this
code java android or any other platform please explain  me step by step.
On Tue, Dec 1, 2015 at 9:56 PM, Scott Wiedemann<EMAIL_ADDRESS>wrote:

Closed #2 https://github.com/lemmingapex/Trilateration/issues/2.
—
Reply to this email directly or view it on GitHub
https://github.com/lemmingapex/Trilateration/issues/2#event-479145510.

Hi
This library works with android screen coordinate system?
I have four beacons and I need draw user position, but always return same position regardless of which beacon is closest to the user


Create Jenkins pipeline to release the operator

Adds a Jenkins pipeline to release the operator.
Related back-end PR: kiali/kiali#3567
Related kiali/kiali#2895

@jmazzitelli Indentation is fixed.
As cfcosta seems to be out, I'm wondering if you could also review the back-end repo PR: https://github.com/kiali/kiali/pull/3567

@jmazzitelli Indentation is fixed.
As cfcosta seems to be out, I'm wondering if you could also review the back-end repo PR: https://github.com/kiali/kiali/pull/3567


Consolidate all YNAB converter scripts into just one?

Hello Rafael!
I've noticed that there are more than 80 GitHub projects related to YNAB converter scripts. Would it be helpful and would there be interest in an effort to consolidate all (most) of these into a single tool? On the plus side, clearly there's more than enough solutions and contributors. On the minus side, these are spread across different spoken languages, programming languages, and of course bank export formats.
You're probably familiar with the YNAB subreddit. I've posted this idea here for a joint discussion: https://www.reddit.com/r/ynab/comments/75fctx/tools_can_we_consolidate_all_our_ynab_import/
Regards from Vienna,
Ben

Hi again!
We have built a really nice and flexible solution and it would be wonderful if you would share your bank's format with us?
You could either respond here (something like "date";"account";payee";...), or use this form where we collect all our formats.
Regards from Vienna,
Ben


Upgrade Serverless to 1.65.0

SO OLD!!! 😱

@mt-dfcowell

we don't merge ourselves our PRs
this repository is deprecated and should never be used. We made a PR with our feature (by that time) to the master branch and got accepted. In any case, that doesn't work like that any more, so please, any repository that is using THIS repository, upgrade it 🙇

https://github.com/moneytree/serverless/releases

@mt-sergio

Noted! Thought this case may be an exception since it was only merging upstream. I'll keep this in mind for the future. 🙇♂
This is for my work on https://github.com/moneytree/mt-utility-lambdas which uses that feature in its CI script. I'm a bit wary of changing how it works, since I'm not sure what might break in AWS if I change how restartHerokuDynos is deployed.
Please put a deprecation notice in this repo's description so that other newcomers know that it's deprecated!

Well, restartHerokuDynos is a pretty special piece in our environment as it helped a lot to recover from incidents. I think it had some custom permissions from Ross, so we kind of denied access to everybody.
My suggestion is that if you are not sure or consider newcomer to a project, start asking questions and waiting feedback from some peer (point number 1).
I love you take care of that project, but I understand it's challenging. Empathize with the pressure we had in that moment :D
I will add the "deprecated" as it's the first time to check the repository in years ;)

Glad to know we contributed successfully to Open Source.


KMS access

It's a question, rather than an issue. When using KmsMasterKeyProvider, does it actually call KMS service each time you encrypt / decrypt data? I walked through the Concepts and Architecture documents but I'm a bit confused about the network calls AWS Encryption SDK will make when combined with AWS KMS.
Could you elaborate it more using the String Example?
Does it actually access KMS for the master key decryption when encryptString is called?
Could you somehow use KmsMasterKeyProvider as singleton and re-use the same master key for multiple encryptions?

Yes. Every encrypt/decrypt will result in a call to KMS.
There is nothing wrong with making the KmsMasterKeyProvider a singleton and reusing it. (That's proper usage even.) However, since KMS keys live in KMS and are non-exportable, each use of them (for an encrypt/decrypt) will result in a network call to KMS.

Please correct me if I am wrong. Theoretically we reuse the data key for all the requests. I don't see a need to make calls to KMS api for each encryptXXX and decryptXXX methods. Reusing the data key could be optional.

@jamesanto That is correct. The underlying implementation and data-formats were built specifically with this use in mind. However, we do not currently have code available to support this use-case.


Sudden Crash

Suddenly crashes while searching the fav channels tab. After i've gone through all my channels only "Shylily" seem to trigger the crash. She's not even banned or was banned in the past days so i really have no idea why just her channel triggers it. Also i get no crash log sadly (no new log in AppData\Roaming\Twitch Leecher DX\logs).

Thanks!
It also happens when searching the channel directly.
I will investigate this asap

This turned out to be a bug in the UI Framework "Avalonia", which Twitch Leecher DX uses and it seems to be unable to render certain characters which appear in some video titles in Shylily's channel. Upgrading the version of Avalonia to latest resolved this issue


contextmenuWidth ignored with bindContextMenu

Hello,
If you bind the contextmenu later with bindContextMenu the contextmenuWidth property gets ignored
Fiddle: http://jsfiddle.net/7anskd8s/

actually it's ignored for everything except for the map

Fixed with #107


cannot open source file "iostream". Please run the 'Select IntelliSense Configuration...' - #include errors detected. Please update your includePath

This problem popped up out of nowhere in the middle of October.
" cannot open source file "iostream". Please run the 'Select IntelliSense Configuration...' command to locate your system headers.C/C++(1696)"
I'm getting this error in all of my .cpp files.
I am on a MacBook Air btw and I have Clang version 16. The same problem happened on my PC, but I was able to fix that by following the instructions on the VSC website. But nothing seems to work for my Mac. Can some one please help?
(I've included a simple code to show the problem)

(This is what my c_cpp_properties.json looks like)

(When I type in #include , usually the option for "iostream would pop up but it isn't even showing anymore)

I've tried clean uninstalling and reinstalling, changing the compiler path, and the Include path, and I've tried changing the "c_cpp_properties.json" file, but nothing seems to work. I've watched countless youtube videos trying to fix this issue but nothing seemed to work so far. I've spent at least 4+ hours trying to fix this :( .`

/extC++


[4pt]  - Create feature: add new reservation - Student C

Implement the "add new reservation" form on the Reservations pop up according to the wireframe:

When the user clicks on the "Reserve" button, the data is recorded in the Involvement API and the screen is updated

We only had 2 group members.


Update Chrome support for before/after print event handlers

This PR updates Chrome Desktop and Chrome for Android support for before/after print events. Which I have verified as follows:

Chrome Desktop: v63+
Chrome for Android: v64+

Demo used:
https://codepen.io/joseluisq/full/VzRpGb/
References:

MDN - WindowEventHandlers onafterprint browser compatibility
https://caniuse.com/#feat=beforeafterprint

Also https://www.chromestatus.com/feature/5700595042222080: Opera 50+, too :)

Also https://www.chromestatus.com/feature/5700595042222080: Opera 50+, too :)

true! 😄

Perhaps, if Opera 50+ support should come in separated PR. what do you think?

@Schweinepriester What property correspond to Opera for Android, op_mob ?

I don't think Opera for Android is tracked (anymore) really, but it probably was op_mob, yeah.
FYI the checks are currently failing because you added and_chr 63, but https://github.com/Fyrd/caniuse/blob/master/CONTRIBUTING.md#unsupported-changes
Add a new browser or browser version

I don't think Opera for Android is tracked (anymore) really, but it probably was op_mob, yeah.
FYI the checks are currently failing because you added and_chr 63, but https://github.com/Fyrd/caniuse/blob/master/CONTRIBUTING.md#unsupported-changes
Add a new browser or browser version

Done.

Thanks both!

Stop
On Feb 17, 2018 8:26 PM, "Alexis Deveria"<EMAIL_ADDRESS>wrote:

Thanks both!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/Fyrd/caniuse/pull/4086#issuecomment-366485035, or mute
the thread
https://github.com/notifications/unsubscribe-auth/Ai1QiTaVLz7R6GA5gM_bbB3UPko3ydFnks5tV3xdgaJpZM4SGgw4
.


feat: #12 add key benefits and high level structure to ./README

fix #12

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Pavel Vostrikov seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


ERROR: release-argus/argus, Get "http://<IP_ADDRESS>:8111/api/v1/version": EOF

I noticed yesterday that the current version for Argus started failing.  I can curl http://<IP_ADDRESS>:8111/api/v1/version from the container and get the expected results so it does not appear to be a connectivity issue.
I see the following in the logs:
ERROR: release-argus/argus, Get "http://<IP_ADDRESS>:8111/api/v1/version": EOF
I checked the docs after the project rename thinking something might have been updated, but not seeing anything other than the project name in the Github URL.
Config.xml for Argus
  release-argus/argus:
    type: github
    url: release-argus/argus
    web_url: https://github.com/release-argus/Argus/blob/master/CHANGELOG.md
    icon: https://github.com/release-argus/Argus/raw/master/web/ui/static/favicon.svg
    deployed_version:
      url: http://<IP_ADDRESS>:8111/api/v1/version
      json: version

I have also noticed that once it does that check a blank line is added in config.xml after the Argus section.  On container restart, this causes it to be the only service parsed.  Removing the container, removing the blank line from config.xml, and restarted the container then parses all services again.

Here is what I get from curl inside the container:
{"version":"0.4.1","buildDate":"2022-05-16T03:09:05Z","goVersion":"go1.18.2"}

Above you can see two commits I've done that fix the two issues w.r.t saving/parsing the save.
Looking into the Docker one now. I'm guessing it's something Docker networking related

Trying it in my own docker container, it manages to query /api/v1/version just fine. Maybe try replacing <IP_ADDRESS> with localhost and see if that helps?
Mine worked for localhost as well as my private IP

From what I can tell, it is the arbitrary blank line that is being added to the config.xml when the container is removed and readded that is causing the issue.
If I add a new service at the bottom of config.xml and restart the container, I get a new blank line added after the release-argus/argus block which is the first block in the service section.  Argus is then the only one that shows up in the UI.  I think the EOF message is coming from the fact that there is now a blank line in config.xml.
I see in the logs that it found 11 services, but argus is the only one then that it parses.

From what I can tell, it is the arbitrary blank line that is being added to the config.xml when the container is removed and readded that is causing the issue.
If I add a new service at the bottom of config.xml and restart the container, I get a new blank line added after the release-argus/argus block which is the first block in the service section. Argus is then the only one that shows up in the UI. I think the EOF message is coming from the fact that there is now a blank line in config.xml.
I see in the logs that it found 11 services, but argus is the only one then that it parses.

https://github.com/release-argus/Argus/commit/9a1b4a1a26fe02791e77df4962fc2dce058dcc4f

Just built a new local container and the blank line issue seems to be resolved.  Argus is also now updating properly after I added semantic_versioning: false.  Of course my current version now shows development.  ;-)
Still seeing an initial error connecting in the logs as soon as the container starts.  My gut says maybe it is a timing thing.
I can curl the URL in the container with no issue so I feel quite confident it is not a connectivity issue at least by the time I can get into the container and run the command.
ERROR: release-argus/argus, Get "http://localhost:8111/api/v1/version": dial tcp <IP_ADDRESS>:8111: connect: connection refused


Fatal error: Cannot redeclare class Zend\Loader\SplAutoloader

Maked application from latest ZendSkeletonApplication with --recursive, added your EDPSUPERLUMINAL,
 array()));

.....
?>
Error stack:
Fatal error: Cannot redeclare class Zend\Loader\SplAutoloader in D:\xampp\htdocs\zf2\vendor\ZendFramework\library\Zend\Loader\SplAutoloader.php on line 31
Time	Memory	Function	Location
1	0.0005	346080	{main}( )	..\index.php:0
2	0.0389	5201696	require_once( 'D:\xampp\htdocs\zf2\vendor\ZendFramework\library\Zend\Loader\AutoloaderFactory.php' )	..\index.php:7
3	0.0390	5208392	require_once( 'D:\xampp\htdocs\zf2\vendor\ZendFramework\library\Zend\Loader\SplAutoloader.php' )	..\AutoloaderFactory.php:24

Can you please pull the latest master and see if you're still experiencing this issue? Thanks.

I pulled latest master and this issue has been resolved.
But, when I enabled SpiffyDoctrineORM Module and tried /?buildCache=1, new issue came on:
Fatal error: Cannot redeclare class Doctrine\ORM\Mapping\Annotation in D:\xampp\htdocs\zf2\vendor\SpiffyDoctrineORM\vendor\doctrine2-orm\lib\Doctrine\ORM\Mapping\Annotation.php on line 22

Interesting.. I will look into this. On a side note, SpiffyDoctrine has been ported over to an official ZF2 module under the Doctrine project: DoctrineModule.

Great thanks for your development.
It's good news.

I'm going to mark this as resolved until I hear otherwise.


Saving model weights

Hi,
I have a couple questions about variables, keep_backup and old_wgts in

I needed to modify the default save_interval = 5 to 1 epoch, so I wasn't sure how to update default keep_backup = 5? Do I change it to 1 as well and what's the purpose of old_wgts?

Old_wgts points the file to be deleted. Every weight file is about 200 mbytes. Therefore, if you don not delete old files, may be you suffered out of space in your disk.
Sent with GitHawk

Keep_backup means the number of weight files will be saved. And save_interval indicates the interval saving weights.
Sent with GitHawk

I commented out lines to disable the interval weight saving, since my training is only a few epochs.


teb_local_planner having difficulty making a 90 degree turn

HI All
I have applied the teb_local_planner to my move_base for for my physical car like robot..
It reaches the goals i set successfully in my living room when just travelling in a straight line is required, or just soft turns are required to reach the goal pose
However when bigger turns are required the planner seems to abandon the initial feasible plan to make the turn and adopt more and more complex turn sequences, without giving the original plan a chance to work. I  have a link to a youtube video of what happens
https://youtu.be/vGLCs41Mf8k
Im sure i have mis-configured it. But can anyonepoint me the direction what parameters i should be looking at?
here is my
base_local_planner_params.yaml
TebLocalPlannerROS:

  odom_topic: odom

    # Trajectory

  teb_autosize: True
  dt_ref: 0.3
  dt_hysteresis: 0.1
  max_samples: 500
  global_plan_overwrite_orientation: True
  allow_init_with_backwards_motion: True
  max_global_plan_lookahead_dist: 3.0
  global_plan_viapoint_sep: -1
  global_plan_prune_distance: 1
  exact_arc_length: False
  feasibility_check_no_poses: 2
  publish_feedback: True

    # Robot

  max_vel_x: 0.18
  max_vel_x_backwards: 0.18
  max_vel_y: 0.0
  max_vel_theta: 0.5 # the angular velocity is also bounded by min_turning_radius in case of a carlike robot (r = v / omega)
  acc_lim_x: 0.5
  acc_lim_theta: 0.7

  # ********************** Carlike robot parameters ********************
  min_turning_radius: 2.0        # Min turning radius of the carlike robot (compute value using a model or adjust with rqt_reconfigure manually)
  wheelbase: 0.3                 # Wheelbase of our robot
  cmd_angle_instead_rotvel: True # stage simulator takes the angle instead of the rotvel as input (twist message)
  # ********************************************************************

  footprint_model: # types: "point", "circular", "two_circles", "line", "polygon"
    type: "two_circles"
    radius: 0.2 # for type "circular"
    line_start: [0.0, 0.0] # for type "line"
    line_end: [0.4, 0.0] # for type "line"
    front_offset: 0.3 # for type "two_circles"
    front_radius: 0.2 # for type "two_circles"
    rear_offset: 0.0 # for type "two_circles"
    rear_radius: 0.2 # for type "two_circles"
    vertices: [ [0.25, -0.05], [0.18, -0.05], [0.18, -0.18], [-0.19, -0.18], [-0.25, 0], [-0.19, 0.18], [0.18, 0.18], [0.18, 0.05], [0.25, 0.05] ] # for type "polygon"

      # GoalTolerance

  xy_goal_tolerance: 0.4
  yaw_goal_tolerance: 0.25
  free_goal_vel: False
  complete_global_plan: True

    # Obstacles

  min_obstacle_dist: 0.27 # This value must also include our robot's expansion, since footprint_model is set to "line".
  inflation_dist: 0.1
  include_costmap_obstacles: True
  costmap_obstacles_behind_robot_dist: 1.0
  obstacle_poses_affected: 15

  dynamic_obstacle_inflation_dist: 0.6
  include_dynamic_obstacles: True

  costmap_converter_plugin: "costmap_converter::CostmapToLinesDBSRANSAC"
  costmap_converter_spin_thread: True
  costmap_converter_rate: 5

    # Optimization

  no_inner_iterations: 5
  no_outer_iterations: 4
  optimization_activate: True
  optimization_verbose: False
  penalty_epsilon: 0.1
  obstacle_cost_exponent: 4
  weight_max_vel_x: 2
  weight_max_vel_theta: 1
  weight_acc_lim_x: 1
  weight_acc_lim_theta: 1
  weight_kinematics_nh: 1000
  weight_kinematics_forward_drive: 1
  weight_kinematics_turning_radius: 1
  weight_optimaltime: 1 # must be > 0
  weight_shortest_path: 0
  weight_obstacle: 100
  weight_inflation: 0.2
  weight_dynamic_obstacle: 10 # not in use yet
  weight_dynamic_obstacle_inflation: 0.2
  weight_viapoint: 1
  weight_adapt_factor: 2

  # Homotopy Class Planner

  enable_homotopy_class_planning: True
  enable_multithreading: True
  max_number_classes: 4
  selection_cost_hysteresis: 1.0
  selection_prefer_initial_plan: 0.95
  selection_obst_cost_scale: 1.0
  selection_alternative_time_cost: False

  roadmap_graph_no_samples: 15
  roadmap_graph_area_width: 5
  roadmap_graph_area_length_scale: 1.0
  h_signature_prescaler: 0.5
  h_signature_threshold: 0.1
  obstacle_heading_threshold: 0.45
  switching_blocking_period: 0.0
  viapoints_all_candidates: True
  delete_detours_backwards: True
  max_ratio_detours_duration_best_duration: 3.0
  visualize_hc_graph: False
  visualize_with_time_as_z_axis_scale: False

  # Recovery

  shrink_horizon_backup: True
  shrink_horizon_min_duration: 10
  oscillation_recovery: True
  oscillation_v_eps: 0.1
  oscillation_omega_eps: 0.1
  oscillation_recovery_min_duration: 10
  oscillation_filter_duration: 10

The other config files i have not changes apart from ajusting to the robots foot print.. Bellow is the output from the teb_local_planner also included in the log is the velocity and steering sent to the ackerman drive
[ WARN] [1603053883.684021065]: global_costmap: Pre-Hydro parameter "static_map" unused since "plugins" is provided
[ WARN] [1603053883.687325975]: global_costmap: Pre-Hydro parameter "map_type" unused since "plugins" is provided
[ INFO] [1603053883.689678261]: global_costmap: Using plugin "static_layer"
[ INFO] [1603053883.714386789]: Requesting the map...
[ INFO] [1603053885.119749955]: Resizing costmap to 106 X 88 at 0.050000 m/pix
[ INFO] [1603053885.219555149]: Received a 106 X 88 map at 0.050000 m/pix
[ INFO] [1603053885.227196674]: global_costmap: Using plugin "obstacle_layer"
[ INFO] [1603053885.232439371]:     Subscribed to Topics: laser_scan_sensor
[ INFO] [1603053885.268588499]: global_costmap: Using plugin "inflation_layer"
[ WARN] [1603053885.370971354]: local_costmap: Pre-Hydro parameter "static_map" unused since "plugins" is provided
[ WARN] [1603053885.371692269]: local_costmap: Pre-Hydro parameter "map_type" unused since "plugins" is provided
[ INFO] [1603053885.373204099]: local_costmap: Using plugin "static_layer"
[ INFO] [1603053885.380242417]: Requesting the map...
[ INFO] [1603053885.383072536]: Resizing static layer to 106 X 88 at 0.050000 m/pix
[ INFO] [1603053885.483166063]: Received a 106 X 88 map at 0.050000 m/pix
[ INFO] [1603053885.489507091]: local_costmap: Using plugin "obstacle_layer"
[ INFO] [1603053885.493790914]:     Subscribed to Topics: laser_scan_sensor
[ INFO] [1603053885.591892405]: Created local_planner teb_local_planner/TebLocalPlannerROS
[ INFO] [1603053885.689316396]: Footprint model 'two_circles' (front_offset: 0.3m, front_radius: 0.2m, rear_offset: 0m, rear_radius: 0.2m) loaded for trajectory optimization.
[ INFO] [1603053885.689512062]: Parallel planning in distinctive topologies enabled.
[ INFO] [1603053886.258780463]: Costmap conversion plugin costmap_converter::CostmapToLinesDBSRANSAC loaded.
[ INFO] [1603053887.559205945]: Recovery behavior will clear layer 'obstacles'
[ INFO] [1603053887.566967344]: Recovery behavior will clear layer 'obstacles'
[ INFO] [1603053938.091310244]: Resizing costmap to 107 X 88 at 0.050000 m/pix
[ INFO] [1603053938.091879784]: Resizing static layer to 107 X 88 at 0.050000 m/pix
[ INFO] [1603053943.091325121]: Resizing costmap to 108 X 88 at 0.050000 m/pix
[ INFO] [1603053943.091799120]: Resizing static layer to 108 X 88 at 0.050000 m/pix
[INFO] [1603053964.611536]: message:{"velocity": 0.0, "steering": 0.0}
[INFO] [1603053965.160557]: message:{"velocity": 0.19, "steering": 13.888723302019102}
[INFO] [1603053965.663271]: message:{"velocity": 0.19, "steering": 13.888723302019102}
[ INFO] [1603053966.091574623]: Resizing costmap to 110 X 89 at 0.050000 m/pix
[ INFO] [1603053966.092169413]: Resizing static layer to 110 X 89 at 0.050000 m/pix
[INFO] [1603053966.157327]: message:{"velocity": 0.10989223804505407, "steering": 22.2773236227058}
[INFO] [1603053966.657248]: message:{"velocity": -0.16800924618966745, "steering": 15.190033651758181}
[ INFO] [1603053967.093728727]: Resizing costmap to 137 X 101 at 0.050000 m/pix
[ INFO] [1603053967.094346892]: Resizing static layer to 137 X 101 at 0.050000 m/pix
[INFO] [1603053967.152982]: message:{"velocity": -0.09639740146564793, "steering": 25.08306404163337}
[INFO] [1603053967.653306]: message:{"velocity": 0.19, "steering": 13.888723302019102}
[INFO] [1603053968.153615]: message:{"velocity": 0.15516064931167317, "steering": 16.22307184744839}
[INFO] [1603053968.671711]: message:{"velocity": 0.1567401917013205, "steering": 15.917114169782378}
[ INFO] [1603053969.092987821]: Resizing costmap to 140 X 122 at 0.050000 m/pix
[ INFO] [1603053969.094202610]: Resizing static layer to 140 X 122 at 0.050000 m/pix
[INFO] [1603053969.138596]: message:{"velocity": 0.19, "steering": 13.888723302019102}
[INFO] [1603053969.645336]: message:{"velocity": 0.19, "steering": 13.888723302019102}
[ INFO] [1603053970.094072345]: Resizing costmap to 141 X 124 at 0.050000 m/pix
[ INFO] [1603053970.094764426]: Resizing static layer to 141 X 124 at 0.050000 m/pix
[INFO] [1603053970.152572]: message:{"velocity": 0.157085226037504, "steering": -15.87299020556166}
[INFO] [1603053970.652028]: message:{"velocity": 0.14958147768053567, "steering": 16.62645188424539}
[ INFO] [1603053971.093823412]: Resizing costmap to 141 X 124 at 0.050000 m/pix
[ INFO] [1603053971.094642785]: Resizing static layer to 141 X 124 at 0.050000 m/pix
[INFO] [1603053971.153260]: message:{"velocity": 0.17994646606031478, "steering": 14.465086504043498}
[INFO] [1603053971.648675]: message:{"velocity": 0.18221940256730185, "steering": 13.77384053597151}
[ INFO] [1603053972.094260937]: Resizing costmap to 143 X 124 at 0.050000 m/pix
[ INFO] [1603053972.095232143]: Resizing static layer to 143 X 124 at 0.050000 m/pix
[INFO] [1603053972.154345]: message:{"velocity": 0.19, "steering": 13.888723302019102}
[INFO] [1603053972.655336]: message:{"velocity": 0.19, "steering": 8.07889278095199}
[INFO] [1603053973.154964]: message:{"velocity": 0.142306098654834, "steering": -17.426177706739683}
[INFO] [1603053973.648808]: message:{"velocity": 0.14657802805769973, "steering": -16.950059361518434}
[INFO] [1603053974.154317]: message:{"velocity": 0.19, "steering": -13.888723302019102}
[INFO] [1603053974.638130]: message:{"velocity": 0.19, "steering": -13.888723302019102}
[ INFO] [1603053975.094530679]: Resizing costmap to 143 X 127 at 0.050000 m/pix
[ INFO] [1603053975.095246261]: Resizing static layer to 143 X 127 at 0.050000 m/pix
[INFO] [1603053975.144288]: message:{"velocity": 0.14957502857852584, "steering": -16.62711432427568}
[INFO] [1603053975.658574]: message:{"velocity": 0.17237280199585817, "steering": -14.544530994577318}
[INFO] [1603053976.146751]: message:{"velocity": 0.15681261959386938, "steering": -6.483617910525095}
[INFO] [1603053976.673788]: message:{"velocity": 0.16096289651362394, "steering": -15.62024572025678}
[ INFO] [1603053977.094809105]: Resizing costmap to 146 X 127 at 0.050000 m/pix
[ INFO] [1603053977.095565103]: Resizing static layer to 146 X 127 at 0.050000 m/pix
[INFO] [1603053977.174403]: message:{"velocity": 0.14946121800906448, "steering": -16.651839684197473}
[INFO] [1603053977.674965]: message:{"velocity": 0.14422454447112273, "steering": -17.23164445965255}
[INFO] [1603053978.172781]: message:{"velocity": 0.19, "steering": -13.888723302019102}
[INFO] [1603053978.673368]: message:{"velocity": 0.19, "steering": -13.888723302019102}
[INFO] [1603053979.174779]: message:{"velocity": 0.19, "steering": 3.8121634567749303}
[INFO] [1603053979.673570]: message:{"velocity": 0.15654355902563646, "steering": 16.12015239814894}
[INFO] [1603053980.172913]: message:{"velocity": 0.16180767019597248, "steering": 15.643644279449108}
[INFO] [1603053980.664372]: message:{"velocity": 0.1723745591007421, "steering": 14.765155688994692}
[ INFO] [1603053981.095889329]: Resizing costmap to 147 X 127 at 0.050000 m/pix
[ INFO] [1603053981.096645994]: Resizing static layer to 147 X 127 at 0.050000 m/pix
[INFO] [1603053981.162882]: message:{"velocity": 0.17121367741466736, "steering": 14.837118010477532}
[INFO] [1603053981.666922]: message:{"velocity": 0.16760487341146701, "steering": 15.129496346862569}
[INFO] [1603053982.163811]: message:{"velocity": 0.17466605325823645, "steering": 14.559698047581685}
[INFO] [1603053982.661197]: message:{"velocity": 0.16975629243693774, "steering": 14.952262868768113}
[INFO] [1603053983.157166]: message:{"velocity": 0.19, "steering": -13.888723302019102}
[INFO] [1603053983.666969]: message:{"velocity": 0.1593422437671581, "steering": -15.842045431465138}
[ INFO] [1603053984.096806945]: Resizing costmap to 147 X 127 at 0.050000 m/pix
[ INFO] [1603053984.097523527]: Resizing static layer to 147 X 127 at 0.050000 m/pix
[INFO] [1603053984.174264]: message:{"velocity": 0.19, "steering": -13.888723302019102}
[INFO] [1603053984.667880]: message:{"velocity": 0.18496460630770492, "steering": 4.345956139460004}
[INFO] [1603053985.161706]: message:{"velocity": 0.16663444001602204, "steering": 15.171797817902506}
[INFO] [1603053985.668552]: message:{"velocity": 0.17035605509820967, "steering": 15.01833994804229}
[INFO] [1603053986.162085]: message:{"velocity": 0.14602740780631962, "steering": 17.61115292457004}
[INFO] [1603053986.665442]: message:{"velocity": 0.19, "steering": 13.888723302019102}
[INFO] [1603053987.173497]: message:{"velocity": 0.19, "steering": -13.888723302019102}
[INFO] [1603053987.706693]: message:{"velocity": 0.19, "steering": -13.888723302019102}
[ INFO] [1603053988.097532295]: Resizing costmap to 148 X 127 at 0.050000 m/pix
[ INFO] [1603053988.098260752]: Resizing static layer to 148 X 127 at 0.050000 m/pix
[INFO] [1603053988.231491]: message:{"velocity": -0.11132887272481111, "steering": -22.879648171587206}
[INFO] [1603053988.742786]: message:{"velocity": -0.19, "steering": -13.888723302019102}
[INFO] [1603053989.275368]: message:{"velocity": -0.19, "steering": -13.888723302019102}
[ WARN] [1603053989.774449607]: TebLocalPlannerROS: trajectory is not feasible. Resetting planner...
[INFO] [1603053989.778356]: message:{"velocity": 0.0, "steering": 0.0}
[ INFO] [1603053990.097118347]: Resizing costmap to 152 X 127 at 0.050000 m/pix
[ INFO] [1603053990.097837637]: Resizing static layer to 152 X 127 at 0.050000 m/pix
[ WARN] [1603053990.206409771]: TebLocalPlannerROS: trajectory is not feasible. Resetting planner...
[INFO] [1603053990.208966]: message:{"velocity": 0.0, "steering": 0.0}
[INFO] [1603053990.681383]: message:{"velocity": 0.19, "steering": -13.888723302019102}
[INFO] [1603053991.226944]: message:{"velocity": 0.15722066497171583, "steering": -16.63715867104567}
[ WARN] [1603053991.737042366]: TebLocalPlannerROS: trajectory is not feasible. Resetting planner...
[ERROR] [1603053991.750099712]: Failed to get a plan.

Your robot position drifts a lot, which you can see base on the that laser scans move. Robot position drifting causes planner to change plane after every position estimation jump. You may try different mapping package to stabilize position estimation. I would recommend mapping package that takes odometry as input.


changing request parameters causes authentication failures

I'm posting it here, because I'm not getting much luck on devise, and I have a feeling it might be related to warden specifically. Either way, I'd appreciate some help.
I'm trying to normalize emails inside devise, by changing request params in the controller. e.g<EMAIL_ADDRESS>will be normalized to<EMAIL_ADDRESS>Whenever devise's current_user is accessed beforehand, it causes authentication to fail.
Reproducible problem at https://github.com/gingerlime/rails-devise/tree/reproduce (specifically this commit)
When anything accesses current_user (e.g. in a before_action inside the application controller), then:

authenticating with the real email<EMAIL_ADDRESS>works, but
authenticating with a different email<EMAIL_ADDRESS>that normalizes to<EMAIL_ADDRESS>fails...

I'd really appreciate some help on how to tackle this. I'm not particularly familiar with devise or warden's internals, but happy to do more digging if necessary. Just not too sure where to even start.

I investigated this problem and found out that warden caches the strategies in the methodclear_strategies_cache!, which is called by authenticate! but also when current_user is called, which happens most likely in some ApplicationController before filter. The password authentication strategy then already used the content of request.params which isn't normalized yet.
The solution would be this:
def create
  params[:user][:email] = normalize(params[:user][:email])
  request.params[:user].merge!(params[:user])
  warden.clear_strategies_cache!(:scope => :user)
  super
end

clear_strategies_cache! can be used without parameters or for a specific scope or even for a specific strategy.


Unify benchmark tool based on stresscli library

Description
Unify benchmark tool based on stresscli library
Issues
n/a
Type of change
List the type of change like below. Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds new functionality)
[ ] Breaking change (fix or feature that would break existing design and interface)

Dependencies
None
Tests
Local test.

Thank you for the issue @lvliang-intel. Does the usage guidance in README need to be changed accordingly?

Great work @lvliang-intel !
So the file stress_benchmark.py can be removed also?

Thank you for the issue @lvliang-intel. Does the usage guidance in README need to be changed accordingly?

Yes, README updated.

Great work @lvliang-intel !
So the file stress_benchmark.py can be removed also?

removed.


feat(*): auto complete

Adds the new auto-complete feature to various commands.

[x] tags (local)
[x] docs (org controlled)
[ ] algolia commands (write-ahead is exactly how the algolia API is meant to be used in websites) (due to fallback algolia needs to wait. (will potentially pass the entire guide URL as value, since content previews are not available on either guide or dapi docs))
[ ] remove workaround ephemeral selectors (once confirmed released and fully stable)
[x] tags: re-introduce discerning emojis (currently disabled due to upstream bug sending names instead of values)

:gear: the bot currently runs on this PR at b36c46c

❓ MDN and node might not be feasible to port to auto complete, as it could increase hits on their search endpoints potentially a lot (maybe with caching approaches)

ℹ Currently involves a fair bit of avoidable code duplication, do note however, that the ephemeral selection workflow is going to be removed after the feature is confirmed stable. (Currently still necessary to keep as fallback for unsupported devices and clients)

Tags are sorted by exact match in any keyword -> query included in any keyword -> query included in tag content. Pinned tags are controlled via the tag toml. and shown before input is given (specific pinned tags not yet set it stone)

Note: "doc gen" is included in category 2 (query included in any keyword) because of the alias keyword "djs docs"

Applying structural changes to the project as a whole, making it easier to expand in the future.
This update will come with AC as well, replacing this draft eventually


feat: add superbrothers/ksort

https://github.com/aquaproj/aqua-registry/pull/5360 superbrothers/ksort: Sort manfest files in a proper order by Kind
$ aqua g -i superbrothers/ksort

Thank you for your contribution!

Updated. 626a565
Uninstall kubectl plugin.
$ krew uninstall sort-manifests
Uninstalled plugin: sort-manifests

$ ls ~/.krew/bin

$ kubectl sort-manifests --help
error: unknown command "sort-manifests" for "kubectl"

$ ksort --help
When installing manifests, they should be sorted in a proper order by Kind.
For example, Namespace object must be in the first place when installing them.

ksort sorts manfest files in a proper order by Kind, which is implementd by
using sortManifestsByKind function in Kubernetes Helm.

Usage:
  ksort -f FILENAME [flags]

Examples:
  # Sort manifest files under the manifests directory, and output the result to the stdout.
  ksort -f ./manifests

  # To pass the result into the stdin of kubectl apply command is also convenient.
  ksort -f ./manifests | kubectl apply -f -

  # Sort manifests contained the manifest file.
  ksort -f app.yaml

  # Sort manifests in uninstall order.
  ksort -f ./manifests --delete

  # Sort manifests passed into stdin.
  cat app.yaml | ksort -f -

Flags:
      --add_dir_header                   If true, adds the file directory to the header of the log messages
      --alsologtostderr                  log to standard error as well as files
  -d, --delete                           Sort manifests in uninstall order
  -f, --filename strings                 Containing the resource to sort in a proper order by Kind
  -h, --help                             help for ksort
      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
      --log_dir string                   If non-empty, write log files in this directory
      --log_file string                  If non-empty, use this log file
      --log_file_max_size uint           Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited. (default 1800)
      --logtostderr                      log to standard error instead of files (default true)
      --one_output                       If true, only write logs to their native severity level (vs also writing to each lower severity level)
  -R, --recursive                        Process the directory used in -f, --filename recursively. Useful when you want to manage related manifests organized within the same directory.
      --skip_headers                     If true, avoid header prefixes in the log messages
      --skip_log_headers                 If true, avoid headers when opening log files
      --stderrthreshold severity         logs at or above this threshold go to stderr (default 2)
  -v, --v Level                          number for the log level verbosity
      --version                          Print the version and exit
      --vmodule moduleSpec               comma-separated list of pattern=N settings for file-filtered logging

$ k sort-manifests --help
When installing manifests, they should be sorted in a proper order by Kind.
For example, Namespace object must be in the first place when installing them.

ksort sorts manfest files in a proper order by Kind, which is implementd by
using sortManifestsByKind function in Kubernetes Helm.

Usage:
  ksort -f FILENAME [flags]

Examples:
  # Sort manifest files under the manifests directory, and output the result to the stdout.
  ksort -f ./manifests

  # To pass the result into the stdin of kubectl apply command is also convenient.
  ksort -f ./manifests | kubectl apply -f -

  # Sort manifests contained the manifest file.
  ksort -f app.yaml

  # Sort manifests in uninstall order.
  ksort -f ./manifests --delete

  # Sort manifests passed into stdin.
  cat app.yaml | ksort -f -

Flags:
      --add_dir_header                   If true, adds the file directory to the header of the log messages
      --alsologtostderr                  log to standard error as well as files
  -d, --delete                           Sort manifests in uninstall order
  -f, --filename strings                 Containing the resource to sort in a proper order by Kind
  -h, --help                             help for ksort
      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
      --log_dir string                   If non-empty, write log files in this directory
      --log_file string                  If non-empty, use this log file
      --log_file_max_size uint           Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited. (default 1800)
      --logtostderr                      log to standard error instead of files (default true)
      --one_output                       If true, only write logs to their native severity level (vs also writing to each lower severity level)
  -R, --recursive                        Process the directory used in -f, --filename recursively. Useful when you want to manage related manifests organized within the same directory.
      --skip_headers                     If true, avoid header prefixes in the log messages
      --skip_log_headers                 If true, avoid headers when opening log files
      --stderrthreshold severity         logs at or above this threshold go to stderr (default 2)
  -v, --v Level                          number for the log level verbosity
      --version                          Print the version and exit
      --vmodule moduleSpec               comma-separated list of pattern=N settings for file-filtered logging

👍 It works.

Released. https://github.com/aquaproj/aqua-registry/releases/tag/v3.25.0


conda not installing in ksh

Describe the bug
Installing conda in a ksh as opposed to bash flags up issues.
How to reproduce
Steps to reproduce the behaviour:
1.conda activate cset-dev
Expected behaviour
ksh[253]: local: not found [No such file or directory]
CondaError: Run 'conda init' before 'conda activate'
Environment

Version: [e.g. from cset --version]
Browser (if UI issue): [e.g. Chrome, Firefox]

The scripts within CSET explicitly use bash, so they shouldn't be at issue here. The issue is conda activate not supporting ksh. While it would be possible to hack together a solution to this, the real solution is just to switch everyone to bash as ksh will be removed when we switch from RHEL7 this year.
To switch you just need to ping Mark Crossland and he'll switch it for you. See https://engage.cloud.microsoft/main/threads/eyJfdHlwZSI6IlRocmVhZCIsImlkIjoiMjI2MzQ3MzgyMTEwNjE3NiJ9

The current workaround is to run a bash shell (with the bash command) explicitly before activating the conda environment.


[CO-390] WIP - Run Demo Cluster

Description
@disassembler, you'll recognize some parts of an old PR. I've resurrected it and adjusted a few things:

Now we pick a system temporary directory as a state-dir (unless given one)
Topology is generated on the fly
TLS certs are generated on the fly

This is still WIP but I'd appreciate your inputs before I move forward :heart:
Linked issue
[CO-390]
Type of change

[ ] 🐞 Bug fix (non-breaking change which fixes an issue)
[ ] 🛠 New feature (non-breaking change which adds functionality)
[ ] ⚠ Breaking change (fix or feature that would cause existing functionality to change)
[ ] 🏭 Refactoring that does not change existing functionality but does improve things like code readability, structure etc
[ ] 🔨 New or improved tests for existing code
[ ] ⛑ git-flow chore (backport, hotfix, etc)

Developer checklist

[ ] I have read the style guide document, and my code follows the code style of this project.
[ ] If my code deals with exceptions, it follows the guidelines.
[ ] I have updated any documentation accordingly, if needed. Documentation changes can be reflected in opening a PR on cardanodocs.com, amending the inline Haddock comments, any relevant README file or one of the document listed in the docs directory.
[ ] CHANGELOG entry has been added and is linked to the correct PR on GitHub.

Testing checklist

[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

QA Steps

Screenshots (if available)

NOTE (@disassembler): I haven't tested anything yet so don't try to run it ^^" ... It's more about grooming the general idea.

Just realised there's no need to run most of those actions in plain IO. The ENV can be read once and put in a Map, then we can use normal lenses to perform the same operation in a pure fashion.
Will do.

Still waiting for review from @akegalj, though this is needed for debugging and assessing results for a fix of 1.4.0.
It's quite isolated work so we can simply iterate with @akegalj 's feedback (if any).


getDirectories()

What feature do you want to get added on the base engine?
PLEASE NOTE THIS WILL NEED TO BE REFINED A LITTLE BIT BUT THE MAIN CONCEPT IS HERE...
getDirectories(path:string)
You give it a directory (preferably a folder) and it will return an array of all the items in that directory...
EXAMPLE:
function onCreate()
	local dir = getDirectories("weeks")
	for i, v in pairs(dir) do
		debugPrint(v)
	end
end

This code returns:
tutorial.json
week1.json
week2.json
week3.json
week4.json
week5.json
week6.json
week7.json
weekList.txt

To test your sight, and reliability, please select the option of what should NOT be requested.
Stage Editor, 6K+ support, and winning icons.

After pokin around in the source I've learned that directoryFileList() does exactly what I'm looking for...


Use bgpd and zebra sockets to collect bgp metrics

Use bgpd.vty and zebra.vty sockets to collect metrics as it greatly reduces scrape daration (1.5sec -> 0.0015sec).
I had to remove concurrency as it reduces reliability when reading from the socket directly. Sometimes when too many clients try to open socket it returns an error. It doesn't have a big impact on scrape duration as it is still acceptable.
@vinted/sre

You can use mutex lock/unlock when reading. this will allow you to wait for first read to finish and still allow concurrency.

@Seitanas thanks for a good tip. I will look into that in the second iteration of making this exporter better 🙂


support for amazon cloud drive?

https://developer.amazon.com/public/apis/experience/cloud-drive/

https://github.com/handyman5/acd_fuse

+1
I've tried acd_fuse, it seems sporadically slow and weird (I blame FUSE) and there's some visibility differences between the filesystem and the web UI on Amazon which I don't know who is to blame.
One thing I would request is before doing that, if there can be encryption applied beforehand.
file | encrypt > remote

A good idea - thanks. It certainly looks possible...
As for encryption - my plan would be to put that into rclone as a encrypting local filesystem which would make it available for all remote file system types.

+1
Going to check out the API right now and may submit a pull request within the next couple of weeks.

+1
Would be nice to see support for acd.

+1
The ACD App for mac provided by Amazon really sucks. I would really like to have rclone support for ACD.

https://github.com/sgeb/acdcli and its backing library https://github.com/sgeb/go-acd might help.

@jbuchbinder cool - thanks!

I'm using ACD_CLI, it's pretty good! Would be great to get this integrated into rclone :-) :+1:

Yes, please integrate into rclone :+1:

Support for Amazon Cloud Drive is now live in v1.20.  It should be pretty stable, but if you find any issues with it then you know what to do!
Thanks
Nick

Any chance of rclone latest ver being compiled for windows? Struggling with compiling on my end.

@OldGlory747 Find it on the downloads page http://rclone.org/downloads/ :-)

That was my mistake :) Was grabbing it from here:
http://www.craig-wood.com/nick/pub/rclone/windows/
Thanks much!!

@OldGlory747 Gosh, those are ancient builds - I didn't even know they were there!  Did you see a link to them somewhere I need to change?

Haha, nope, just good old fashion Google.
On Oct 7, 2015 1:51 PM, "Nick Craig-Wood"<EMAIL_ADDRESS>wrote:

@OldGlory747 https://github.com/OldGlory747 Gosh, those are ancient
builds - I didn't even know they were there! Did you see a link to them
somewhere I need to change?
—
Reply to this email directly or view it on GitHub
https://github.com/ncw/rclone/issues/45#issuecomment-146324180.

@OldGlory747 Ah OK!  I've deleted those, so hopefully they will disappear from the google search soon. Thanks for letting me know.


Added check to LibGit2SharpRepository to see if we are checking out a…

… tag instead of a branch #1435
To deploy a tag, the branch name passed to the function must start with "refs/tags/". From there, we can assume we want to deploy a tag instead of a branch. After testing locally, it seemed to work okay. I was able to deploy a master branch for example to Kudu, change the deployment source to a tag, create a tag locally, push to Kudu and see a successful deployment of the tag reference.
I also then created further commits on the master branch and pushed to Kudu. The tag reference was only ever deployed and not the further commits. My last test was to delete and re-create a tag, pushing to Kudu after. The deployment for the application deployed the updated tag reference.
I imagine I've not covered all bases but it's a start (I hope)

Hi @danlister, I'm your friendly neighborhood .NET Foundation Pull Request Bot (You can call me DNFBOT). Thanks for your contribution!

This seems like a small (but important) contribution, so no Contribution License Agreement is required at this point. Real humans will now evaluate your PR.

TTYL, DNFBOT;

Didn't get to play with it much, but the change seems unlikely to regress anything given the conditional. We should add coverage though.
Thanks for your contrib!

@danlister this is now deployed. Can you give it a try in Azure and make sure everything works as you expect? Thanks!

@davidebbo It seems to work as expected. Below were my test steps:

Navigate to a Web App on management.azure.com
Change the "Branch to Deploy" configuration property to a tag
Navigate back to portal.azure.com
Re-deploy the App Service

Perfect, thanks!


[do not merge] Typedoc->Gitbook Implementation for Typescript

Sample typedoc/gitbook implementation for an EOS typescript repository.
Typescript implementations are a bit more difficult due to lack of support for static files (tutorials, guides, explainer docs, etc). Solution would appear to be copying the files and then prepending the SUMMARY.md file dynamically based on files in docs/static, some ugly psuedocode
filesInStatic.each
  str += "[{{url.replace('-', ' ') }}]({{url}})"
Summary.prepend str

The above is not presently implemented due to a desire to maintain a separation of concerns between the repository and the documentation generation itself. There's some obvious solutions to this, from unideal and easy to ideal and difficult.

[unideal] Bash script in docs directory to handle copying and prepending generated SUMMARY.md. This is unideal because there's non-eosjs2 logic that needs to be maintained in the directory. Having the bash script executed in-line as an npm script has similar drawbacks
[acceptable] Write a typedoc plugin to make typedoc do what we want to do or modify typedoc-plugin-markdown (likely easier) There's a possibility this isn't possible with the Typedoc API and it creates maintenance overhead
[ideal] Create a standalone cli utility that generates documentation for any EOS repository for supported languages, removing any implementation requirements from the repository. This is probably the ideal route because then absolutely no documentation logic needs to exist in the repository outside of an optional configuration file. The tool simply takes arguments or reads a config file, parses files based on config and then places documentation into a directory

TODO:

Skin [This is a todo for all these implementations]

#45

Most pieces are in place, next I need to modularize some components used in here so they are reusable.

@dskvr This is looking great. I ran the build-serve-docs script, and the resulting documentation looks very nice.
After this is settled and merged, we can follow up with improvements to the documentation, itself. For example, in the API documentation, it's listing all of the API constructor params in a single block of text. We can improve this by moving those out into an ApiArgs interface. Will generate much nicer documentation.
Anyway, I'm in favor of proceeding with this approach after a fixup/squash of the commits herein. Nice work!

Closing in favor of #69