:arrow_up: [#93] Upgrade django to 4.2.17

Fixes #93 partially
Changes

Upgrade django to 4.2.17

There are several dependencies of dependencies that have vulnerabilities (e.g. flower->tornado, django->jinja2), but since we don't explicitly pin them here, I guess will upgrade them in each project with bin/compile-dependencies.sh -P ....

Now that I'm thinking about it, this does make this package feel kind of pointless, should I maybe pin these dependencies to the latest security release in open-api-framework as well? I could put them under a separate section to indicate that they are dependencies of dependencies? What do you think @annashamray?

Do you want to put them into optional dependencies?

I don't know if adding them to optional would be too useful right now, since they are already included implicitly (because we pin the dependency that installs it).
Once we fix https://github.com/maykinmedia/open-api-framework/issues/67 they should be moved to the dependency group that installs that dependency though

@stevenbal I agree and it feels like we would manage dependencies for external libraries
I'll approve this PR but not sure if we can effectively do something to django deps in OAF


Can't upload new package, after switch to MS

Hi
After switching from Nuget user to Microsoft account can't upload package.
The package ID 'Nlog.RabbitMQ.Target' is not available.

It looks like Nlog.RabbitMQ.Target is owned by the username "adolya". Could you log in using your Microsoft account and verify that the username in the top navbar is "adolya"? Perhaps a new account was created with a new username?
Try taking a look at our account management FAQ. This answers some common nuget.org account and Microsoft account questions:
https://docs.microsoft.com/en-us/nuget/faqs/nuget-faq#nugetorg-account-management

Yes, after linking MS account, it displays artdolya username...
But I was still able to login using old username

Okay, it looks like during the process of logging in with your Microsoft account, you created a new nuget.org user. From what I understand, you would like to keep you "adolya" username owning the "Nlog.RabbitMQ.Target" package but also enable Microsoft log in for this "adolya" account.
Our team received an account deletion request for your "artdolya" username. I have deleted this account. This should allow you to associate your Microsoft account with your "adolya" username.
Please let me know if you have any further questions.

Thanks, now it was linked correctly


[0.2.x] Creating theme is ~30-40x slower than alpha

Bug report
Describe the bug
Hey
So we have something like this implemented in our moderately large sized React app which is running on alpha version of stitches.
Earlier we tried migrating to the beta 0.1.x versions and quickly ran into perf issues.
It seems most of the issues have been solved since except for this one.
To Reproduce
Visit the below codesandbox and check console:
alpha: https://codesandbox.io/s/agitated-boyd-sxmql?file=/src/stitches.config.ts

beta (canary): https://codesandbox.io/s/vigorous-rubin-08xmr?file=/src/stitches.config.ts

Expected behavior
theme should go brrrr
Additional context
In continuation to https://github.com/modulz/stitches/issues/535#issuecomment-860943486

Ah! This was resolved in the v0.2.x version and remains resolved in v1. I see https://github.com/modulz/stitches/issues/535 was closed, so I’m going to close this one as well. Of course, please raise an issue if you see any regressions.

Hey @jonathantneal,
As noted in https://github.com/modulz/stitches/issues/535#issuecomment-856087834 and https://github.com/modulz/stitches/issues/535#issuecomment-860943486, this was in fact  not resolved. This issue was created on @peduarte's suggestion in the above mentioned comment on #535.
I know a lot of perf issues were resolved in #535 but I would love to know your thoughts on the possibility of any further improvements in the theme function. And whether it would be possible to get it anywhere to v0.1.x speed.

Hey 👋  — I'll pick up this thread since I have this situation loaded in memory.
Yes, the theme function did become slower in the beta due to picking up more functionality.
However, as far as I can tell, it's still very fast in real world applications, to the point it should never be the bottleneck.
Jonathan did add some caching after you provided the initial benchmark, so now the 1000 benchmark completes about  in 10ms on my machine (M1 Mac Mini).
Looking at your color picker use case, a single theme call, like in your use case, completes in 1/10th of a millisecond, which should be ok for your frame budget? If you're not already, can also consider throttling or raf'ing the change handler so it only runs every 16ms.
Please let me know if you aren't seeing the same thing and apologies for not communicating our take on this particular issue better.


Allow to search blog posts with unicode characters

This fixes #734.
The search engine is now more strict because it's case sensitive (if the post title is Lorém ipsum, lorém and Lorem gives no results; you must search for Lorém). I'd say it's OK because we're not building a real search engine. Thoughts?

Thanks for this.
Why did you remove mb_strtolower in extractSearchTerms though? Case insensitive searches works great with just your changes in sanitizeSearchQuery. It just won't work for uppercase non-ASCII characters in some DBMS like sqlite but it works out of the box with MySQL for example.

It is better to leave worrying about case sensitivity to database. Depending on configured collation it can compare strings according to local conventions. MySQL deals with this issue well, Sqlite not so much (at least in the default configuration) and it is ok — we should not expect much from the little Sqllite.
I think the strtolower is an ugly hack to fix incompetent database. I would not expect such a thing in example application.

@jlagneau I prefer to not add this warning message yet because that requires a new content to translate in all languages and because by default we use SQLite and non-accented characters, so this always work as expected for most users.
@jkufner I know that your comments were well intended, but I'd like to add a minor clarification about this --> "... to fix incompetent database"  We, the Symfony project, don't believe that SQLite is incompetent and we respect SQLite profoundly, as we do with all the other databases.


Use factory_boy to make model factories for writing tests

This is just a nice to have, to make our lives easier

I'm looking into this. Would like to learn the technology anyway.


Fixed #33618 -- Fixed MTI updates outside of primary key chain.

ticket-33618

@charettes Thanks :+1:


Test failure: VersionedApi_CanUseOrderToDisambiguate_OverlappingVersionRanges

This test fails occasionally with the following error:
Microsoft.AspNetCore.Routing.RouteCreationException : An error occurred while creating the route with name 'default' and pattern '{controller=Home}/{action=Index}/{id?}'.
---- System.InvalidOperationException : The route parameter 'controller' has both an inline default value and an explicit default value specified. A route parameter cannot contain an inline default value when a default value is specified explicitly. Consider removing one of them.
   at Microsoft.AspNetCore.Builder.MvcEndpointInfo..ctor(String name, String pattern, RouteValueDictionary defaults, IDictionary`2 constraints, RouteValueDictionary dataTokens, ParameterPolicyFactory parameterPolicyFactory) in /_/src/Microsoft.AspNetCore.Mvc.Core/Builder/MvcEndpointInfo.cs:line 39
   at Microsoft.AspNetCore.Builder.MvcApplicationBuilderExtensions.UseMvc(IApplicationBuilder app, Action`1 configureRoutes) in /_/src/Microsoft.AspNetCore.Mvc.Core/Builder/MvcApplicationBuilderExtensions.cs:line 109
   at VersioningWebSite.StartupWithEndpointRouting.Configure(IApplicationBuilder app) in /Users/asplab/TeamCity/work/33bdfc1cae7b2a38/modules/Mvc/test/WebSites/VersioningWebSite/StartupWithEndpointRouting.cs:line 27
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.AspNetCore.Hosting.ConventionBasedStartup.Configure(IApplicationBuilder app)
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.BuildApplication()
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.StartAsync(CancellationToken cancellationToken)
   at Microsoft.AspNetCore.TestHost.TestServer..ctor(IWebHostBuilder builder, IFeatureCollection featureCollection)
   at Microsoft.AspNetCore.Mvc.FunctionalTests.MvcTestFixture`1.CreateServer(IWebHostBuilder builder) in /_/test/Microsoft.AspNetCore.Mvc.FunctionalTests/Infrastructure/MvcTestFixture.cs:line 43
   at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.EnsureServer() in /_/src/Microsoft.AspNetCore.Mvc.Testing/WebApplicationFactory.cs:line 126
   at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.CreateDefaultClient(DelegatingHandler[] handlers) in /_/src/Microsoft.AspNetCore.Mvc.Testing/WebApplicationFactory.cs:line 335
   at Microsoft.AspNetCore.Mvc.FunctionalTests.VersioningTestsBase`1..ctor(MvcTestFixture`1 fixture) in /_/test/Microsoft.AspNetCore.Mvc.FunctionalTests/VersioningTestsBase.cs:line 19
----- Inner Stack Trace -----
   at Microsoft.AspNetCore.Routing.Patterns.RoutePatternFactory.<PatternCore>g__VisitPart|10_3(RoutePatternPart part, <>c__DisplayClass10_0& )
   at Microsoft.AspNetCore.Routing.Patterns.RoutePatternFactory.<PatternCore>g__VisitSegment|10_2(RoutePatternPathSegment segment, <>c__DisplayClass10_0& )
   at Microsoft.AspNetCore.Routing.Patterns.RoutePatternFactory.PatternCore(String rawText, IDictionary`2 defaults, IDictionary`2 parameterPolicies, IEnumerable`1 segments)
   at Microsoft.AspNetCore.Builder.MvcEndpointInfo..ctor(String name, String pattern, RouteValueDictionary defaults, IDictionary`2 constraints, RouteValueDictionary dataTokens, ParameterPolicyFactory parameterPolicyFactory) in /_/src/Microsoft.AspNetCore.Mvc.Core/Builder/MvcEndpointInfo.cs:line 24

Other tests within that build may have failed with a similar message, but they are not listed here. Check the link above for more info.
This test failed on master.
CC @mkArtakMSFT
This issue was made automatically. If there is a problem contact ryanbrandenburg.

Please use this workflow to address this flaky test issue, including checking applicable checkboxes and filling in the applicable "TODO" entries:

Is this actually a flaky test?

No, this is a regular test failure, fix the test/product (TODO: Link to commit/PR)
Yes, proceed below...

Is this test failure caused by product code flakiness? (Either this product, or another product this test depends on.)

[ ] File a bug against the product (TODO: Link to other bug)
Is it possible to change the test to avoid the flakiness?

Yes? Go to the "Change the test!" section.
No?

[ ] Disable the test (TODO: Link to PR/commit)
[ ] Wait for other bug to be resolved
[ ] Wait for us to get build that has the fix
[ ] Re-enable our test (TODO: Link to PR/commit)
[ ] Close this bug

Is it that the test itself is flaky? This includes external transient problems (e.g. remote server problems, file system race condition, etc.)

Is there is a way to change our test to avoid this flakiness?

Yes? Change the test!

[ ] Change the test to avoid the flakiness, for example by using a different test strategy, or by adding retries w/ timeouts (TODO: Link to PR/commit)
[ ] Run the test 100 times locally as a sanity check.
[ ] Close this bug

No?

Is there any logging or extra information that we could add to make this more diagnosable when it happens again?

Yes?

[ ] Add the logging (TODO: Link to PR/commit)

No?

[ ] Delete the test because flaky tests are not useful (TODO: Link to PR/commit)

This comment was made automatically. If there is a problem contact ryanbrandenburg.

VersionedApi_CanReachOtherVersionOperationsWithParameters_OnTheSameController failed with about the same error on master.
This comment was made automatically. If there is a problem contact ryanbrandenburg.

@JamesNK \ @rynowak could you have a look?

This was not flakiness, it was a regression that was fixed a few weeks ago 👍


ci: test for openjdk 21 (LTS)

Pull Request Test Coverage Report for Build #348
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 72.606%

Totals

Change from base Build #347:
0.0%

Covered Lines:
872

Relevant Lines:
1201

💛  - Coveralls


Score2Perf and relative positional encoding

Hello,
Reporting a possible issue with Score2Perf and tensor2tensor.
Based on its blog post, my understanding is that Music Transformer uses self_attention_type = dot_product_relative_v2.
Looking at the implementation of dot_product_self_attention_relative_v2() in tensor2tensor codebase, it sounds as if the intention was to only look at past tokens (to the left of any token). However, there are multiple lines in the code (see below) that allow a token to look at future tokens, because of the absence of proper masking. I have marked these lines with a numbered comment .
logits = tf.matmul(q, k, transpose_b=True)    # 1
key_relative_embeddings = get_relative_embeddings_left(
    max_relative_position, length, depth_k, num_heads,
    heads_share_relative_embedding, "key_relative_embeddings")

rel_logits = matmul_with_relative_keys(q, key_relative_embeddings,
                                       heads_share_relative_embedding)  # 2
rel_logits = _relative_position_to_absolute_position_masked(rel_logits)
logits += rel_logits     # 3
if bias is not None:
  logits += bias    # 4

weights = tf.nn.softmax(logits, name="attention_weights")

Shouldn't the upper triangle of logits be replaced with very small negative numbers before being passed to softmax? I do see that kind of logic used in other types of attention in the tensor2tensor code (e.g., using attention_bias_lower_triangle()) , but not in the above code.
Meanwhile, I am training a model with the original code above: 50K steps of training so far, eval NLL quite higher than reported in the paper, generated performances a bit too much structured and taking a very long time to generate on a new Mac. My main concern is that in the absence of masking in the above code, the eval NLL would not be accurate, because while during training/eval tokens pay attention to all future tokens in the complete sequence, that is not the case during generation, because of the autoregressive nature of generation (though, with the above code, even during generation each token will pay attention to all future tokens generated so far that are ahead of it). In the absence of masking attention to future tokens, the correct way of measuring eval NLL would be to run the model auto-regressively. And yet, the model would not be trained correctly anyway, because it won't be causal. Is your paper/results based on a different/corrected implementation of above code? Or is the absence of masking intentional, because in spite of the technical issues the result still works rather well? If so, was the reported eval NLL correctly measured?
Would appreciate hearing from you, and about your plans to update/release code for Music Transformer.
Thanks.

Please ignore the above! I had missed that the passed in bias doubles as the mask and is supposedly lower-diagonal.
Best.


chromebook dpad/stick issues

I am currently playing on a chromebook and am trying to play a game called the new tetris, but for some reason it just cant read the stick and dpad inputs (im using a keyboard, i use arrow keys for dapad and nothing happens same for cstick controls), all the other button controls work, just not dpad or stick

Yes the D-pad on keyboard currently does not work, it is on my list of things to fix. The stick buttons should work though, if you mapped your arrow keys to the up/down/left/right that translates to stick movements.

I added the D-pad controls, you can try it here
https://neilb.net/n64wasm/v2/


No license and copyright information in the files

Since this is an open Source compliance toolkit, I would love to see the best practises for a good OSS project realized in this project.
Some of the best practises are:

Put a copyrihgt notice in each file
Put the relevant Strandard License Header in file (or a statement like This file is licensed under --SPDX Short Identifier --

The FSFE hast set up a Project with best practizes relevant for smart license compliance support, the proejct is called reuse. Please see https://reuse.software/
No file of the current release carries a standarde license haeder, there is no license information in the files at all. The same applies to the copyright information, except the README.md file.
Regarding the README.md file, please remove the statement "See the LICENSE file in the root of this project for license details." because no license scanner can "auto conclude" such a statement.
The file oss-review-toolkit-preview/analyzer/src/funTest/assets/projects/synthetic/mixed/NPMNestedMaven/package.json
{
"name": "npmnestedmaven",
"version": "1.0.0",
"description": "A dummy NPM project with a nested Maven project",
"scripts": {
"test": "echo "Error: no test specified" && exit 1"
},
"author": "Sebastian Schuberth",
"license": "ISC",
"repository": {
"type": "git",
"url": "https://github.com/heremaps/oss-review-toolkit.git"
}
}
Carries a license information which doea not fit. Even in case of test files you should care for license hygiene

Thanks for pointing out https://reuse.software/! We'll certainly follow best practices regarding copyright / license headers and test data once we publish the final history.
I believe we should keep the reference from README.md to LICENSE. This is a hint for humans only, as GitHub only shows the contents of README.md on the front page, but inlining LICENSE into README.md does not really make sense.

We just pushed the final history to the master branch which has the copyright / license headers added.


traefik: do not leak the private keys

Signed-off-by: Christian Berendt<EMAIL_ADDRESS>

Let's still do this now, seems easy enough


pep8 fixes.

Fixes all of the (minor) pep8 warnings when scanning with flake8: whitespace, column width warnings and an "unused import" warning.

Neat, thank you. I need to add my pep8 checker to a pre-commit hook. :-)

:)


[Parser] add benchmarks

looking good @hereisnaman ! at first glance, there are these issues:

package-lock.json file appears in the PR, which doesn't work because we are using yarn workspaces
formatting errors occur in CI, you need to run yarn format locally first

@acao Thanks for the review. I have updated the PR.


Directories raise "Error: no write since last change"

Directories that have been browsed raise "Error: no write since last change" on quit.  Directories are not editable in vim.  Quitting again closes the directory, but other directories can also raise this error.

did you modify those directory buffers?

@chrisbra What does that mean?

Anyway, the directory buffers are read only, so I don't see why they should be allowed to raise that error?

looks like something (a plugin perhaps?) is setting the modified setting on those buffers, hence Vim will complain. Can you check, if this happens when you start Vim with vim -u NONE -N
Also you might want to check with a recent Vim (you didn't say what version you are using) with updated runtime files, as well try charles latest netrw from here
(I assume, this happens with the builtin file explorer and not any other plugin. If that is the case, please report as bug there).

I don't see this problem using Linux and the latest netrw (v162c).  I suggest upgrading (http://www.drchip.org/astronaut/vim/index.html#NETRW).  If you persist in having this problem, please give us a specific set of commands/actions that evoke the issue so I can try to duplicate it.

@cecamp what does netrw have to do with this?

netrw is used for browsing directories.

How are you browsing directories?  If you type ":e .", for example, netrw (by default) is running and showing a display of the directory contents.  There are other tools to do directory browsing; if you're using one of them, please be specific and let their authors know.

Oh!  I didn't realize what it was called. Yes, I'm doing ":e ." as you said.  I can't seem to figure out how to upgrade.  I am using MacVim 8.0 installed with MacPorts.

Visit http://www.drchip.org/astronaut/vim/index.html#NETRW and click the link just below the header "Network Oriented Reading, Writing, and Browsing" to download the file http://www.drchip.org/astronaut/vim/vbafiles/netrw.vba.gz .
Then, if you are using Safari, the file is to be downloaded and unzipped in the Downloads folder; you'll find netrw.vba there.
Open MacVim and have it read the file.
Next, type :so %, which extracts files from netrw.vba and put those extracted files in place (most likely into ~/.vim).
That's it. :)
As to uninstallation, you can use :RmVimball netrw.
What is used to install netrw here is Vim's native package management system for plugins, called Vimball.   For details, :help vimball.

Thanks, that makes sense.  I hope the vundle mirror gets updated (https://github.com/eiginn/netrw/issues/17)

@nuko8 Okay, I followed your instructions.  How do I check the version?

Hi Neil

How do I check the version?

:echo g:loaded_netrwPlugin
--
Best regards,
Hirohito Higashi (a.k.a. h_east)

Or, open MacVim and type :Explore.
You should see (netrw v162d) on the top-right corner of the window.  If not and you only see a file listing, type I to toggle the banner.

Thank you, I have 162d.  If this error happens again, I'll reopen this bug.


fix: tabBar component example

the current example is passing a wrong type of argument to the navigate method.

Thanks for the PR, however the current code is correct and this change is incorrect.


Add more docs

I have been working with OpenSfM, and needed an understanding of code that is overwhelmingly un-documented. After spending many weeks gradually building up a rough picture, I wrote up what I'd learned as private documentation for my company.
However, I was filling a gap here and if possible I'd like to spare others the same work. What I've written is "figured out" from the code and is not written from a designer's standpoint. I hope it will be useful as at least a first step towards more complete documentation of some fundamental parts of this library. Please edit for corrections and style / tone as you see fit.

That's awesome, @jdjakub! Thanks for the effort. It will be useful to others. Will merge and edit.


application.db is growing too quickly

application.db seems to be growing too quickly.
It's currently 29GB in size.
29G	./application.db
1.3G	./snapshots
2.7G	./blockstore.db
3.8G	./state.db
22G	./tx_index.db
36K	./evidence.db
1021M	./cs.wal
59G	.

The /home/zetachain/.zetacored/data directory has grown by about 55GB in the last 30 days even with pruning turned on. That'll be 660GB/year if growth continues at this pace.
### Tasks
- [ ] https://github.com/zeta-chain/node/issues/942
- [ ] https://github.com/zeta-chain/node/issues/2781
- [ ] https://github.com/zeta-chain/node/issues/2885

Bumping this issue
Data size from from Mainnet - June 2024
1.1G    ./cs.wal
20K     ./snapshots/metadata.db
24K     ./snapshots
20K     ./evidence.db
697G    ./tx_index.db
2.1T    ./application.db
91G     ./blockstore.db
229G    ./state.db
3.0T    .

Bumping this issue
Data size from from Mainnet - June 2024
1.1G    ./cs.wal
20K     ./snapshots/metadata.db
24K     ./snapshots
20K     ./evidence.db
697G    ./tx_index.db
2.1T    ./application.db
91G     ./blockstore.db
229G    ./state.db
3.0T    .

This is very big; what's the pruning setting
and what is the first block available for this node? This can be queried like
curl https://zetachain-mainnet.g.allthatnode.com/archive/tendermint/status | jq '.result.sync_info.earliest_block_height'

Presently , the full-node is 27 gb where as the archive is 3.5 tb
Posting the analysis for the full node here

I have not been able to run the same script to analyze the archive data , as it runs out of memory , but we can assume that it would follow a similar distribution

The parts of this data that we can consider cleaning up are
Observer : Ballots
We can clean up matured ballots easily : // https://github.com/zeta-chain/node/issues/942
Crosschain : CCTX
We should be able to clean up cctxs which have been saved to outboundmined and reverted, as we are already storing the finalization index , I don't think we need to store the entire CCTX as well
https://github.com/zeta-chain/zeta-node/blob/1ad2bde785c62698bce88c0f1171c985cb1f93c1/x/crosschain/keeper/finalized_inbounds.go#L15-L18

What is the difference between average and total value size?
Like lightclient the average is 0B but total 13B while sometime total is smaller

interesting,
any way that we can see similar result for "growth"?
I think the full node snapshot represents the size of state;
More concern is probably the archive node size which contains all history.

at is the difference between average and total value size?

The avg is just total /divided by the count, the 0 is just because of using integer and the decimal part being truncated

interesting, any way that we can see similar result for "growth"?
I think the full node snapshot represents the size of state; More concern is probably the archive node size which contains all history.

Not sure what you mean , the state.db would represent the current state , this data is for application.db only.
I think the only difference between archive and nonarchive nodes would be the fact that nonarchive nodes do not have all the history; it would only have applications.db data going back to a certain height instead of height 1

Did some research around tx_logs
We use the same design as etherming evm.
The only things written to the application db are LogSize and Blockbloom
https://github.com/zeta-chain/zeta-node/blob/c3d19297b4d345e9d7ff746f510a0e3e25eb6355/x/fungible/keeper/evm.go#L831-L832
Moreover, they are written to the transient store, so it's discarded at the end of every block.
The actual logs, however, are part of the events, which are part of the tx_result. This gets written to the tx_index.db , which is different from the application db, but that can be a cause for concern as well, the tx_index.db is around ~7 gb , in the snapshot I am using for testing; as opposed to the 27GB application db . This is like almost 1/3, which is a bit alarming, considering the fact that number of mainnet tx is expected to go up from here .
Posting about the DBs for context
application.db -> cosmos and zetachain related state data  (cosmos)  [ Grows with the data we save , immediate concern]
tx_index.db -> transactions indexed by hash and height (comet-bft) [ grows with the number of txs that happen , next possible concern]
state.db -> state related to consensus , validators etc (comet-bft) [ would depend on valdiators , and changes in validator set ]
evidence.db -> evidence store (comet-bft) [ Not sure , would need more research , but not a concern for us ]
blockstore.db -> blocks , block commits etc (comet-bft) [ Grows as the blockchain gets older , might be a concern down the line in a few years ]

Posting about the DBs for context

application.db -> cosmos and zetachain related state data  (cosmos)  [ Grows with the data we save , immediate concern]
tx_index.db -> transactions indexed by hash and height (comet-bft) [ grows with the number of txs that happen , next possible concern]
state.db -> state related to consensus , validators etc (comet-bft) [ would depend on valdiators , and changes in validator set ]
evidence.db -> evidence store (comet-bft) [ Not sure , would need more research , but not a concern for us ]
blockstore.db -> blocks , block commits etc (comet-bft) [ Grows as the blockchain gets older , might be a concern down the line in a few years ]

Posting data from a archive mainnet node
+--------------+--------------+----------------+----------------+------------------+-----------------+
| MODULE       | AVG KEY SIZE | AVG VALUE SIZE | TOTAL KEY SIZE | TOTAL VALUE SIZE | TOTAL KEY PAIRS |
+--------------+--------------+----------------+----------------+------------------+-----------------+
| acc          | 48 B         | 64 B           | 454.3 GB       | 597.7 GB         |<PHONE_NUMBER> |
| authority    | 23 B         | 32 B           | 28.1 MB        | 39.2 MB          |         1223875 |
| authz        | 19 B         | 21 B           | 81.1 MB        | 93.5 MB          |         4256295 |
| bank         | 49 B         | 64 B           | 243.5 GB       | 316.6 GB         |<PHONE_NUMBER> |
| crosschain   | 55 B         | 105 B          | 12.5 GB        | 23.8 GB          |       224964010 |
| distribution | 57 B         | 59 B           | 229.6 GB       | 237.2 GB         |<PHONE_NUMBER> |
| emissions    | 48 B         | 76 B           | 1.1 GB         | 1.7 GB           |        22117319 |
| evidence     | 22 B         | 0 B            | 93.5 MB        | 13 B             |         4249350 |
| evm          | 48 B         | 78 B           | 393.7 GB       | 631.9 GB         |<PHONE_NUMBER> |
| feemarket    | 50 B         | 41 B           | 1.5 GB         | 1.2 GB           |        29745434 |
| fungible     | 22 B         | 21 B           | 93.5 MB        | 89.8 MB          |         4249657 |
| gov          | 19 B         | 34 B           | 88.7 MB        | 158.8 MB         |         4584770 |
| group        | 19 B         | 32 B           | 82.6 MB        | 138.8 MB         |         4285923 |
| lightclient  | 25 B         | 0 B            | 30.6 MB        | 13 B             |         1223873 |
| misc         | 8 B          | 905 B          | 40.4 MB        | 4.2 GB           |         4612073 |
| observer     | 53 B         | 102 B          | 7.7 GB         | 14.8 GB          |       144684798 |
| params       | 20 B         | 32 B           | 85.0 MB        | 136.0 MB         |         4249605 |
| slashing     | 53 B         | 64 B           | 78.0 GB        | 92.7 GB          |<PHONE_NUMBER> |
| staking      | 52 B         | 501 B          | 13.0 GB        | 124.0 GB         |       247116237 |
| upgrade      | 21 B         | 22 B           | 89.3 MB        | 96.1 MB          |         4250072 |
+--------------+--------------+----------------+----------------+------------------+-----------------+
| TOTAL        |              |                | 1.4 TB         | 2.0 TB           |<PHONE_NUMBER>3 |
+--------------+--------------+----------------+----------------+------------------+-----------------+

Interesting to see the weekly growth

Distribution module seems tobe growing at the fastest rate

Distribution module runs the following function
func (k Keeper) AllocateTokens(ctx sdk.Context, totalPreviousPower int64, bondedVotes []abci.VoteInfo)

This function is essentially responsible for creating allocations , which updates the commission and rewards for each validator.

This amount is based on whatever is collected in the fee pool for the previous block.

Note that the allocations will be created irrespective of the amount collected in the fee pool.

The contributors to fee pool are
a) Tx gas fees
b) Emissions

Comparison with other cosmos SDK project archive node
https://www.notion.so/zetachain/Comparisons-caf5a427493942ea8e75bef4d33376d6?pvs=4

@lumtis @CharlieMc0  .
Added 3 tasks we can immediately do in v20, to help with the growth.
We can keep this issue open to continue tracking the growth after we implement the changes

Pushed back the issue to October release to revise the consideration ion the future.
I think we should let the current validator number and cctx in the store.
Gonna have for this release a new check for the ballot clean up refactoring.

Bumping this. Our mainnet archive snapshots will pass 5 TB in the next week or so.

Can we try updating the MaxValidators and see how much it helps with the growth, It would not reduce the exisiting size , but should help the month over month increase


sql, jobs: stop queuing schema change jobs for in-txn schema changes

PR: https://github.com/cockroachdb/cockroach/pull/58888
From release notes:

Creating a table and changing its schema within a transaction no longer schedules a schema change job. [#58888][#58888] {% comment %}doc{% endcomment %}

I don't think this requires documentation... We don't specifically list the changed behavior in the limitations of online schema changes. cockroachdb/cockroach#58888 seems to be more of a bug fix.


Reopening a notebook from the running panel doesn't work for jupyter drive managed files.

Under the running panel in JupyterLab, if you select the "Kernels" section and view the files attached to the kernels. If you click on a notebook that is in the jupyter-drives, it doesn't reopen as expected.

Here is the UI that I am referring to:

Interestingly, the "recently closed" section links lower in that panel do work.


host the tarballs somewhere else?

Could put them on retro.haleblian.com and pull them from there.
Then, copying the repo to the host would be less onerous.

Uploaded them all to http://retro.haleblian.com/backstep/warehouse .
But actually considering fetching them with wget instead when the source is not likely to disappear.


Consumer stuck in infinite loop (?)

Hello,
I am having some trouble with what I believe to be a rather simple pykafka use case.
I have set up a loop that goes on retrieving messages from a topic (one producer, one partition, no replication, only one consumer having exclusive access to the server), but sometimes it sort of gets stuck in a loop.
I have wrapped the usage of pykafka within a class of my own that, among other things, prints the offset of the retrieved message along with the first characters of its body and from time to time the offset coming out are arranged in an endlessly repeating sequence like:
...
99
100
101
102
103
104
101 ( x )
102
103
104
101 ( x )
102
103
104
102 ( xx )
103
104
101 ( xxx )
102
103
104
...
(I omit the text of the messages, they correspond 1:1 to the offsets).
(In real cases the loop has an extension up to few hundreds of messages. My current offsets are in the half-millions range).
This happens regardless of whether I manually invoke commit_offsets() or set auto commit with a few seconds interval; the exact offset where the sequence restarts is not always the same (e.g. above it restarts from 101 in places marked x and xxx but from 102 in xx ).
I believe this is a very strange behaviour since it forces me to manually store the highest offset seen so far and check against it: whenever I detect this infinite looping behaviour, since commits are not effective, I have to stop(), wait some seconds, and start() again the simple_consumer.
I noticed this tends to happen (but perhaps not exclusively) when the consumer has reached the current end of the topic: however, if new messages are produced to it when the consumer is looping, this is not sufficient to make it exit towards the fresh messages.
Here are the kafka_server/pykafka/python versions I tested this on:

kafka server version <IP_ADDRESS>
Python 2.7.10 (default, Oct 14 2015, 16:09:02)  [GCC 5.2.1 20151010] on linux2
pykafka==2.2.1

I provide below what I deem the relevant parts of the class that provides an interface between pykafka and my other code. I'd be glad if someone can point me to the right direction -- it might well be that this is indeed the desired behaviour, in which case I may just need to rethink some parts of my code.
CLASS CODE:
from pykafka import KafkaClient
kserver_default_string='RE.DA.CT.ED:port'

socket_timeout_ms=30000
offsets_channel_socket_timeout_ms=10000

class kafka_agent:

    def __init__(self, topic, kserver=ks_default_string, consumer_timeout_ms=350):
        self.topic=topic
        self.kserver=kserver
        self.kclient=KafkaClient(hosts=kserver,socket_timeout_ms=socket_timeout_ms,offsets_channel_socket_timeout_ms=offsets_channel_socket_timeout_ms)
        self.ktopic=self.kclient.topics[topic]
        self.kconsumer=self.ktopic.get_simple_consumer(consumer_group='cgroup_'+topic,
            consumer_timeout_ms=consumer_timeout_ms,auto_commit_enable=True, auto_commit_interval_ms=10000)

    def retrieve(self, block=True):
        if not self.kconsumer._running:
            self.kconsumer.start()
        tmsg=self.kconsumer.consume(block=block)
        if tmsg is not None:
            tstring=tmsg.value
            toff=tmsg.offset
            return tstring
        else:
            return None

    def commit_last_retrieve(self):
        self.kconsumer.commit_offsets()

    def isrunning(self):
        return self.kconsumer._running

    # those are for usage by context managers
    def __exit__(self, exc_type, exc_value, traceback):
        self.kconsumer.stop()

    def __enter__(self):
        self.kconsumer.start()

USER CODE: something along the lines of:

with kafka_agent('TOPIC_NAME') as kagent:
    while True:
        obj=kagent.retrieve()
        if obj is not None:
            # do stuff with obj
        else:
            # None was retrieved
        kagent.commit_last_retrieve()

===
Thanks a lot!

Hi @hemidactylus. I don't see anything in your code that I'd expect to cause this looping behavior. If there's some place in your code where you're destroying and recreating the SimpleConsumer instance, it's definitely possible for this issue to occur. Maybe you can figure out if you've got any logic in your code that would cause these checks not to function as intended.

Hallo @emmett9001 and thanks for the quick reply! I do not think the consumer is reinstantiated at a fast pace, however I noticed that the code as it is issues a lot of commit_offsets() calls, up to several per second over the internet to the server. I suspect that may somehow lead to more than one commit traveling at the same time to the server and piling up in an unclean way, so that the local client code may end up retaining offset information that does not correspond to the latest commit I sent.
Considering I set auto_commit_enable=True, I am now testing the same code as above with no commits by hand and the issue seems not to show up anymore. I'll let you know if I find something that might be related to wrongdoings taking place outside of my wrapper class.
Thanks again and have a nice day!
Stefano


How to integrate Report NG with jenkins?

Or it will be same as default Test NG report works it will automatically take ReportNG reports as Test Results?

You don't need to do anything extra for you to be able to generate ReportNG reports (since its added via Service Providers). You should be able to find the ReportNG reports in the target/surefire-reports/

My query was related to Jenkins Integration though. Do I need to add any new plugin there to enable to see this report?

@aliasgerkw - There is no plugin etc for this. Whatever mechanism you are using to have Jenkins show the default HTML report that comes baked into TestNG, you would use the same approach to show this report as well.
Not every report needs a Jenkins plugin for it to be rendered. Its true in the case of sophisticated reporting mechanism such as allure because, the html gets generated in runtime. But that's not the case with ReportNG.

Thanks @krmahadevan for the clarification


ohai_plugin recipe can't be used with chef-solo

Cookbook version
7.0.0
Chef-client version
13.2.20
Platform Details
Ubuntu 16.04
Scenario:
Attempting to use the cookbook with chef-solo (chef-client -z) triggers a fatal error in the ohai_plugin resource.
Steps to Reproduce:
Execute a chef-client run in local mode (chef-client -z). May need the --disable-config flag to trigger the issue
Expected Result:
No fatal errors are raised
Actual Result:
ohai_plugin resource raises an error.

I think the most appropriate (and simple) fix here is to add an optional attribute to the cookbook that will supply the path value to the ohai_plugin resource.

I use chef-solo with berkshelf. My cookbook_path has two folders berks-cookbooks, and cookbooks. All stock cookbooks downloaded from supermarket are stored in berks-cookbooks. Any customizations are done using wrapper cookbooks stored in cookbooks folder.
the stock nginx cookbook is in berks-cookbooks along with dependencies including ohai cookbook.
I have a wrapper cookbook called _nginx inside cookbooks folder, for my customizations. My recepies never call nginx directly but always call it via _nginx.
The problem happens because nginx cookbook is creating a folder called ohai inside my cookbooks folder for the ohia plugin. However this creates a conflict since there is already an ohai cookbook inside the berks-cookbooks.
Adding a path attribute to the ohai_plugin block of the nginx/recipes/ohai_plugin.rb could have solved the problem but I do not see a way to do it in my wrapper cookbook, without also modifying the stock nginx cookbook.
Any Suggestions?

I was able to get around this by creating an empty config file and then providing it to the chef-client via the --config or -c command line flag.
$ chef-client -z -c client.rb

Closing due to inactivity.
If this is still an issue please reopen or open another issue. Alternatively drop by the #sous-chefs channel on the Chef Community Slack and we'll be happy to help!
Thanks,
Sous-Chefs


Does fine-tuning support the multi-GPU training?

Does fine-tuning support the multi-GPU training?
When trying to fine-tune with multiple GPUs, got the following error.

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0!

File "/home/ubuntu/miniconda3/envs/mpt/lib/python3.10/site-packages/transformers/trainer.py", line 1664, in train
    return inner_training_loop(
  File "/home/ubuntu/miniconda3/envs/mpt/lib/python3.10/site-packages/transformers/trainer.py", line 1940, in _inner_training_loop
    tr_loss_step = self.training_step(model, inputs)
  File "/home/ubuntu/miniconda3/envs/mpt/lib/python3.10/site-packages/transformers/trainer.py", line 2735, in training_step
    loss = self.compute_loss(model, inputs)
  File "/home/ubuntu/miniconda3/envs/mpt/lib/python3.10/site-packages/transformers/trainer.py", line 2767, in compute_loss
    outputs = model(**inputs)
  File "/home/ubuntu/miniconda3/envs/mpt/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/ubuntu/miniconda3/envs/mpt/lib/python3.10/site-packages/peft/peft_model.py", line 678, in forward
    return self.base_model(
  File "/home/ubuntu/miniconda3/envs/mpt/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/ubuntu/miniconda3/envs/mpt/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward
    output = old_forward(*args, **kwargs)
  File "/home/ubuntu/miniconda3/envs/mpt/lib/python3.10/site-packages/falcontune-0.1.0-py3.10.egg/falcontune/model/falcon/model.py", line 1070, in forward
    transformer_outputs = self.transformer(
  File "/home/ubuntu/miniconda3/envs/mpt/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/ubuntu/miniconda3/envs/mpt/lib/python3.10/site-packages/falcontune-0.1.0-py3.10.egg/falcontune/model/falcon/model.py", line 965, in forward
    outputs = block(
  File "/home/ubuntu/miniconda3/envs/mpt/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/ubuntu/miniconda3/envs/mpt/lib/python3.10/site-packages/falcontune-0.1.0-py3.10.egg/falcontune/model/falcon/model.py", line 720, in forward
    mlp_output += attention_output
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0!

Same issue for me as well.

Am just trying with single gpu for now but running into OOM issue - https://github.com/rmihaylov/falcontune/issues/19

I have the same issue , device_map=auto does not work for training. I guess we should move tokenizer to all devices ? Why does device_map=auto can not handle this ?

Same here. While training on multi-GPU I get the following error:
ValueError: DistributedDataParallel device_ids and output_device arguments only work with single-device/multiple-device GPU modules or CPU modules, but got device_ids [0], output_device 0, and module parameters {device(type='cuda', index=0), device(type='cuda', index=1)}.

It seems it doesn't rely on Accelerate framework.
An easy quick win would be to rely on the new HuggingFace SFT Trainer instead: https://huggingface.co/docs/trl/sft_trainer


add language PR

I can add another language (tested on audio messages), but I don't know the best way to add a language selection feature.
I saw a PR with adding Korean (https://github.com/Xewdy444/Playwright-reCAPTCHA/pull/43), but it was rejected for some reason. Maybe the author can tell me how he wants to implement multi-language?

I would be willing to implement this. The author of that pull request closed it as it was incomplete, however his idea can still be used. I will need translations for the text used for locators in recaptcha_box.py.
This format would be fine:
TRANSLATIONS = {
    "im_not_a_robot": ["I'm not a robot", "로봇이 아닙니다"],
    "get_an_audio_challenge": ["Get an audio challenge", "음성 보안문자 듣기"],
    "get_a_new_challenge": ["Get a new challenge", "보안문자 새로 받기"],
}

Another possible concern is transcribing the audio challenge for other languages of reCAPTCHA. The reason for my concern is that the recognize_google() method of speech_recognition.Recognizer() requires a language code and defaults to en-US. With this being said, I'm not sure if it would be able to transcribe other languages without changing this code.

For Russian captchas, the audio check is also in English, so your script works fine on Russian captchas with translation.
Did I understand you correctly that you want to select a language by index? Something like that:
@property
def checkbox(self) -> Locator:
    """The reCAPTCHA checkbox locator."""
    return self.anchor_frame.get_by_role("checkbox", name=TRANSLATIONS['im_not_a_robot'][language_code])

Too bad playwright can't select the tag itself from the list, or am I missing something?

For Russian captchas, the audio check is also in English, so your script works fine on Russian captchas with translation.

Great, this will work fine then.

I'll probably end up doing something like this:
@property
def checkbox(self) -> Locator:
    """The reCAPTCHA checkbox locator."""
    return self.anchor_frame.get_by_role("checkbox", name=re.compile("|".join(TRANSLATIONS["im_not_a_robot"])))

This will generate a regex pattern that will match any of the translations for the text of that locator.

Do you have the Russian translations for the reCAPTCHA text?

Yeah, I have all the translations except:

"Multiple correct solutions required - please solve more."

I don't know how to call this case.
I know I'm a little slow on the PR, I'll try to get it done in the next 2 days ok? Too little time.
I want to add my first PR to my piggy bank :)

That text is shown when an incorrect answer is provided to the audio challenge.


Move PushUpdateOptions from models to repofiles

As title.

Codecov Report

Merging #9124 into master will increase coverage by <.01%.
The diff coverage is 75%.

@@            Coverage Diff             @@
##           master    #9124      +/-   ##
==========================================
+ Coverage   41.06%   41.06%   +<.01%
==========================================
  Files         553      553
  Lines       72041    72041
==========================================
+ Hits        29582    29584       +2
+ Misses      38727    38726       -1
+ Partials     3732     3731       -1

Impacted Files
Coverage Δ

models/update.go
19.04% <ø> (ø)
:arrow_up:

routers/private/push_update.go
0% <0%> (ø)
:arrow_up:

modules/repofiles/update.go
38.77% <100%> (ø)
:arrow_up:

routers/repo/branch.go
56.25% <100%> (ø)
:arrow_up:

routers/private/hook.go
50.24% <100%> (ø)
:arrow_up:

modules/task/migrate.go
25% <0%> (-3.95%)
:arrow_down:

models/error.go
32.67% <0%> (-1.19%)
:arrow_down:

models/repo.go
47.22% <0%> (+0.04%)
:arrow_up:

modules/migrations/gitea.go
9.45% <0%> (+0.63%)
:arrow_up:

modules/migrations/migrate.go
25.4% <0%> (+1.62%)
:arrow_up:

... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 54dab5a...375e198. Read the comment docs.


Feature Request: Mapping Yank Operation to Specific Register

I'm interested in a feature that sets a key mapping for yanking text directly into a chosen register.
Is there a method to provide a specific register as an option to <Plug>(YankyYank) or yanky.yank?
This would streamline the process of yanking text into a specific register, like the system register.
If this already exists, any guidance would be appreciated.

Hey!
You can just use : vim.keymap.set({"n","x"}, "y", '"5<Plug>(YankyYank)') where "5 is the register you want to yank to.

Thanks! This works great:
vim.keymap.set({ "n", "x" }, "Y", '"+<Plug>(YankyYank)')
One other question:
How would I set YY to do the same thing, but with an operator after.
I want to call YankyYank with the * clipboard, and then execute y (yank line) after.
Essentially I want to be able to have YY yank the current line to the global clipboard register.

Also doing this:
vim.keymap.set({ "n", "x" }, "Y", '"+<Plug>(YankyYank)')
sets both the unnamed register and the system clipboard. I was wondering if there was a way to only have it set the system register?

Ah yes, looks like I can do this:
    vim.keymap.set("n", "Y", function()
      return require("yanky").yank({ register = "+" })
    end, { expr = true })


Precompile Binaries

Can you please publish precompiled binaries for arm64. It is not so easy to install with m1.
gyp: No Xcode or CLT version detected!

@mkadirtan that's a great suggestion. Will add the precompiled binaries for arm64 soon.

@mkadirtan added the new assets to the release. Kindly close the issue if it solves your issue.

@mkadirtan added the new assets to the release. Kindly close the issue if it solves your issue.

Thank you!


Too many ways to set and read the branch.io keys (updated, was: Please remove hardcoding the _branchKey in BranchiOSWrapper.mm)

Hi,
handling the branch key(s) is not implemented straight forward in the branch.io Unity SDK.
I understand you wanted to make the Unity SDK as easy as possible for single Unity developers, but using the current SDK in a professional and fully automated build pipeline is very cumbersome.
The first main issue is: The key can be set by multiple methods and it's read from different sources:
Setting the key:

Serialized via Unity Editor into the BranchData asset
Replaced via the PostBuildScript directly within the BranchiOSWrapper (see below)
Manually or via PostBuildScript in the Info.plist (there it's also possible to set it in two different ways: as a dict for both test and live keys OR as a single string key only).

Reading the key:

Unity is reading the key in Branch.cs's Awake method

A better practice would be to hand over the key in the initSession method, in order to be able to define it during runtime. Important for automated build pipelines. The key can also be stored in the Branch.cs MonoBehaviour directly, which would allow to get rid of the BranchData.cs class completely (you don't really need it).

Also the BranchData class contains a static constructor, which is a bad practice and Unity 5.4 will throw compiler errors if you`re doing something like Resources.Load() or FindObject...() in it!

iOS / ObjC is reading the key from it's BranchiOSWrapper.mm's *_branchKey variable

iOS / ObjC is also reading it from the Info.plist

You see, that's pretty puzzling.
In my optimal world, I'd love to see branch.io integrating like this:
(C#/Unity pseudo code):
#if DEBUG
    Branch.initSession(Branch.TestKey, callback);
#else
    Branch.initSession(Branch.LiveKey, callback);
#endif

..where Branch.xxKey is serialized into the according script/scene and initSession is a fictitious method name.
This also allows for setting key values which come from a totally different origin (--> balancing data for games etc.)
Please remove hardcoding the _branchKey in BranchiOSWrapper.mm
It should not be necessary to hardcode the branch key in BranchiOSWrapper.mm like this:
https://github.com/BranchMetrics/Unity-Deferred-Deep-Linking-SDK/blob/1a74f0c696a7effc0701c033654dd53e405d29b1/BranchUnityTestBed/Assets/Branch/Editor/BranchEditor.cs#L50
This adds a lot of overhead when having an automated build pipeline - we would have to replace the key in the BranchiOSWrapper.mm everytime we want to switch between live and test et cetera.
Especially because the branch keys (both test and live) are already stored in the Info.plist (according to https://dev.branch.io/getting-started/integration-testing/guide/ios/#specifying-both-test-and-live-keys-in-your-app) - you should read them from there and get completely rid of the changes made regarding setting/updating the branch key from within the Unity editor in 1a74f0c696a7effc0701c033654dd53e405d29b1.

Hi derFunk,
thanks for the report!
We added hardcoded key to fix cold launch trouble, when key isn't initialised from Unity (sorry but we can't manage order of calls under Unity).
Initialisation of Unity Branch SDK is different than native iOS SDK because our plugin supports Android too, so we make Unity realisation without platform specific things (you don't need to keep keys in plist).
It will be very helpful if you will describe your auto-building and how we can change our Unity SDK to support your pipeline. If you can send us test project with auto-building then it will be great!
Nevertheless we will think how we can remove hardcoded key and I think we will change this in the next version of SDK.

Hi Anton,
thanks again for the fast response (wow, it's Sunday! :) )
Short roundup about our pipeline:
Our automated build pipeline basically follows the convention that we want to be able to configure every plugin setting manually and centrally.
That is because we're integrating more than 5 different plugins into almost every project/game, in average. We're totally avoiding PostBuildProcesses and stripping them out as soon as we see them, to be able to keep track what configuration and dependenvies every plugin needs. We also don't necessarily want to configure plugin keys in Editor Windows (but I understand this is comfortable for the average dev). Configuration in code is just fine (which does not necessarily mean that everything is hardcoded, instead it's in different datastores which get loaded at runtime).
We don't want the plugins to just "do magic" without us knowing what exactly is happening (as this can lead to hard to resolve problems with the builds, and we need the builds to be deterministic).
To be deterministic, even the Xcode projects are generated completely new for every build (no Unity "Xcode append" etc).
All plugin configuration values are coming from external data stores (so called "balancing data" for example). Normally we're having at least 2 different accounts for every plugin, which we want to be able to switch at build time (because we're working with partners a lot which often give us access to the plugin backends only late in the project phase, and we have to be able to test early, that's where we're using our own accounts).
We do this by having a "build parameter" (e.g. a preprocessor define) which just tells the code at build time which configuration to load (our keys or the ones from our partners/publishers).
That's why it's important to have a very central point to configure the plugin keys in code.
Having a plugin which loads it's configuration via "static constructor magic" or within the Awake methods of Unity are making our life harder, as at the same time we don't want to change the plugin's code, as it makes plugin updates a pain (checking the diffs.., documenting what we changed.. et cetera ;) ).
Hope this all makes sense to you ;)

derFunk,
We will think how to make our plugin better, but we need time.
Sorry, but right now you need to use "hard way" to set keys :)
(May be light post-processing will help you?)
Nevertheless I added this issue to dev plan, thanks!

Added to dev plan.


stuff from custom does not make it to default

Hi, this code isnt making it to default.
and I don't see the connection appearing in the integration.
It i do things wrong can you give a working example please. I'm a example guy.
and the manual is cutting it a bit short.
[core-ssh custom]$ ls
extentions.conf    pjsip_custom.conf
[core-ssh custom]$ cat extentions.conf
[default]
exten => _X.,1,Dial(PJSIP/sascha-trunk/${EXTEN})
[core-ssh custom]$ cat pjsip_custom.conf
[karin]
type=endpoint
context=from-sascha
disallow=all
allow=opus
allow=ulaw
auth=karin
aors=karin
direct_media = no
rtp_symmetric = yes
force_rport = yes
allow=h263p
allow=h264
allow=vp8

[karin]
type=auth
auth_type=userpass
password=ikbenje<3je
username=karin

[karin]
type=auth
auth_type=userpass
password=bedenkmaariets
username=karin

a that helps a lot.
the content is shown and if i run asterisk -r the connection is shown but unavailable.

Meaning the custom config file is being read by Asterisk correctly, right?

I think you can also use docker exec addon_b35499aa_asterisk asterisk -rx "pjsip show endopoints" to display list of devices you set in pjsip_custom.

Yeah that worked. figuring out the connection.
asterisk is a beast for me to handle.  But i have the best low budget voicemail system in the world


Thank You <3

Just wanted to say thanks to everyone who contributed to this amazing emoji library!
They're so fun and special ^_^ Can't wait to use them in my projects!!!
One thing I had thought of that may help in the future as our displays and devices become more 3D, if there's a way to share 3D models of the emoji. I understand this is a big ask as the original files were never intended to be fully 3D or in a format like .gltf, but it would be very cool to see these in 3D on the HoloLens (or maybe a Looking Glass display or Northstar headset 😉)
Very excited to be able to use these emoji, great work to all involved! They're stunning 🥰
Thank you for continuing to contribute to open source <3

@BryanChrisBrown Was in sync with you +/- 3 minutes with #14.

Thanks for the feedback @BryanChrisBrown - as noted in #14 - we are discussing this. If we did, we'd really want to set expectations as explained here the emoji aren't fully 3D, they are created in 3D space to look great as a flat object.


Add support for Block-based playgrounds

try this simple html/js vega-lite bar chart gist:
https://gist.github.com/domoritz/455e1c7872c4b38a58b90df0c3d7b1b9
Expected:
to see a rendered vega-lite bar chart web page preview similar to:
https://blockbuilder.org/domoritz/455e1c7872c4b38a58b90df0c3d7b1b9
with bar.vl.png image preview and bar.vl.json editor open.
Actual result: only index.html is open by gist pad with a blank web page preview.

btw, a block gist with just html and inline js does work.
try: https://gist.github.com/cjbassin/4b34001dcda4d20ce57b410a89096681

related: image view is broken.
should this displaly an image?

the good news is if you have Vega Viewer installed, it just works loading those custom .json graph files.
I am not sure how, but I am guessing it's b/c I have https vega specs loading in that ext.
so, I guess we can claim https://github.com/RandomFractals/vscode-vega-viewer as GistPad compatible :)  w00t!

I’d expect a single HTML file to work, but there’s a bit more work to support the broader class of Blocks out there.

true. I think it's close enough for now.

I just shipped an update that includes a bunch of improvements to make many (but not all!) Blocks work in GistPad. I'll keep churning through the issues, but things are much better now. Also, when you open a gist, you can paste a bl.ocks.org URL directly as opposed to needing to extract the username/id or id.

@lostintangent that is awesome! Will play with it tomorrow.
Thanks for going the extra mile on this!

btw, that last blocks update did take care of js loading and Vega blocks do load now ... thanks!

looks like simple d3.js with csv data loading blocks work too. nicely done!
You can close this ticket if you'd like.

Closing this since the bulk of Blocks support is shipped, and we can track any bugs in more specific issues. Thanks!


Use custom typescript compiler

I need to use ttsc instead of tsc in order to compile with some custom transformation.
There is a way to specify the compilation command?
https://github.com/cevek/ttypescript

@edobrb did you find a solution to this?

hi @tuminzee no unfortunately


netclient-userspace-docker workflow

according to the kitemaker ticket, testing cannot be done until merged into master. I will test when that happens


fix: adding iam_engine_enabled option

Description
Resolves: https://github.com/terraform-ibm-modules/stack-ibm-core-security-services/issues/59

Release required?

[x] No release
[ ] Patch release (x.x.X)
[ ] Minor release (x.X.x)
[ ] Major release (X.x.x)

Release notes content

Run the pipeline
If the CI pipeline doesn't run when you create the PR, the PR requires a user with GitHub collaborators access to run the pipeline.
Run the CI pipeline when the PR is ready for review and you expect tests to pass. Add a comment to the PR with the following text:
/run pipeline

Checklist for reviewers

[ ] If relevant, a test for the change is included or updated with this PR.
[ ] If relevant, documentation for the change is included or updated with this PR.

For mergers

Use a conventional commit message to set the release level. Follow the guidelines.
Include information that users need to know about the PR in the commit message. The commit message becomes part of the GitHub release notes.
Use the Squash and merge option.

/run pipeline

/run pipeline

/run pipeline

/run pipeline

/run pipeline

n we be consistent and

I see you updated all the icons in the diagram except the event notifications one - can we be consistent and update the one too?

Can you tell me which icon to use for event notification as I see the same icon being used here in the event-notification DA reference-architecture.

@iamar7 No idea - where did you get the other icons from? Maybe check with people who worked on the other DA diagrams

@iamar7 No idea - where did you get the other icons from? Maybe check with people who worked on the other DA diagrams

I've used the icons from ibm-cloud stencils repo here.

@iamar7 No idea - where did you get the other icons from? Maybe check with people who worked on the other DA diagrams

I asked about the icon in the IBM Cloud Stencils channel, and they confirmed that it is not available in the current library. I have created an issue for further tracking.

/run pipeline

/run pipeline

/run pipeline

/run pipeline

Blocked by known issue with the tests which is being debugged internally

/run pipeline

/run pipeline

/run pipeline

:tada: This issue has been resolved in version 1.3.0 :tada:
The release is available on:

GitHub release
v1.3.0

Your semantic-release bot :package::rocket:


Operand of null-aware operation '!' has type 'String' which excludes null.

New Issue Checklist

[x] I have searched for a similar issue in the project and found none

Issue Info

Info
Value

Platform Name
flutter web

Platform Version
Flutter Channel dev, 1.27.0-8.0.pre

Dio Version
4.0.0-beta2

Repro rate
always

Issue Description and Steps
My app needs to be compiled with proper null-safety. Therefore I am using dio version 4.0.0-beta2.
In this version a warning appears on startup of the app, when running on flutter for web:
[+7709 ms] /usr/local/flutter/.pub-cache/hosted/pub.dartlang.org/dio-4.0.0-beta2/lib/src/adapters/browser_adapter.dart:30:22: Warning: Operand of null-aware operation '!' has type 'String' which excludes null.
[        ]       ..open(options.method!, options.uri.toString(), async: true)

When the app tries to send a dio.request() the following error occurs:
[  +16 ms] Error: Bad state: No element
               at Object.throw_ [as throw] (http://localhost:61764/portal/dart_sdk.js:5342:11)
               at http://localhost:61764/portal/dart_sdk.js:19611:27
               at _RootZone.runGuarded (http://localhost:61764/portal/dart_sdk.js:38923:11)
               at _DoneStreamSubscription.new.[_sendDone] (http://localhost:61764/portal/dart_sdk.js:36181:47)
               at Object._microtaskLoop (http://localhost:61764/portal/dart_sdk.js:39301:13)
               at _startMicrotaskLoop (http://localhost:61764/portal/dart_sdk.js:39307:13)
               at http://localhost:61764/portal/dart_sdk.js:34814:9

The request is not sent to the server which makes this a serious bug.

It is fixed  in 4.0.0-beta3 (preview version)

Thanks a lot @wendux !

还是没有解决这个问题

[√] Flutter (Channel beta, 1.26.0-17.8.pre, on Microsoft Windows [Version 10.0.19042.804], locale zh-CN)
[√] Android toolchain - develop for Android devices (Android SDK version 30.0.2)
[√] Chrome - develop for the web
[√] Android Studio (version 4.1.0)
[√] VS Code (version 1.53.2)
[√] Connected device (4 available)

This is my solution, can run on both web and non-Web platforms.
Donn't Importing 'package:dio/adapter_browser.dart', so build is OK!
      if (Platform.I.isWeb) {
        _dio = Dio(BaseOptions(connectTimeout: 60000, sendTimeout: 60000, receiveTimeout: 60000));
        // (_dio.httpClientAdapter as BrowserHttpClientAdapter).withCredentials = true;
        (_dio.httpClientAdapter as dynamic).withCredentials = true;
      } else {
        _dio = Dio(BaseOptions(connectTimeout: 60000, sendTimeout: 60000, receiveTimeout: 60000));
        (_dio.httpClientAdapter as DefaultHttpClientAdapter).onHttpClientCreate = (client) {
          client.badCertificateCallback = (X509Certificate cert, String host, int port) {
            return true;
          };
        };
        _cookieJar = CookieJar();
        _dio.interceptors.add(CookieManager(_cookieJar));
      }
    }

In my project everything is correct, run on web and non-Web platforms.
_dio = Dio(BaseOptions(connectTimeout: 60000, sendTimeout: 60000, receiveTimeout: 60000))
will create correct httpClientAdapter for web platform and non-Web platforms.
(_dio.httpClientAdapter as dynamic)
will not need BrowserHttpClientAdapter, but can set withCredentials attribute value.


iOS iPhone 6 Icon Blurry

Looks like the iPhone 6 icon is blurry

It's because it's generating the wrong sizes for iphone.
If you check the build log in xcode it's got warnings.
warning: Ambiguous Content: AppIcon.appiconset/iphone-60x60-2x.png is 60x60 but should be 120x120.


Meaning of "min_len" parameter when constructing sparseSA?

Hi,
What is the effect of specifying a min_len value greater than zero for the sparseSA::create_auto method? And how does this relate to the min_len parameter for the sparseSA::find_MxM_each methods?
Is there any additional time cost to using a suffix array that has been constructed with min_len 0 if I expect to only query MEMs longer than 10, for example?
Thanks

7b571f88453733e1ed75bafb96c75ae5


Update Dockerfile

Running Docker-Compose fails without the -y flag:
After this operation, 395 kB of additional disk space will be used.
Do you want to continue? [Y/n] Abort.

Good catch, thanks @beevelop


eslint-comments/require-description is not suitable for generated code

Almost every automatically generated code/typings starts with eslint-disable:
/* eslint-disable */
/* prettier-ignore */
// @ts-nocheck
// noinspection JSUnusedGlobalSymbols
// Generated by unplugin-auto-import

That makes it unsuitable for eslint-comments/require-description.

ESLint is not suitable for generated code.


Does Chatterbot use Neural Networks? What other machine learning algorithms does it use other than search-based algorithms?

In addition to search based algorithms, ChatterBot also uses Bayesian networks for various text classification tasks.


Add padding support to TrackLocalStaticSample

To add padding-only samples call GeneratePadding
Description
v4 version of https://github.com/pion/webrtc/pull/2745

Fantastic LGTM @alexpokotilo
Can you add a E2E test that just shows sending padding (and getting it as expected) then it is good to merge :)
I will make sure to do /v3 also

Fantastic LGTM @alexpokotilo
Can you add a E2E test that just shows sending padding (and getting it as expected) then it is good to merge :)
I will make sure to do /v3 also
You are right ! Added test. Please propagate this fix into v3

Anything you need on /v3 I will make sure to propagate @alexpokotilo !


Enable azure-identity based authentication for azure batch BatchServiceClient

Is your feature request related to a problem? Please describe.
I want to move away from using service principal secret authentication to either managed identities or certificate authentication to create azure.batch.BatchServiceClient.
Describe the solution you'd like
I want to be able to use azure-identity APIs for authentication. DefaultCredential is super convenient for development.
Describe alternatives you've considered
When using DefaultCredential I get an error related to no method signed_session so i've tried using the credential wrapper others have used for management APIs but it doesn't work either. Using the wrapper, i get a connection error "Failed to establish new connection"
Additional context
I've tried this with azure-batch version 14.2.0 and azure-identity version 1.15.0.

#30468

thanks @xiangyan99  i looked at that issue but didn't see when this might be supported. The workarounds provided by others do not work for me.

FYI this work is in progress (see this PR). We're hoping to have a beta version of an updated SDK which is compatible with the azure-identity objects (and based on the modern SDK stack) in June.


v5.0.0 marked as still in progress

[x] The issue is present in the latest release.
[x] I have searched the issues of this repository and believe that this is not a duplicate.

Current Behavior 😯

In the docs, in release schedule, v5.0.0  and v5.0.beta.0 are listed as still in progress.

Expected Behavior 🤔

Now that v5 has been released, we might want to remove v5.0.beta.0 and mark v5.0.0 as released.

Steps to Reproduce 🕹

Steps:

Navigate to Release Schedule Section
Notice, that v5.0.beta.0 and v5.0.0 are still in progress, while v5.0 has been released.

Context 🔦

N/A

Your Environment 🌎

N/A

if this is an issue, and it would be a good first issue, I would like to work on it.

@owais635 nice catch! please submit a PR and you can tag me as a reviewer.

Thanks,
I have a few questions.

I was thinking of updating the table to be the following. Is this ok?

| Date                | Version     | Status           |
| :------------------ | :---------- | :--------------- |
| May 2018            | v1.0.0      | Released         |
| September 2018      | v3.0.0      | Released         |
| May 2019            | v4.0.0      | Released         |
| September 2021      | v5.0.0      | Released         |

I have noticed we have some translated files, do I also have to make the change there? Or are they automatically generated?

@owais635

I was thinking of updating the table to be the following. Is this ok?

for v5.0.0 the release date is September 16th, 2021

I have noticed we have some translated files, do I also have to make the change in there? Or are they automatically generated?

only versions.md, other languages will be auto generated. cc @oliviertassinari

Should we start to think about a release timeline for v6? It seems that we start to have a few opportunities:

Drop IE 11 entirely
Rework theme structure to use CSS variables
Update Material Design (we didn't work on it much in v5)
Move material design to use the @mui/material-next
etc.

I kept track of this into Trello, but it feels like moving everything to GitHub issues would work best for collaboration. We could open a v6 issue to collect candidates for BCs. The incentive being to balance between not fragmenting the community too much, while keep iterating. Given the depth of the breaking changes that we did in v5, maybe we can target for v6 in 12+ months, with an easy migration path.

Here is the PR: https://github.com/mui-org/material-ui/pull/28700

@oliviertassinari I like this library and thank you and your team for making it. I was wondering if Material You is planned for v6?

I kept track of potential opportunities in Trello, but it feels like moving everything to GitHub issues would work best for collaboration. We could open a v6 issue to collect candidates for BCs. The incentive being to balance between not fragmenting the community too much, while not preventing innovations (BCs are not innovations, they are pain for developers). Given the depth of the breaking changes that we did in v5, maybe we can target for v6 in 12+ months, with an easy migration path.

👍 we already have the milestone for v6 https://github.com/mui-org/material-ui/milestone/42


DNS challenge with cloudflare issue

I am using your docker-compose file with latest letsencrypt-dns and cloudflare as dns provider.
When starting the docker container it get the following output:
letsencrypt-dns | 2020-03-09 16:18:55 circus[1] [INFO] Starting master on pid 1
letsencrypt-dns | 2020-03-09 16:18:55 circus[1] [INFO] Arbiter now waiting for commands
letsencrypt-dns | 2020-03-09 16:18:55 circus[1] [INFO] crond started
letsencrypt-dns | 2020-03-09 16:18:55 circus[1] [INFO] watch-domains started
letsencrypt-dns | 2020-03-09 16:18:55 [17] | #### Registering Let's Encrypt account if needed ####
letsencrypt-dns | 2020-03-09 16:18:55 [17] | Saving debug log to /etc/letsencrypt/logs/letsencrypt.log
letsencrypt-dns | 2020-03-09 16:18:55 [17] | There is an existing account; registration of a duplicate account with this command is currently unsupported.
letsencrypt-dns | 2020-03-09 16:18:55 [17] | #### Clean autorestart/autocmd jobs
letsencrypt-dns | 2020-03-09 16:18:55 [17] | #### Creating missing certificates if needed (~1min for each) ####
letsencrypt-dns | 2020-03-09 16:18:55 [17] | >>> Creating a certificate for domain(s): -d nextcloud.mydomain.de
letsencrypt-dns | 2020-03-09 16:18:56 [17] | Saving debug log to /etc/letsencrypt/logs/letsencrypt.log
letsencrypt-dns | 2020-03-09 16:18:56 [17] | Plugins selected: Authenticator manual, Installer None
letsencrypt-dns | 2020-03-09 16:18:57 [17] | Obtaining a new certificate
letsencrypt-dns | 2020-03-09 16:18:57 [17] | Performing the following challenges:
letsencrypt-dns | 2020-03-09 16:18:57 [17] | dns-01 challenge for nextcloud.mydomain.de
letsencrypt-dns | 2020-03-09 16:18:57 [17] | Running manual-auth-hook command: /var/lib/letsencrypt/hooks/authenticator.sh
letsencrypt-dns | 2020-03-09 16:18:59 [17] | manual-auth-hook command "/var/lib/letsencrypt/hooks/authenticator.sh" returned error code 1
letsencrypt-dns | 2020-03-09 16:18:59 [17] | Error output from manual-auth-hook command authenticator.sh:
letsencrypt-dns | 2020-03-09 16:18:59 [17] | Traceback (most recent call last):
letsencrypt-dns | 2020-03-09 16:18:59 [17] |   File "/usr/local/bin/lexicon", line 8, in <module>
letsencrypt-dns | 2020-03-09 16:18:59 [17] |     sys.exit(main())
letsencrypt-dns | 2020-03-09 16:18:59 [17] |   File "/usr/local/lib/python3.8/site-packages/lexicon/cli.py", line 117, in main
letsencrypt-dns | 2020-03-09 16:18:59 [17] |     results = client.execute()
letsencrypt-dns | 2020-03-09 16:18:59 [17] |   File "/usr/local/lib/python3.8/site-packages/lexicon/client.py", line 77, in execute
letsencrypt-dns | 2020-03-09 16:18:59 [17] |     self.provider.authenticate()
letsencrypt-dns | 2020-03-09 16:18:59 [17] |   File "/usr/local/lib/python3.8/site-packages/lexicon/providers/base.py", line 69, in authenticate
letsencrypt-dns | 2020-03-09 16:18:59 [17] |     return self._authenticate()
letsencrypt-dns | 2020-03-09 16:18:59 [17] |   File "/usr/local/lib/python3.8/site-packages/lexicon/providers/cloudflare.py", line 32, in _authenticate
letsencrypt-dns | 2020-03-09 16:18:59 [17] |     payload = self._get('/zones', {
letsencrypt-dns | 2020-03-09 16:18:59 [17] |   File "/usr/local/lib/python3.8/site-packages/lexicon/providers/base.py", line 142, in _get
letsencrypt-dns | 2020-03-09 16:18:59 [17] |     return self._request('GET', url, query_params=query_params)
letsencrypt-dns | 2020-03-09 16:18:59 [17] |   File "/usr/local/lib/python3.8/site-packages/lexicon/providers/cloudflare.py", line 146, in _request
letsencrypt-dns | 2020-03-09 16:18:59 [17] |     response.raise_for_status()
letsencrypt-dns | 2020-03-09 16:18:59 [17] |   File "/usr/local/lib/python3.8/site-packages/request
letsencrypt-dns | 2020-03-09 16:18:59 [17] | s/models.py", line 941, in raise_for_status
letsencrypt-dns | 2020-03-09 16:18:59 [17] |     raise HTTPError(http_error_msg, response=self)
letsencrypt-dns | 2020-03-09 16:18:59 [17] | requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://api.cloudflare.com/client/v4/zones?name=mydomain.de&status=active
letsencrypt-dns | 2020-03-09 16:18:59 [17] | Waiting for verification...
letsencrypt-dns | 2020-03-09 16:19:01 [17] | Challenge failed for domain nextcloud.mydomain.de
letsencrypt-dns | 2020-03-09 16:19:01 [17] | dns-01 challenge for nextcloud.mydomain.de
letsencrypt-dns | 2020-03-09 16:19:01 [17] | Cleaning up challenges

I am a little bit lost as this is the first time i use your docker container and also the first time i use cloudflare. I was creating the dns zone at cloudflare very minimalistic (just a A-record for main domain, but not an A-record for nextcloud.mydomain.de, as it should not be required when doing dns challenges). As far as i understand it is not even necessary that cloudflare is my active nameserver (at least not in this phase of the process: when trying to seed the TXT record) - at validation of dns challenge it is required to have cloudflare being the active nameserver for my domain.
At cloudflare this token has all possilble permissions (account + zone) across all resources (accounts and zones), so i think it is not a permission problem.
I made cloudflare nameservers active for my domain (confirmed as active by cloudflare webUI) for a second run but still get the same issue (do i have to wait 24h to be sure, that letsencrypt CA is picking the right nameservers?).
Please help.

OMG!
I filled the env variable LEXICON_PROVIDER_OPTIONS with<EMAIL_ADDRESS>--auth-token=cloudflare_api_token, but instead of a cloudflare API Token i should use the cloudflare API Key for --auth-token - naming is a bit confusing here...
Now it works. Maybe we should improve the documentation with more examples (at least for all DNS providers with confusing naming).

A user-friendly error message instead of a stacktrace would be cool.

Yes, I agree that the error is absolutely not obvious here. For cloudflare specifically pathological here, since you have global API keys, and more regular scoped API tokens. Here currently only global API keys are available, and an upstream issue is opened to get the scoped API tokens (https://github.com/AnalogJ/lexicon/pull/460).
Getting a nice error from my Docker is quite difficult however, since Lexicon is not sending an fine-grain error about what was wrong. I think it should be quite useful to raise an issue on the Lexicon GitHub project (and continue there, since I also one of the maintainers for this project).
I close the issue here, since the error is "technically" solved, but do not hesitate to give further feedbacks!

Hello guys, I try to use the Globar API Key as @daniela-waranie suggested but it didn't work for me either. I've been using the docker image for this but I don't know if there is any difference between native and docker run.
API Token log
dnsrobocert    | Saving debug log to /etc/letsencrypt/logs/letsencrypt.log
dnsrobocert    | Plugins selected: Authenticator manual, Installer None
dnsrobocert    | Obtaining a new certificate
dnsrobocert    | Performing the following challenges:
dnsrobocert    | dns-01 challenge for api.mydomain.com
dnsrobocert    | Running manual-auth-hook command: /usr/bin/python3 -m dnsrobocert.core.hooks -t auth -c "/tmp/tmprcwq5zn8/dnsrobocert-runtime.yml" -l "api.mydomain.com"
dnsrobocert    | manual-auth-hook command "/usr/bin/python3 -m dnsrobocert.core.hooks -t auth -c "/tmp/tmprcwq5zn8/dnsrobocert-runtime.yml" -l "api.mydomain.com"" returned error code 1
dnsrobocert    | Error output from manual-auth-hook command python3:
dnsrobocert    | 2020-04-19 23:23:51 0f6824afd7a9 __main__[73] ERROR Error while executing the `auth` hook:
dnsrobocert    | 2020-04-19 23:23:51 0f6824afd7a9 __main__[73] ERROR 400 Client Error: Bad Request for url: https://api.cloudflare.com/client/v4/zones?name=mydomain.com&status=active
dnsrobocert    | Traceback (most recent call last):
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/dnsrobocert/core/hooks.py", line 48, in main
dnsrobocert    |     globals()[parsed_args.type](dnsrobocert_config, parsed_args.lineage)
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/dnsrobocert/core/hooks.py", line 63, in auth
dnsrobocert    |     _txt_challenge(profile, token, domain, action="create")
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/dnsrobocert/core/hooks.py", line 174, in _txt_challenge
dnsrobocert    |     Client(lexicon_config).execute()
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/lexicon/client.py", line 77, in execute
dnsrobocert    |     self.provider.authenticate()
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/lexicon/providers/base.py", line 69, in authenticate
dnsrobocert    |     return self._authenticate()
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/lexicon/providers/cloudflare.py", line 32, in _authenticate
dnsrobocert    |     payload = self._get('/zones', {
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/lexicon/providers/base.py", line 142, in _get
dnsrobocert    |     return self._request('GET', url, query_params=query_params)
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/lexicon/providers/cloudflare.py", line 146, in _request
dnsrobocert    |     response.raise_for_status()
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/requests/models.py", line 941, in raise_for_status
dnsrobocert    |     raise HTTPError(http_error_msg, response=self)
dnsrobocert    | requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://api.cloudflare.com/client/v4/zones?name=mydomain.com&status=active
dnsrobocert    |
dnsrobocert    | Waiting for verification...
dnsrobocert    | Challenge failed for domain api.mydomain.com
dnsrobocert    | dns-01 challenge for api.mydomain.com
dnsrobocert    | Cleaning up challenges
dnsrobocert    | Running manual-cleanup-hook command: /usr/bin/python3 -m dnsrobocert.core.hooks -t cleanup -c "/tmp/tmprcwq5zn8/dnsrobocert-runtime.yml" -l "api.mydomain.com"
dnsrobocert    | manual-cleanup-hook command "/usr/bin/python3 -m dnsrobocert.core.hooks -t cleanup -c "/tmp/tmprcwq5zn8/dnsrobocert-runtime.yml" -l "api.mydomain.com"" returned error code 1
dnsrobocert    | Error output from manual-cleanup-hook command python3:
dnsrobocert    | 2020-04-19 23:23:56 0f6824afd7a9 __main__[75] ERROR Error while executing the `cleanup` hook:
dnsrobocert    | 2020-04-19 23:23:56 0f6824afd7a9 __main__[75] ERROR 400 Client Error: Bad Request for url: https://api.cloudflare.com/client/v4/zones?name=mydomain.com&status=active
dnsrobocert    | Traceback (most recent call last):
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/dnsrobocert/core/hooks.py", line 48, in main
dnsrobocert    |     globals()[parsed_args.type](dnsrobocert_config, parsed_args.lineage)
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/dnsrobocert/core/hooks.py", line 125, in cleanup
dnsrobocert    |     _txt_challenge(profile, token, domain, action="delete")
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/dnsrobocert/core/hooks.py", line 174, in _txt_challenge
dnsrobocert    |     Client(lexicon_config).execute()
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/lexicon/client.py", line 77, in execute
dnsrobocert    |     self.provider.authenticate()
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/lexicon/providers/base.py", line 69, in authenticate
dnsrobocert    |     return self._authenticate()
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/lexicon/providers/cloudflare.py", line 32, in _authenticate
dnsrobocert    |     payload = self._get('/zones', {
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/lexicon/providers/base.py", line 142, in _get
dnsrobocert    |     return self._request('GET', url, query_params=query_params)
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/lexicon/providers/cloudflare.py", line 146, in _request
dnsrobocert    |     response.raise_for_status()
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/requests/models.py", line 941, in raise_for_status
dnsrobocert    |     raise HTTPError(http_error_msg, response=self)
dnsrobocert    | requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://api.cloudflare.com/client/v4/zones?name=mydomain.com&status=active
dnsrobocert    |
dnsrobocert    | Some challenges have failed.
dnsrobocert    | IMPORTANT NOTES:
dnsrobocert    |  - The following errors were reported by the server:
dnsrobocert    |
dnsrobocert    |    Domain: api.mydomain.com
dnsrobocert    |    Type:   dns
dnsrobocert    |    Detail: DNS problem: NXDOMAIN looking up TXT for
dnsrobocert    |    _acme-challenge.api.mydomain.com - check that a DNS record
dnsrobocert    |    exists for this domain
dnsrobocert    | ----------
dnsrobocert    | 2020-04-19 23:23:56 0f6824afd7a9 dnsrobocert.core.main[1] ERROR An error occurred while processing certificate config `{'domains': ['api.mydomain.com'], 'profile': 'maestro_profile'}`:
dnsrobocert    | Command '['/usr/bin/python3', '-m', 'dnsrobocert.core.certbot', 'certonly', '-n', '--config-dir', '/etc/letsencrypt', '--work-dir', '/etc/letsencrypt/workdir', '--logs-dir', '/etc/letsencrypt/logs', '--manual', '--preferred-challenges=dns', '--manual-auth-hook', '/usr/bin/python3 -m dnsrobocert.core.hooks -t auth -c "/tmp/tmprcwq5zn8/dnsrobocert-runtime.yml" -l "api.mydomain.com"', '--manual-cleanup-hook', '/usr/bin/python3 -m dnsrobocert.core.hooks -t cleanup -c "/tmp/tmprcwq5zn8/dnsrobocert-runtime.yml" -l "api.mydomain.com"', '--manual-public-ip-logging-ok', '--expand', '--deploy-hook', '/usr/bin/python3 -m dnsrobocert.core.hooks -t deploy -c "/tmp/tmprcwq5zn8/dnsrobocert-runtime.yml" -l "api.mydomain.com"', '--server', 'https://acme-staging-v02.api.letsencrypt.org/directory', '--cert-name', 'api.mydomain.com', '-d', 'api.mydomain.com']' returned non-zero exit status 1.
dnsrobocert    | 2020-04-19 23:23:56 0f6824afd7a9 dnsrobocert.core.main[1] INFO Revoke and delete certificates if needed

Global API Key log
dnsrobocert    | Saving debug log to /etc/letsencrypt/logs/letsencrypt.log
dnsrobocert    | Plugins selected: Authenticator manual, Installer None
dnsrobocert    | Obtaining a new certificate
dnsrobocert    | Performing the following challenges:
dnsrobocert    | dns-01 challenge for api.mydomain.com
dnsrobocert    | Running manual-auth-hook command: /usr/bin/python3 -m dnsrobocert.core.hooks -t auth -c "/tmp/tmprcwq5zn8/dnsrobocert-runtime.yml" -l "api.mydomain.com"
dnsrobocert    | manual-auth-hook command "/usr/bin/python3 -m dnsrobocert.core.hooks -t auth -c "/tmp/tmprcwq5zn8/dnsrobocert-runtime.yml" -l "api.mydomain.com"" returned error code 1
dnsrobocert    | Error output from manual-auth-hook command python3:
dnsrobocert    | 2020-04-19 23:30:44 0f6824afd7a9 __main__[81] ERROR Error while executing the `auth` hook:
dnsrobocert    | 2020-04-19 23:30:44 0f6824afd7a9 __main__[81] ERROR 403 Client Error: Forbidden for url: https://api.cloudflare.com/client/v4/zones?name=mydomain.com&status=active
dnsrobocert    | Traceback (most recent call last):
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/dnsrobocert/core/hooks.py", line 48, in main
dnsrobocert    |     globals()[parsed_args.type](dnsrobocert_config, parsed_args.lineage)
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/dnsrobocert/core/hooks.py", line 63, in auth
dnsrobocert    |     _txt_challenge(profile, token, domain, action="create")
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/dnsrobocert/core/hooks.py", line 174, in _txt_challenge
dnsrobocert    |     Client(lexicon_config).execute()
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/lexicon/client.py", line 77, in execute
dnsrobocert    |     self.provider.authenticate()
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/lexicon/providers/base.py", line 69, in authenticate
dnsrobocert    |     return self._authenticate()
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/lexicon/providers/cloudflare.py", line 32, in _authenticate
dnsrobocert    |     payload = self._get('/zones', {
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/lexicon/providers/base.py", line 142, in _get
dnsrobocert    |     return self._request('GET', url, query_params=query_params)
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/lexicon/providers/cloudflare.py", line 146, in _request
dnsrobocert    |     response.raise_for_status()
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/requests/models.py", line 941, in raise_for_status
dnsrobocert    |     raise HTTPError(http_error_msg, response=self)
dnsrobocert    | requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://api.cloudflare.com/client/v4/zones?name=mydomain.com&status=active
dnsrobocert    |
dnsrobocert    | Waiting for verification...
dnsrobocert    | Challenge failed for domain api.mydomain.com
dnsrobocert    | dns-01 challenge for api.mydomain.com
dnsrobocert    | Cleaning up challenges
dnsrobocert    | Running manual-cleanup-hook command: /usr/bin/python3 -m dnsrobocert.core.hooks -t cleanup -c "/tmp/tmprcwq5zn8/dnsrobocert-runtime.yml" -l "api.mydomain.com"
dnsrobocert    | manual-cleanup-hook command "/usr/bin/python3 -m dnsrobocert.core.hooks -t cleanup -c "/tmp/tmprcwq5zn8/dnsrobocert-runtime.yml" -l "api.mydomain.com"" returned error code 1
dnsrobocert    | Error output from manual-cleanup-hook command python3:
dnsrobocert    | 2020-04-19 23:30:50 0f6824afd7a9 __main__[83] ERROR Error while executing the `cleanup` hook:
dnsrobocert    | 2020-04-19 23:30:50 0f6824afd7a9 __main__[83] ERROR 403 Client Error: Forbidden for url: https://api.cloudflare.com/client/v4/zones?name=mydomain.com&status=active
dnsrobocert    | Traceback (most recent call last):
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/dnsrobocert/core/hooks.py", line 48, in main
dnsrobocert    |     globals()[parsed_args.type](dnsrobocert_config, parsed_args.lineage)
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/dnsrobocert/core/hooks.py", line 125, in cleanup
dnsrobocert    |     _txt_challenge(profile, token, domain, action="delete")
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/dnsrobocert/core/hooks.py", line 174, in _txt_challenge
dnsrobocert    |     Client(lexicon_config).execute()
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/lexicon/client.py", line 77, in execute
dnsrobocert    |     self.provider.authenticate()
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/lexicon/providers/base.py", line 69, in authenticate
dnsrobocert    |     return self._authenticate()
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/lexicon/providers/cloudflare.py", line 32, in _authenticate
dnsrobocert    |     payload = self._get('/zones', {
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/lexicon/providers/base.py", line 142, in _get
dnsrobocert    |     return self._request('GET', url, query_params=query_params)
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/lexicon/providers/cloudflare.py", line 146, in _request
dnsrobocert    |     response.raise_for_status()
dnsrobocert    |   File "/usr/lib/python3.8/site-packages/requests/models.py", line 941, in raise_for_status
dnsrobocert    |     raise HTTPError(http_error_msg, response=self)
dnsrobocert    | requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://api.cloudflare.com/client/v4/zones?name=mydomain.com&status=active
dnsrobocert    |
dnsrobocert    | Some challenges have failed.
dnsrobocert    | IMPORTANT NOTES:
dnsrobocert    |  - The following errors were reported by the server:
dnsrobocert    |
dnsrobocert    |    Domain: api.mydomain.com
dnsrobocert    |    Type:   dns
dnsrobocert    |    Detail: DNS problem: NXDOMAIN looking up TXT for
dnsrobocert    |    _acme-challenge.api.mydomain.com - check that a DNS record
dnsrobocert    |    exists for this domain
dnsrobocert    | ----------
dnsrobocert    | 2020-04-19 23:30:50 0f6824afd7a9 dnsrobocert.core.main[1] ERROR An error occurred while processing certificate config `{'domains': ['api.mydomain.com'], 'profile': 'maestro_profile'}`:
dnsrobocert    | Command '['/usr/bin/python3', '-m', 'dnsrobocert.core.certbot', 'certonly', '-n', '--config-dir', '/etc/letsencrypt', '--work-dir', '/etc/letsencrypt/workdir', '--logs-dir', '/etc/letsencrypt/logs', '--manual', '--preferred-challenges=dns', '--manual-auth-hook', '/usr/bin/python3 -m dnsrobocert.core.hooks -t auth -c "/tmp/tmprcwq5zn8/dnsrobocert-runtime.yml" -l "api.mydomain.com"', '--manual-cleanup-hook', '/usr/bin/python3 -m dnsrobocert.core.hooks -t cleanup -c "/tmp/tmprcwq5zn8/dnsrobocert-runtime.yml" -l "api.mydomain.com"', '--manual-public-ip-logging-ok', '--expand', '--deploy-hook', '/usr/bin/python3 -m dnsrobocert.core.hooks -t deploy -c "/tmp/tmprcwq5zn8/dnsrobocert-runtime.yml" -l "api.mydomain.com"', '--server', 'https://acme-staging-v02.api.letsencrypt.org/directory', '--cert-name', 'api.mydomain.com', '-d', 'api.mydomain.com']' returned non-zero exit status 1.
dnsrobocert    | 2020-04-19 23:30:50 0f6824afd7a9 dnsrobocert.core.main[1] INFO Revoke and delete certificates if needed

As you may notice, errors are like follow:

API Token: 400 Client Error: Bad Request for url
Global API KEY: 403 Client Error: Forbidden for url

I'm setting the variables directly in the config.yml as suggested in the documentation
draft: false
acme:
  email_account<EMAIL_ADDRESS>  staging: true
profiles:
  - name: maestro_profile
    provider: cloudflare
    provider_options:
      auth_username<EMAIL_ADDRESS>      auth_token: <global_api_key|api_token>
certificates:
  - domains:
      - api.mydomain.com
    profile: maestro_profile

Do you have any suggestions I could follow? I already checked the issues on the other repositories but they are not merged yet.


use weldx-widgets k3d implementation if available

Changes

optionally use code from weldx-widgets if installed

Related Issues
Closes # (add issue numbers)
Checks

[ ] updated CHANGELOG.rst
[ ] updated tests
[ ] updated doc/
[ ] update example/tutorial notebooks
[ ] update manifest file

I think we should also start to remove the matplotlib and k3d dependencies and add import-guards in the respective visualization modules. These guards should show an error message indicating that these dependencies have moved to weldx-widgets, or can be installed manually.

I think we should also start to remove the matplotlib and k3d dependencies and add import-guards in the respective visualization modules. These guards should show an error message indicating that these dependencies have moved to weldx-widgets, or can be installed manually.

eventually yes, let's do this a little slower with the above approach but keep the default weldx behavior (and dependencies) stable for now

The approach I suggested would result in a pre-tested release with the conditions we would like to achieve. Otherwise I wonder how the road map for this change would look like? At some point we will do it any ways, right?


ytt formatting converts multi line strings to single line strings due to trailing spaces

What steps did you take:
I am running ytt on a config map with a multi line string and a trailing space after the colon in line 7, e.g.:
  1 apiVersion: v1
  2 kind: ConfigMap
  3 metadata:
  4   name: my-config.map
  5 data:
  6   config: |
  7     metricsConfig:
  8       enabled: true
  9       path: /metrics
 10       port: 9090
 11       ignoreErrors: false
 12       secure: false

What happened:
Ytt unexpectedly converts the multi-line string to a single-line string:
apiVersion: v1
kind: ConfigMap
metadata:
  name: my-config.map
data:
  config: "metricsConfig: \n  enabled: true\n  path: /metrics\n  port: 9090\n  ignoreErrors: false\n  secure: false\n"

What did you expect:
The multi-line string should be kept. The trailing white space could be removed.
Environment:

ytt version (use ytt --version): 0.44.0
OS (e.g. from /etc/os-release): Ubuntu 20.04.4 LTS

Vote on this request
This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.
👍 "I would like to see this addressed as soon as possible"
👎 "There are other more important things to focus on right now"
We are also happy to receive and review Pull Requests if you want to help working on this issue.

I don't have any errant trailing spaces in my multi-line strings but recently also saw that ytt is now collapsing everything into a very long single line string, so it would be great to get this behaviour reverted or configurable.

I'm able to successfully reproduce this issue. Will run our thoughts on this internally and set the priority accordingly.
Meanwhile, if you are willing to contribute we will be happy to review it on priority.

Also, could you please let us know if this is a blocking issue for your use case or you found any workaround?

My current workaround is to run sed -r " s#:\s+$#:#g" on my yamls before handing them to ytt.

@sethiyash Please have a look at my PR to fix the issue

@fritzduchardt Thank you for reaching out with this issue and we appreciate the work you have done to fix it.
However, this seems to be an issue present in yaml.v2 library which is used in ytt and also we verified that this issue is present in yaml.v3 library. It would be nice if we can raise this PR in API v2 of the yaml package for Go.