[nextra] i18n - add middleware to handle unhandled languages

Description
This adds '/' -> '/en' and '/fr' etc. -> '/en' for now.
I wanted to change all of '/en' to just '/', but there's some hydration issue when using a rewrite.
Checklist

Do all Lints pass?

[x] Have you ran pnpm spellcheck?
[x] Have you ran pnpm fmt?
[x] Have you ran pnpm lint?

A bit worried about overwriting the Nextra routing but down to try and see how it does

Yeah, it didn't seem to fail in my testing, but we can always try it and remove it later.


fix: make bash version detection locale independ

The BASH_VERSION variable holds the bash version number in a much
better consumable and especially locale independent format than the
--version flag.
This fixes #1940
Checklist:

[ ] Add tests that demonstrate the correct behavior of the change. Tests should fail without the change.
[ ] Add or update relevant docs, in the docs folder and in code.
[ ] Add an entry in CHANGES.rst summarizing the change and linking to the issue.
[ ] Add .. versionchanged:: entries in any relevant code docs.
[ ] Run pre-commit hooks and fix any issues.
[ ] Run pytest and tox, no tests failed.

Please rebase to 8.0.x and add a changelog entry.

Please rebase to 8.0.x and add a changelog entry.

Done.

I would change it this for now:
        output = subprocess.run(["bash", "--version"], stdout=subprocess.PIPE)
        match = re.search(r"(\d+)\.(\d+)\.\d+", output.stdout.decode())

This solves the locale dependent spelling of version ans also adds support for multi-digit versions (e.g. 5.12.23).  Since this still uses bash --version, it won’t introduce new issues.
So while this may not be the perfect solution, it will certainly improve the current state. :)

Why not use the version variable. This would completely avoid future surprises that always happen when one tries to parse data out of human readable descriptions one does not control.
The variable is at least clearly defined:

BASH_VERSION
The version number of the current instance of Bash.

(From  https://www.gnu.org/software/bash/manual/html_node/Bash-Variables.html)


Numba sampler list to tuple.

Thank you for putting so much effort into this Yaoyu!
The numba sampler threw an error for me, complaining about k_sample_coor[[1024,1,1],[256,1,1]]( d_output, d_out_offsets, d_xyz, d_offsets ) that
TypeError: unhashable type: 'list'

Changing the inner lists to a tuple seemed to have done the trick for me. Could you please try it out to make sure it does not break anything else?

Great. I have tested it on my machine and it also works. Merging and closing the PR now.

And for merged PRs, I think we can directly remove the branch after merging. Already removed the fix/numba-sampler-list-to-tuple branch.

Perfect! Yes I agree -- removing branches is a good practice in my opinion as well


1000 SELECT queries takes lot of time. Bad performance.

I have a page where it is having atleast 1000 select queries. It takes 1-2mins of time load fetch data. even if i need to push 1 Lakh record in 20 tables. It takes 2mins of time to do so. I am using 500 insert in a single hit to sqlite. Can we make performance batter using the following concept?
ConfigureAwait(false)
Please check this link if helpful: http://blogs.msdn.com/b/andy_wigley/archive/2013/11/21/how-to-massively-improve-sqlite-performance-using-sqlwinrt.aspx

1000 selects should take only msecs. But without knowing the structure of your db and potential cross table queries happening in your select statement its impossible to guess.
For bulk inserts use transactions.


If you have a deleted cask installed, you cannot do anything

brew gist-logs <formula> link OR brew config AND brew doctor output
config
HOMEBREW_VERSION: 3.0.11-152-g6359af5
ORIGIN: https://github.com/Homebrew/brew.git
HEAD: 6359af5cca90fdb732f379a11660a21001554768
Last commit: 13 hours ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 70939b7ae8d953162fc4014e1ec4ecd456b1c86b
Core tap last commit: 6 minutes ago
Core tap branch: master
HOMEBREW_PREFIX: /usr/local
HOMEBREW_CASK_OPTS: []
HOMEBREW_CLEANUP_MAX_AGE_DAYS: 0
HOMEBREW_DISPLAY: /private/tmp/com.apple.launchd.u3rjONq4we/org.xquartz:0
HOMEBREW_EDITOR: /usr/local/bin/emacs
HOMEBREW_FORCE_BREWED_CURL: set
HOMEBREW_GITHUB_API_TOKEN: set
HOMEBREW_MAKE_JOBS: 4
Homebrew Ruby: 2.6.3 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
CPU: quad-core 64-bit haswell
Clang: 12.0 build 1200
Git: 2.31.1 => /usr/local/bin/git
Curl: 7.76.0 => /usr/local/opt/curl/bin/curl
macOS: 11.2.3-x86_64
CLT: <IP_ADDRESS>.1.1610135815
Xcode: 12.4

brew doctor
Error: Cask 'flash-ppapi' is unavailable: No Cask with this name exists.

The brew doctor above contains no "Warning" lines.
What were you trying to do (and why)?
brew upgrade (initially, see below)
What happened (include all command output)?
I tried
brew upgrade
and i saw that flash-ppapi was now a deleted cask. I had installed it in the past, and still have it installed.
The output of brew upgrade was
Error: Cask 'flash-ppapi' is unavailable: No Cask with this name exists.
Doing
brew uninstall flash-ppapi
also gives
Error: No available formula or cask with the name "flash-ppapi".
And
brew doctor
also gives
Error: Cask 'flash-ppapi' is unavailable: No Cask with this name exists.
It appears that having this deleted cask installed just causes all commands to return an error, including the command to delete the offending cask. It seems I am stuck, and cannot remove it.
What did you expect to happen?
I expected to be able to remove references to the deleted cask so I could continue using homebrew
Or alternatively, that homebrew would silently forget about the deleted cask, perhaps.
Step-by-step reproduction instructions (by running brew commands)
You need to have flash-ppapi already installed previosuly, then just do
brew upgrade

Workaround: manually delete /usr/local/Caskroom/flash-ppapi
While this gets things working again, it is still clearly desirable to fix brew itself.

Actually I believe I had removed the flash-ppapi plugin / control panel either manually or using an adobe uninstaller previously, when flash was deprecated, so it is perhaps my fault that things were not in a consistent state.
Still, it might help for brew to do some sanity check so it does not end up stuck like this.


flaky test: app test TestShutdown unexpected number of goroutines

Description

https://github.com/spacemeshos/go-spacemesh/runs/3443623393
app_test.go:775:
        	Error Trace:	app_test.go:775
        	Error:      	Not equal:
        	            	expected: 4
        	            	actual  : 5
        	Test:       	TestShutdown

this turned out to be quite difficult to reproduce
$ go test -timeout 0 -v ./cmd/node -run TestShutdown -count 100
3 times now and still could not reproduce it

https://github.com/spacemeshos/go-spacemesh/runs/4275261405

Obsolete.


java.lang.NoClassDefFoundError: com/intellij/json/JsonFileType

Plugin got the following exception whenever I attempt to autocomplete (as I type 'require') using plugin version 0.1.1 on IntelliJ 13.1.6:
com/intellij/json/JsonFileType: com/intellij/json/JsonFileType
java.lang.NoClassDefFoundError: com/intellij/json/JsonFileType
at completion.JSRequirePathFinder$1.isJSFile(JSRequirePathFinder.java:185)
at completion.JSRequirePathFinder$1.visitFile(JSRequirePathFinder.java:165)
at com.intellij.openapi.vfs.VirtualFileVisitor.visitFileEx(VirtualFileVisitor.java:124)
at com.intellij.openapi.vfs.VfsUtilCore.visitChildrenRecursively(VfsUtilCore.java:243)
at com.intellij.openapi.vfs.VfsUtilCore.visitChildrenRecursively(VfsUtilCore.java:275)
at completion.JSRequirePathFinder.findFilePathForNameHelper(JSRequirePathFinder.java:129)
at completion.JSRequirePathFinder.findPathsForVarName(JSRequirePathFinder.java:77)
at completion.JSRequireCompletionProvider.addCompletions(JSRequireCompletionProvider.java:83)
at com.intellij.codeInsight.completion.CompletionProvider.addCompletionVariants(CompletionProvider.java:36)
at com.intellij.codeInsight.completion.CompletionContributor.fillCompletionVariants(CompletionContributor.java:155)
at completion.JSRequireCompletionContributer.fillCompletionVariants(JSRequireCompletionContributer.java:34)
at com.intellij.codeInsight.completion.CompletionService.getVariantsFromContributors(CompletionService.java:84)
at com.intellij.codeInsight.completion.CompletionResultSet.runRemainingContributors(CompletionResultSet.java:132)
at com.intellij.codeInsight.completion.CompletionResultSet.runRemainingContributors(CompletionResultSet.java:125)
at com.intellij.codeInsight.template.impl.LiveTemplateCompletionContributor$1.addCompletions(LiveTemplateCompletionContributor.java:75)
at com.intellij.codeInsight.completion.CompletionProvider.addCompletionVariants(CompletionProvider.java:36)
at com.intellij.codeInsight.completion.CompletionContributor.fillCompletionVariants(CompletionContributor.java:155)
at com.intellij.codeInsight.completion.CompletionService.getVariantsFromContributors(CompletionService.java:84)
at com.intellij.codeInsight.completion.CompletionService.performCompletion(CompletionService.java:115)
at com.intellij.codeInsight.completion.CompletionProgressIndicator.a(CompletionProgressIndicator.java:769)
at com.intellij.codeInsight.completion.CompletionProgressIndicator.access$900(CompletionProgressIndicator.java:84)
at com.intellij.codeInsight.completion.CompletionProgressIndicator$1CalculateItems.run(CompletionProgressIndicator.java:751)
at com.intellij.codeInsight.completion.AsyncCompletion$1$1$1.run(CompletionThreading.java:93)
at com.intellij.openapi.application.impl.ApplicationImpl.runReadAction(ApplicationImpl.java:911)
at com.intellij.codeInsight.completion.AsyncCompletion$1$1.run(CompletionThreading.java:88)
at com.intellij.openapi.progress.impl.ProgressManagerImpl$2.run(ProgressManagerImpl.java:178)
at com.intellij.openapi.progress.ProgressManager.executeProcessUnderProgress(ProgressManager.java:209)
at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:212)
at com.intellij.openapi.progress.impl.ProgressManagerImpl.runProcess(ProgressManagerImpl.java:171)
at com.intellij.codeInsight.completion.AsyncCompletion$1.run(CompletionThreading.java:84)
at com.intellij.openapi.application.impl.ApplicationImpl$8.run(ApplicationImpl.java:419)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:695)
at com.intellij.openapi.application.impl.ApplicationImpl$1$1.run(ApplicationImpl.java:149)

Thanks for letting me know, I'll get IntelliJ 13.1.6 and take a look.

Cheers!  I hope your plugin gets officially adopted by JetBrains, it's very slick.

Thanks!
The problem is the JSON language library changed in IntelliJ 14, which I didn't realize.
I now need to have a different version of the plugin for IntelliJ 13 / WebStorm 7 that uses the old library, and another version for IntelliJ 14 / WebStorm 8.
I have created a branch with the old JSON language library, please feel free to try it out before I publish it:
https://github.com/jballant/CommonJSAutoComplete/raw/intellij-131-version/CommonJSAutoComplete.jar
Just download the JAR and install it from disk using the IDEA "Plugins" menu, and let me know how it goes.

Yep, that fixed it, I'm seeing autocompletes and no more exceptions in the console on 13.1.6 with the jar you provided.  Bummer the JSON library changed on you, would it be worth it to bundle with a 3rd party library instead?
Hey, while I got you, would it be possible to get a config to enable emitting double quotes in the require's autocomplete instead of single quotes?  Our linter is set to complain when we use single-quoted strings and I always have to change the string myself.  I'll file a new enhancement request if you think it's something worth doing.  Thanks again!

I hadn't thought of linting issues with single quotes--the option to specify makes sense. I can definitely add a feature for that in the next release.
Here is an enhancement issue to track it if you like:
https://github.com/jballant/CommonJSAutoComplete/issues/2

New versions with compatibility restrictions are up. Version 0.1.2 is for IntelliJ 13.1 / WebStorm 7 and 0.1.3 is for IntelliJ 14 / WebStorm  Closing.


[tests] enhance Cert_8_1_01_Commissioning

Add verifications for MLE Discovery, DTLS handshake and JOIN_ENT messages.
Please review https://github.com/openthread/openthread/pull/3441 first.

Are we looking to approve and merge #3441 first?

Are we looking to approve and merge #3441 first?

@jwhui Yes, this PR depends on #3441


Bilibili download HTTP Error 412: Precondition Failed (caused by HTTPError())

Checklist

[x] I'm reporting a broken site support
[x] I've verified that I'm running youtube-dl version 2021.03.03
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar issues including closed ones

Verbose log

C:\Users\Yuxi>youtube-dl https://www.bilibili.com/video/BV1e64y1c7PG --verbose
[debug] System config: []
[debug] User config: ['-o', '~/Desktop/youtube-dl_videos/%(title)s-%(id)s.%(ext)s']
[debug] Custom config: []
[debug] Command-line args: ['https://www.bilibili.com/video/BV1e64y1c7PG', '--verbose']
[debug] Encodings: locale cp936, fs mbcs, out cp936, pref cp936
[debug] youtube-dl version 2021.03.03
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.19041
[debug] exe versions: ffmpeg 4.2, ffprobe git-2019-10-19-31aafda
[debug] Proxy map: {}
[BiliBili] 1e64y1c7PG: Downloading webpage
[BiliBili] 1e64y1c7PG: Downloading video info page
WARNING: Unable to download JSON metadata: HTTP Error 412: Precondition Failed
[BiliBili] 1e64y1c7PG: Downloading video info page
ERROR: Unable to download JSON metadata: HTTP Error 412: Precondition Failed (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpvjygsik_\build\youtube_dl\extractor\common.py", line 632, in _request_webpage
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpvjygsik_\build\youtube_dl\YoutubeDL.py", line 2275, in urlopen
  File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
  File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
  File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
  File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
  File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default

Description

Happens to all bilibili videos I tested.

Does "fixed" mean it will be fixed in the next version? Sorry if this is a dumb question.

Does "fixed" mean it will be fixed in the next version?

yes.


Fix memory leak

@DooblyNoobly says there's a memory leak, possibly related to the ping display. Needs to be identified and fixed.

This is probably related to serverlist code

it doesn't seem like the mem leak is happening anymore.


Added center() function to blit::Rect

Minor enhancement, after I found myself writing code to get the middle of a Rect for the 42nd time.

I dig this! Thank you.

I dig this! Thank you.


Add .gitignore and .gitattributes support to the config by the Finder

Many PHP projects use NPM to build frontend assets. NPM installs third party packages to the node_modules directory. These packages sometimes contain PHP files, for example jquery-ui examples. Avoid running php-cs-fixer on these files in the default configuration.

While an edge case, I think it can be fixed in a broader, generalised way, by parsing (and excluding the results of) ignore files like .gitignore. That, however seems more like something to suggest to Symfony for the Finder component. I'd 👍 that too.
node_modules should be in there anyway 😉

I think ignoring things in .gitignore would be a good idea, there's a ticket for adding support for it in symfony/finder https://github.com/symfony/symfony/issues/26714

I'm not in favor of listing more excludes on default, but I do like the idea in general.
Therefore I've tagged this is as WIP as I think having this through the Finder package is the way to go.

Thanks for all the feedback. I agree, using .gitignore is a better and more general solution.

@jdufresne , maybe you can help with this topic by solving it directly in Symfony Finder component ?

unblocked through; https://github.com/symfony/symfony/pull/30448

let's call ignoreVCSIgnored if it's available

Replaced by #4720.


Don't redeploy binaries that haven't changed

Deploying a new release should keep elixir and relay images from redeploying if their checksum did not change so that websockets are not needlessly closed.

After chatting with @thomaseizinger. This is not really feasible in Rust: binaries' checksums aren't necessarily deterministic

After chatting with @thomaseizinger. This is not really feasible in Rust: binaries' checksums aren't necessarily deterministic

FTR: It is possible but already quite hard if we use the same Git commit. Building an exactly identical binary across different commits of a monorepo is probably even harder.
I don't think it is worth the effort at this point. Maybe once we do things like f-droid for Android we can look more into entirely deterministic builds. It would be nice from a security PoV.


Moves of mewtwo

according to this page:
https://poke-go-breeze.vercel.app/pokemon/150
Mewtwo's best comb is Confusion + Psystrike[Elite].
However, this page doesn't show this comb on non-shadow Mewtwo.
Is this a display bug or Psystrike is only for shadow Mewtwo?

Is this a display bug or Psystrike is only for shadow Mewtwo?

This issue is display not correct in dps&tdo sheet page, I will be fix it later.

@g2384 This issue was fixed, Please check in that page again.
Closing issue
Thank!

@g2384 This issue was fixed, Please check in that page again. If still this comb not display, Tell me again.
Thank!

thank you, it's fixed now.


nuttx-bringup: multicopter: motors won't spin

@davids5 This needs more investigation. I flashed the nuttx-bringup branch onto a pixhawk on a multicopter. I can arm but the motors won't spin if I throttle up. It seems that the controller is running correctly and publishing meaningful data to the actuator controls uorb topic.
Is this a known issue or do you wan't me to investigate further? (E.g. look at pwm output, mixer, ...)

 Only turn on the RC after you hear the confirmation sound of the ESC power-up.

@thomasgubler This is the first report of it. Anything you can do to isolate it would be very much appreciated.

@Zefz that's the same symptom I had in #1649. Does a sleep after starting commander help?

@thomasgubler I just pushed a fix that deals with the communications with the pio. I can imagine that this issue you're seeing might have been related. Would you please retest with the latest nuttn_bringup  9dcd7ef35e0beb8f30f3901759210f14a64287ff

@davids5 unfortunately that didn't help. I will investigate this a bit more...

The issue is that the mixer is not loaded correctly because some variables ate not expanded in the rc.interface script. Fixed in #1708


Fomantic usage

Since Fomantic UI was adopted after this project already had some significant implementation in place, it has ended up that a lot of the codebase has an inconsistent usage of the framework versus vanilla CSS or other hardcoding (colors, etc). We need to take a solid pass through the codebase to ensure we are using Fomantic to its full capabilities consistently.

When working on this issue, it is important to note that we should not use Fomantic to remake the UI exactly as it exists now in vanilla CSS. Rather, we want to take advantage of Fomantic's design choices to make the UI look better, as well as making our code cleaner.


Error with selector that contains ()

Error: Parse error on line 174:
...: middle;}svg:not(:root) {  overflow:
---------------------^
Expecting 'IDENT', '[', 'S', got ':'

This issue is still not resolved, am also facing the same issue

Fix this at new version.


fix wrong log warning

Make sure set the target branch to develop
What is the purpose of the change
XXXXX
Brief changelog
XX
Verifying this change
XXXX
Follow this checklist to help us incorporate your contribution quickly and easily. Notice, it would be helpful if you could finish the following 5 checklist(the last one is not necessary)before request the community to review your PR.

[x] Make sure there is a Github issue filed for the change (usually before you start working on it). Trivial changes like typos do not require a Github issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
[x] Format the pull request title like [ISSUE #123] Fix UnknownException when host config not exist. Each commit in the pull request should have a meaningful subject line and body.
[x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[x] Write necessary unit-test(over 80% coverage) to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in test module.
[x] Run mvn -B clean apache-rat:check findbugs:findbugs checkstyle:checkstyle to make sure basic checks pass. Run mvn clean install -DskipITs to make sure unit-test pass. Run mvn clean test-compile failsafe:integration-test  to make sure integration-test pass.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

Codecov Report

Merging #4984 (d546474) into develop (86e4354) will decrease coverage by 0.03%.
The diff coverage is 100.00%.

@@              Coverage Diff              @@
##             develop    #4984      +/-   ##
=============================================
- Coverage      43.24%   43.21%   -0.04%
- Complexity      7723     7730       +7
=============================================
  Files            995      995
  Lines          69066    69069       +3
  Branches        9143     9143
=============================================
- Hits           29871    29849      -22
- Misses         35452    35480      +28
+ Partials        3743     3740       -3

Impacted Files
Coverage Δ

...rocketmq/remoting/netty/NettyRemotingAbstract.java
53.95% <100.00%> (+3.95%)
:arrow_up:

...a/org/apache/rocketmq/broker/BrokerController.java
43.84% <0.00%> (-3.52%)
:arrow_down:

...a/org/apache/rocketmq/filter/util/BloomFilter.java
60.43% <0.00%> (-2.20%)
:arrow_down:

...he/rocketmq/broker/processor/PopReviveService.java
38.35% <0.00%> (-1.80%)
:arrow_down:

...ava/org/apache/rocketmq/filter/util/BitsArray.java
58.11% <0.00%> (-1.71%)
:arrow_down:

.../apache/rocketmq/store/ha/DefaultHAConnection.java
65.86% <0.00%> (-1.21%)
:arrow_down:

...cketmq/broker/schedule/ScheduleMessageService.java
55.26% <0.00%> (-1.18%)
:arrow_down:

...a/org/apache/rocketmq/store/StoreStatsService.java
39.77% <0.00%> (-0.56%)
:arrow_down:

.../org/apache/rocketmq/store/ha/DefaultHAClient.java
61.53% <0.00%> (-0.52%)
:arrow_down:

...he/rocketmq/client/impl/consumer/ProcessQueue.java
61.46% <0.00%> (-0.46%)
:arrow_down:

... and 16 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Minimum bar - compile on Windows (containerd dependency)

Signed-off-by: John Howard<EMAIL_ADDRESS>@icecrime - Minimum bar to even start looking at containerd. Containerd pulls in opencontainer\specs which does not compile on Windows. This needs merging here and vendoring into containerd to make any chance of starting forward progress.

I think https://github.com/opencontainers/specs/pull/310 is working on this.

@hqhq Right, yes, that PR does meet the minimum bar for compiling on Windows (but I doubt it's usable in its current form on Windows). @vbatts Do you know what that PR is held up on? I'm kind of stuck making any forward progress without some fix in specs.

Closing for now. Working around it in containerd in the interim pending #310

thanks @jhowardmsft I'll rebase #310 and finish that work.


Allow chainables to be deferred

Details of the functionality has been added to the README

Was great catching up with you over Meet. I really like the concept and think we should merge it into the lib. Please rework it with the following changes in mind:

Make it look like this feature was in the lib from day one and added by me
So try to merge your code and semantic into my single Chainable class and test
Simplify the names of the new artifacts. Less architect like and more Apple-like
As far as I understand there should be only one visible change there is an execute() and executeDeferred() method. Depending on what you call setShared and the rest has different semantics

Please read and factor in but mostly not implement my earlier review comments below.

Hopefully with the new changes it's now better unified, simpler and exposing much less to the user that will use a deferrable chainable.
I haven't replied yet to any of your other comments (although I have some on draft) since some we saw together in a session, but if you think some still require attention or review let me know.

The README has been simplified significantly to only bring attention to the core deferring feature and main considerations in a more concise way
Indentation issues have been fixed (weird behaviour in my editor)


Contentpack error on Windows installer (DatabaseError: no such table: updates_updateprogresslog)

Summary
It seems necessary to run the server at least once in order to be able to run kalite manage retrievecontentpack local en and not get db errors. After a fresh install on Win7 where I tried kalite manage retrievecontentpack local en prior to running server for the first time and got errors below. I run the server once, and re-tried  kalite manage retrievecontentpack and it finished OK.
I then re-tested on a fresh VM and paid more attention on cmd.exe window during install, and there was a an error but too brief for me to see before the window closed and installer finished. The same happened with 165 today and but this time I saved the output of local command (see below).
Branch or installer
Discovered yesterday on build 161, confirmed today on 165, Windows 7. Latest contentpack from Bamboo.
Traceback or server.log snippet
C:\Python27\Scripts>kalite manage retrievecontentpack local en en.zip
C:\Python27\Scripts\kalite:129: RuntimeWarning: Could not determine Python in yo
ur path, defaulting to C:\Python27\python.exe
  warnings.warn("Could not determine Python in your path, defaulting to {0:s}".f
ormat(python_executable), RuntimeWarning)
[WARNING] [2016-03-02 10:58:43,563] kalite: Channel khan does not exist.
Traceback (most recent call last):
  File "C:\Python27\share\kalite\python-packages\django\core\management\base.py"
, line 224, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "C:\Python27\share\kalite\python-packages\fle_utils\django_utils\command.
py", line 214, in execute
    super(LocaleAwareCommand, self).execute(*args, **kwargs)
  File "C:\Python27\share\kalite\python-packages\django\core\management\base.py"
, line 263, in execute
    output = self.handle(*args, **options)
  File "C:\Python27\lib\site-packages\kalite\distributed\management\commands\ret
rievecontentpack.py", line 76, in handle
    self.setup(options)
  File "C:\Python27\lib\site-packages\kalite\updates\management\commands\classes
.py", line 55, in setup
    self.progress_log = UpdateProgressLog.get_active_log(process_name=self.proce
ss_name)
  File "C:\Python27\lib\site-packages\kalite\updates\models.py", line 168, in ge
t_active_log
    if logs.count() > 0:
  File "C:\Python27\share\kalite\python-packages\django\db\models\query.py", lin
e 372, in count
    return self.query.get_count(using=self.db)
  File "C:\Python27\share\kalite\python-packages\django\db\models\sql\query.py",
 line 423, in get_count
    number = obj.get_aggregation(using=using)[None]
  File "C:\Python27\share\kalite\python-packages\django\db\models\sql\query.py",
 line 389, in get_aggregation
    result = query.get_compiler(using).execute_sql(SINGLE)
  File "C:\Python27\share\kalite\python-packages\django\db\models\sql\compiler.p
y", line 840, in execute_sql
    cursor.execute(sql, params)
  File "C:\Python27\share\kalite\python-packages\django\db\backends\sqlite3\base
.py", line 366, in execute
    six.reraise(utils.DatabaseError, utils.DatabaseError(*tuple(e.args)), sys.ex
c_info()[2])
  File "C:\Python27\share\kalite\python-packages\django\db\backends\sqlite3\base
.py", line 362, in execute
    return Database.Cursor.execute(self, query, params)
DatabaseError: no such table: updates_updateprogresslog

C:\Python27\Scripts>

How to reproduce

Install KA Lite.
Do not start the server.
Run C:\Python27\Scripts>kalite manage retrievecontentpack local en en.zip

Real-life consequences
Installation without properly installed content pack.
@MCGallaspy

Thanks for the report!

Update, hope it helps:
Doing the same on XP, gave a different error:
C:\Python27\Scripts>kalite manage retrievecontentpack local en en.zip
C:\Python27\Scripts\kalite:129: RuntimeWarning: Could not determine Python in yo
ur path, defaulting to C:\Python27\python.exe
  warnings.warn("Could not determine Python in your path, defaulting to {0:s}".f
ormat(python_executable), RuntimeWarning)
Traceback (most recent call last):
  File "C:\Python27\share\kalite\kalitectl.py", line 130, in <module>
    os.mkdir(KALITE_HOME)
WindowsError: [Error 183] Cannot create a file when that file already exists: 'C
:\\Documents and Settings\\IEUser\\.kalite'

C:\Python27\Scripts>

Hmm... On XP running the server once before installing contentpack locally did not have the same effect. I'm still getting the same error as above, no matter which languagepack I try to install:
C:\Python27\Scripts>kalite manage retrievecontentpack local fr fr.zip
C:\Python27\Scripts\kalite:129: RuntimeWarning: Could not determine Python in yo
ur path, defaulting to C:\Python27\python.exe
  warnings.warn("Could not determine Python in your path, defaulting to {0:s}".f
ormat(python_executable), RuntimeWarning)
Traceback (most recent call last):
  File "C:\Python27\share\kalite\kalitectl.py", line 130, in <module>
    os.mkdir(KALITE_HOME)
WindowsError: [Error 183] Cannot create a file when that file already exists: 'C
:\\Documents and Settings\\IEUser\\.kalite'

C:\Python27\Scripts>

XP does seem to have an issue with contentpacks:

they do not work correctly at install time
they can't be installed afterwards

Difference in locales...?
On this XP:

I installed the "bare" build 165 installer, and it had errors at the last step, too fast to see
tried to install en.zip locally, with error 3 posts above
tried to install contentpacks in other languages, with error 2 posts above
installed Spanish langpack from Manage > Languages

Lot of content missing here:

If I switch the same exercise to English:

The last error says "video", but this is an exercise.

Last try, kalite manage unpack_assessment_zip, same error on XP:
C:\Python27\Scripts>kalite manage unpack_assessment_zip /f khan_assessment.zip
C:\Python27\Scripts\kalite:129: RuntimeWarning: Could not determine Python in yo
ur path, defaulting to C:\Python27\python.exe
  warnings.warn("Could not determine Python in your path, defaulting to {0:s}".f
ormat(python_executable), RuntimeWarning)
Traceback (most recent call last):
  File "C:\Python27\share\kalite\kalitectl.py", line 130, in <module>
    os.mkdir(KALITE_HOME)
WindowsError: [Error 183] Cannot create a file when that file already exists: 'C
:\\Documents and Settings\\IEUser\\.kalite'

C:\Python27\Scripts>

Ah, I forgot that you made this report! But this is fixed in #374

Hm, except possbily for the issue with XP you identify.

Yes, better to leave it open until it's confirmed on XP.

Still extant on Windows 8.1 with build 176 @MCGallaspy! #374 is merged, did it make into that build?

C:\Python27\Scripts>kalite manage retrievecontentpack local en en.zip
C:\Python27\Scripts\kalite:129: RuntimeWarning: Could not determine Python in yo
ur path, defaulting to C:\Python27\python.exe
  warnings.warn("Could not determine Python in your path, defaulting to {0:s}".f
ormat(python_executable), RuntimeWarning)
[WARNING] [2016-03-14 07:00:57,698] kalite: Channel khan does not exist.
Traceback (most recent call last):
  File "C:\Python27\share\kalite\python-packages\django\core\management\base.py"
, line 224, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "C:\Python27\share\kalite\python-packages\fle_utils\django_utils\command.
py", line 214, in execute
    super(LocaleAwareCommand, self).execute(*args, **kwargs)
  File "C:\Python27\share\kalite\python-packages\django\core\management\base.py"
, line 263, in execute
    output = self.handle(*args, **options)
  File "C:\Python27\lib\site-packages\kalite\distributed\management\commands\ret
rievecontentpack.py", line 70, in handle
    self.setup(options)
  File "C:\Python27\lib\site-packages\kalite\updates\management\commands\classes
.py", line 55, in setup
    self.progress_log = UpdateProgressLog.get_active_log(process_name=self.proce
ss_name)
  File "C:\Python27\lib\site-packages\kalite\updates\models.py", line 168, in ge
t_active_log
    if logs.count() > 0:
  File "C:\Python27\share\kalite\python-packages\django\db\models\query.py", lin
e 372, in count
    return self.query.get_count(using=self.db)
  File "C:\Python27\share\kalite\python-packages\django\db\models\sql\query.py",
 line 423, in get_count
    number = obj.get_aggregation(using=using)[None]
  File "C:\Python27\share\kalite\python-packages\django\db\models\sql\query.py",
 line 389, in get_aggregation
    result = query.get_compiler(using).execute_sql(SINGLE)
  File "C:\Python27\share\kalite\python-packages\django\db\models\sql\compiler.p
y", line 840, in execute_sql
    cursor.execute(sql, params)
  File "C:\Python27\share\kalite\python-packages\django\db\backends\sqlite3\base
.py", line 366, in execute
    six.reraise(utils.DatabaseError, utils.DatabaseError(*tuple(e.args)), sys.ex
c_info()[2])
  File "C:\Python27\share\kalite\python-packages\django\db\backends\sqlite3\base
.py", line 362, in execute
    return Database.Cursor.execute(self, query, params)
DatabaseError: no such table: updates_updateprogresslog

C:\Python27\Scripts>

Still extant on Windows 8.1 with build 176

Did the installer fail to unpack the content pack? Note that you shouldn't need to run the command retrievecontentpack yourself after installation, but if you do for some reason it will fail unless you include the --foreground option. That's the fix incorporated by #374.

Installer did not fail, it was my bad, I re-run the retrievecontentpack and did not include the --foreground option. The first try (after install, before running the server once) locally failed, second (after running server once) didn't. So I basically installed English contentpack twice... :stuck_out_tongue:
We should document this option, for users who will install contentpacks for other languages prior to running the server, right?

Yes, can you open an issue on the ka-lite repo?


Miniscript.substitute_raw_pkh() is not recursive

I hit a case here (the last assert fail) where the RawPkH seems not substituted.
It looks to me it's due to the fact the method is not recursive and the returned Miniscript is created from the last element of translated.

Fixed by #725


How can Stringtie decide the strand of a transcript assembled by a standard RNA library?

Hi,
I am new to the RNA-seq. From my opinion, the strandness is lost after both strands of cDNA are synthesized. My question is :
(1) For a typical RNAseq, is the strand info from Stringtie relaible?
(2) Should we just focus on the interval of transcript created by Stringtie and ignore the strand? In that case, we should search for ORF on both strand based on genome and decide the coding potential.
Thanks

The strand info for alignments of reads from unstranded RNA-Seq libraries can be recovered based on two sources of evidence - the primary one being the genomic splice sites in the case of spliced alignments (intron in the alignment), which is encoded by the aligner into the XS tag for such alignments (that's why the large majority of transcript assemblies lacking strand info are single exon) . A secondary source can be the overlap with reference annotation transcripts (gffcompare can use that). So the answers to your queries can be:
(1) generally yes, but only in so far as the aligner provided strand info (the XS tag) is reliable
(2) yes, that's how it is usually done - you can use programs like TransDecoder or EviGene to help with assessing the ORFs.
Be aware that those single-exon transcripts lacking strand info, especially those with low estimated coverage, can be transcriptional noise or alignment artifacts (i.e. not "real" transcripts). Some of those could be alignments to processed pseudogenes.

OK. Thank you very much!

Hi,
Thank you very much for apply. I still have another question.
First, I used stringtie(v2.1.5) to separately assembly the transcripts for three samples.
stringtie -l SAMPLE1 -p 30 -G  /GRCh38.p13/GRCh38_latest_genomic.gff -A sample1_gene_abund.tab -o sample1_stringtie.gtf sample1.sortedByCoord.out.bam
Then, I used the gffcompare to merge the three gtfs.
gffcompare -o gffcompare -p GFFCOMPARE sample1_stringtie.gtf sample3_stringtie.gtf sample3_stringtie.gtf
At last, I used the stringtie to calculate the TPM using the merged gtf.
stringtie -e -B -p 10 -G gffcompare.combined.gtf -o sample1_abun.gtf sample1.sortedByCoord.out.bam
But I found that the TPM differes very much.
In gffcompare.tracking:
GFFCOMPARE_00000001	XLOC_000002	-	u  q1:SAMPLE1.29|SAMPLE1.29.1|2|0.715953|2.892788|41.071835|4412
But in the sample1_abun.gtf:
gene_id "XLOC_000002"; transcript_id "GFFCOMPARE_00000001"; cov "4.525456"; FPKM "0.099625"; TPM "0.359771";
Which one should I use? If there is no such transcript in the first assembly, its expression value should be 0, then even when the combined gtf (contain more transcripts) is used for quantification, the TPM should not be changed so much.
Am I wrong in some places? Hope to hear your advice.
Best wishes
Mengyu Zhou


Could not install to device Android

Godot version:
Godot 3.1 RC
OS/device including version:
Android 7.1
Issue description:
Could not install to device Android this aperan when i compile the game to Android devices and the generat apk don't work i using Godot_v3.1-rc1_export_templates
Steps to reproduce:
Minimal reproduction project:

Please be more specific. "Wont work" is not a good description.
What happens ?

Can't reproduce the issue, I exported a debug APK to my Android 9 phone and my Android 7.1.2 phone without issue.
So indeed, without more info, we can't do anything.

I will be more specific:
My working enviroment is Ubuntu 16.04 LTS 64bits
What happening is in Godot 3.1 Beta 11 and Godot 3.1 RC is when i compile the platformer demo to Android when it finish and godot try to install in device apperas the window "with Load error could install on device". i check if the editor config was bad but it looks Ok is the same config i have in Godot 2.1.5 and Godot 3.0.6 i have no problems with those Releases and i have the same config. here are somo screenshots i promise be more specifig in the future and thanks you for you reply i really love Godot Engine

ADB should print errors in the terminal, can you paste them here?

2019-03-09 18:51:36.661 920-1025/? E/PackageInstaller: Commit of session<PHONE_NUMBER> failed: Package /mnt/expand/3f7cbaf6-0aba-4589-b109-176472ae748d/app/vmdl1328964467.tmp/base.apk has no certificates at entry AndroidManifest.xml
The log from ADB is weird because in the editor setting i have the same conf in godot 2.1.5, 3.0.6 and 3.1

i think i find the problem was my phone i just formated and complie the game and runs it well
sorry for the inconvenient


deps: bump google.golang.org/protobuf to 1.33.0

Fixes: #3216

security run: https://github.com/keptn/lifecycle-toolkit/actions/runs/8232982954

Closing in favor of https://github.com/keptn/lifecycle-toolkit/pull/3230


dateTime was ignoring specified local_or_utc value

Line 1210: always assumed the provided time_t was in local time.
thus the following output:
Timezone localTz;
localTz.setLocation("Australia/Melbourne");

// -- works for TIME_NOW as it gets hydrated in local time
Serial.println("Current Time: ");
Serial.print("         UTC: "); Serial.println(UTC.dateTime());
Serial.print("       Local: "); Serial.println(localTz.dateTime());
Serial.println();

// Current Time:
//          UTC: Saturday, 03-Aug-2019 03:56:27 UTC
//        Local: Saturday, 03-Aug-2019 13:56:27 AEST

time_t testTime =<PHONE_NUMBER>;
Serial.println("    Simulate: ");
Serial.print("       Local: "); Serial.println(localTz.dateTime(testDate));
Serial.print("       Local: "); Serial.println(localTz.dateTime(testDate, LOCAL_TIME));
Serial.print("       Local: "); Serial.println(localTz.dateTime(testDate, UTC_TIME));

//  Simulate:
//       UTC: Sunday, 04-Aug-2019 21:25:37 UTC
//     Local: Sunday, 04-Aug-2019 11:25:37 AEST
//     Local: Sunday, 04-Aug-2019 11:25:37 AEST
//     Local: Sunday, 04-Aug-2019 11:25:37 AEST <--- wrong

with this PR:
// Current Time:
//          UTC: Saturday, 03-Aug-2019 03:56:27 UTC
//        Local: Saturday, 03-Aug-2019 13:56:27 AEST
//
//  Simulate:
//       UTC: Sunday, 04-Aug-2019 21:25:37 UTC
//     Local: Sunday, 04-Aug-2019 11:25:37 AEST
//     Local: Sunday, 04-Aug-2019 11:25:37 AEST
//     Local: Monday, 05-Aug-2019 07:25:37 AEST <--- correct

Thanks Transend... I can confirm this addresses an issue I was having when converting UTC to BST.

Confirming this issue. Any info when the PR will be merged?


Fix a few math issues on Cygwin (MinGW)

[x] Bug testcase
[x] Fix
[x] Add asserts for ISO C assumptions
[x] Generalize NetBSD workaround config option
[x] NaN ** 0 should be 1, but is NaN
[x] (-1e100) % (-1/0) should be -1e100, but is NaN
[x] (1/0) % (1/0) (and any inf sign combination) should be NaN (not +/- inf)
[x] 0 % x should be 0, preserving sign, for any x not a NaN or 0
[x] 0 % (0/0)) and 0 % 0 should be NaN
[x] Math.atan2(1/0, 1/0) should be 0.785... but is NaN; same NaN issue for infs of any sign
[x] Math.pow(0/0, 0) and Math.pow(0/0, -0) should be 1, but is NaN
[x] 2.0 migration notes
[x] Releases entry

I assume this is an issue with the CRT used in Cygwin.

Most likely, unless intrinsics were used (for % that might be possible). The easiest fix is to special case a few of the required behaviors for Cygwin (same as for NetBSD pow()). It might be a bit overbroad but pinpointing a very specific condition is also quite difficult.

No time to finish the fix right now but there's a specific bug testcase in the pull now.

For simplicity it's probably best to widen the set of NetBSD fixes to include the MinGW issues too, and then change DUK_USE_POW_NETBSD_WORKAROUND to be a bit more generic like DUK_USE_POW_WORKAROUNDS.

As an aside, I find the specified behavior of NaN ** 0 = 1 to be odd - returning NaN for such operation seems more intuitive.

Quite a lot of small things broken with MinGW. I'm not sure if all of them are actual ISO/ANSI C guarantees or just guarantees that are usually in place. There are asserts for the assumptions now however so easier to catch if some other runtime violates them.

After maybe 10 attempts I think I finally got all the cases handled. Phew. :)


The rolling-release branch doesn't build anymore

hash mismatch in fixed-output derivation '/nix/store/rdjbmxz6s8fn4vdq7jmfcdca2bbk3rd3-lorri-vendor':
  wanted: sha256:094w2lp6jvxs8j59cjqp6b3kg4y4crlnqka5v2wmq4j0mn6hvhsj
  got:    sha256:1ix9rmpfhs06xzccvx8xpdl60849ygm25rmixa69wkk0vs3s1bz2

It works on my machine. What system are you building on, how do you invoke it? I’m sorry, I’m afraid can’t help you with just the error message.

Turns out i was doing an over-complicated thing that caused nixpkgs mismatch:
{ pkgs }:
let
  lorri-source = fetchGit {
    url = "https://github.com/target/lorri";
    ref = "rolling-release";
  };
in (import lorri-source) {
  inherit pkgs;
  src = lorri-source;
}

This is the proper way:
{ ... }:
import (fetchTarball {
  url = https://github.com/target/lorri/archive/rolling-release.tar.gz;
}) {}


Meta example integration for all targets

This task is to enable the integration testing of meta examples for all target languages. Currently it only is enabled for cpp. That is, only the cpp meta examples contain will contain code that serialises results and compares them to a reference file. This involves

enabling the storeVars flag in the generator.py file other more languages, one by one
This enables the generated listing to contain serialization code
This code needs to work, there might be issues such as missing typemaps (java), templating problems, etc
once this works, we have integration testing across all target languages

@karlnapf I would like to work on the issue. How do I start?

This one is tricky, I recomend you start understanding our meta example system first, for example via adding a new example (picky any python modular one and translate it to meta example, check this Readme

I solved this at the GSoC 2016 mentor summit


Drop support for old python - cleanup - up version

With version 4.0.0

Drop support for python 2.6 & < 3.4.5
Clean up
Up version

Coverage increased (+1.2%) to 92.241% when pulling 26f96d78ddd9686c15f685d407dae42a757eec35 on staging into b9db1bc87e3b19c5c63ab7a00b19685b7996de5d on master.


Table.rename does not rename keys of Row items

Looks like Table.rename() calls Table.fork(), which calls Table.init() with _is_fork=True which skips the code in this if block. Because of that, the _keys attribute of each Row instance retains the old values.

Hmmm, even when you pass row_names? That would be a rather large oversight on my part...

@onyxfish Yes, I believe so.  I'll try to write a test that breaks this when I have a second

would be a rather large oversight on my part as well...

Sorry @onyxfish, @nbedi. Looks like column names in rows aren't renamed, in terms of dictionary access.  Here's the output from this test:
nosetests tests/test_table.py
...........................................................................E.......................................................
======================================================================
ERROR: test_rename_column_names_renames_row_values (tests.test_table.TestData)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/ghing/workspace/agate/tests/test_table.py", line 1733, in test_rename_column_names_renames_row_values
    self.assertEqual(row[key], row2[key2])
  File "/Users/ghing/workspace/agate/agate/mapped_sequence.py", line 69, in __getitem__
    return self.dict()[key]
KeyError: 'd'

----------------------------------------------------------------------
Ran 131 tests in 0.453s

FAILED (errors=1)

Looks like Table.rename() calls Table.fork(), which calls Table.__init__() with _is_fork=True which skips the code in this if block.  Because of that, the _keys attribute of each Row instance retains the old values.

@ghing @onyxfish That does seem to be the issue. I added a pull request with a possible fix and @ghing's test for changing row keys. Let me know if that works or if you'd like it implemented differently. Sorry for the trouble.

Ah ha, this is an interesting bug. I overlooked it because that bit of state did not used to be part of the Row.
@nbedi Your fix looks good to me. Merged in!

I revised the test to use assertColumnNames and updated that testcase method to properly validate those keys. Should prevent errors like this in the future:
https://github.com/onyxfish/agate/commit/5997dea4365e1ee5f8ec8c839106542bc6144ed1


[Devtools/Objectinspector] Filter

Change Type (required)
Testing cannary version of player/devtools-assets @0.1.2--canary.27.1288

[ ] patch
[ ] minor
[ ] major

/canary

/canary

Closed in favor of signing commits.


Remove v1 resourceset

Towards giantswarm/giantswarm#7896
Completes go part of the flattening. Next will be helm chart changes.

Tested on geckon.


Integrate client-cli with runtime

[x] Update docs
[x] Expose a RuntimeApi class so we can start a full runtime application from within the same process (no worker thread)
[x] Drop requirement for url property in client https://github.com/platformatic/platformatic/issues/1144
[x] Allow generating a client from a service running within the runtime #1086

Fixes #1086
Fixes https://github.com/platformatic/platformatic/issues/1144

@cjihrig PTAL

Looks like there are some valid CI issues.

@mcollina This is working great! 🎉


Update InvertBinaryTree.swift

The space analysis need to be more accurate. Similar problems happen to SameTree as well

Thank you for your comment and appreciate your help!


API to hot-restart (not hot-reload) the Flutter app?

I need to control a Flutter app (running in dev mode), to make it hot-restart. Thus, I wonder how to implement it? What is the API call to, maybe, vmservice?
P.S. I see https://github.com/dart-lang/sdk/blob/main/runtime/vm/service/service.md#reloadsources but it seems to be for hot-reload, not hot-restart.
I guess it is maybe not documented?

Flutter tools has a mechanism to restart an app, I believe the Flutter engine starts the app on a new Isolate. This is also
supported in the IDE tools.
The VM service does not provide a mechanism to restart an app.

Thanks. Indeed I have found the solution: just mimic what devtools does to "hot restart".

Hello @fzyzcjy.
I wanna do the same thing but I am having trouble to mimic how devtools does to "hot restart".
can you please provide me how you did it.
Thank you very much

I guess it is: https://github.com/fzyzcjy/flutter_convenient_test/blob/13b555e0f17921cae2f5940330f850d6b70d265e/packages/convenient_test_manager_dart/lib/services/real_vm_service_wrapper_service.dart#L113

Thanks


tf-2.0.0rc0-gpu tf.dataset bug

Please make sure that this is a bug. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template
System information

Have I written custom code: yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04 LTS
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
TensorFlow installed from (source or binary): docker image
TensorFlow version (use command below): tensorflow/tensorflow:2.0.0rc0-gpu-py3
Python version: 3.6
Bazel version (if compiling from source):
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version: CUDA 10
GPU model and memory:

Describe the current behavior
I am using tf.keras for training with the dataset api and TFrecords. At the beginning of the training the dataset api tries to load and shuffle the whole training dataset so the memory fills up and the process dies.  I cannot train my model at all. I am not using any .shuffle() calls in my dataset pipeline.
Describe the expected behavior
The dataset should not load and shuffle the records unless the .shuffle() call is issued.
Code to reproduce the issue
import numpy as np
import tensorflow as tf
import os
import cv2
import tensorflow.keras.layers as K

def extract_fn(data_record):
    features = {
        'data': tf.io.FixedLenFeature([], tf.string)
    }
    sample = tf.io.parse_single_example(data_record, features)
    data = tf.image.decode_image(sample['data'])

    return data, 1.

class DataGenerator(tf.keras.utils.Sequence):

    def __init__(self, dataset_iterator, len):
        self.dataset_iterator = dataset_iterator
        self.len = len

    def __len__(self):
        # number of batches per epoch
        return self.len

    def __getitem__(self, index):
        # Generate one batch of data

        next_element = next(self.dataset_iterator)
        x = next_element[0]
        y = next_element[1]

        return x, y

with tf.io.TFRecordWriter("dummy_dataset.tfrecords") as writer:
    data = np.float32(np.random.random(size=(1000, 1000, 3)) * 255)
    data = cv2.imencode(".png", data)[1].tostring()
    example = tf.train.Example(features=tf.train.Features(
        feature={'data': tf.train.Feature(bytes_list=tf.train.BytesList(value=[data]))}))
    for i in range(10000):
        writer.write(example.SerializeToString())

dataset = tf.data.TFRecordDataset(["dummy_dataset.tfrecords"])
dataset = dataset.map(extract_fn)
n_batch = 3
dataset = dataset.batch(batch_size=n_batch, drop_remainder=True)

dataset = dataset.repeat(5)
dataset_iterator = iter(dataset)
next_element = next(dataset_iterator)

data_generator = DataGenerator(dataset_iterator, int(10000/n_batch))

input = K.Input(shape=(1000, 1000, 3), name='input')
net = K.Conv2D(1, 3, activation='sigmoid')(input)
output = K.GlobalAveragePooling2D()(net)
model = tf.keras.models.Model(inputs=input, outputs=output)

model.compile(loss='mse', optimizer='sgd')

model.fit(x=data_generator, epochs=10)

Other info / logs
note that code generates about 28GB dummy data in a tfrecord file since I am having this issue when I am trying to use the dataset api with tf.keras.utils.Sequence on a tfrecord file.
EDIT(robieta): code formatting.

@LaCandela ,
In order to expedite the trouble-shooting process, please provide complete code snippet to reproduce the issue reported here. Thanks!

I found that the issue arises when I am trying to use tf.keras.utils.Sequence to create a data generator and using that to feed the model.fit. I provided a small script. Please note that it generates about 28GB dummy data in a tfrecord file since I am having this issue when I am trying to use the dataset api with tf.keras.utils.Sequence on a tfrecord file. I need to use the tf.keras.utils.Sequence because it allows me to do some preprocessing that I can performe on GPU. As best I know, I cannot do that with dataset.map().

Hi! I was wondering if there was any progress in this issue...

@LaCandela I reduced the dummy_data size and ran it with TF2.0. I don't see any issues. Here is the gist. Thanks!

@jvishnuvardhan Well, My TFrecords file size for the training is in the range of ~30GB and I cannot reduce that. The dataset api tries to fill a shuffle buffer which I am not calling at all (no dataset.shuffle() call) and quickly fills up my memory. It seems as if the whole TFrecords is loaded into memory. The issue is still present in TF2.0.

Why are using DataGenerator instead of passing your dataset directly to model.fit?

Because I want to do the preprocessing on the GPU and if I use the dataset.map() function the operations are performed on the CPU (as best I know). Now, I have the feeling that it is against the Tensorflow "mindset" but it would be great to have the possibility to experiment with the preproc (augmentation, ground truth generation) on the GPU approach.

The map dataset op will be placed on CPU but ops inside of the user-defined function passed into the map transformation can be placed on GPU.
As for the problem you are seeing, my best guess is that something in Keras internals might end up prefetching the contents of the entire generator ahead of time. @robieta could you please comment on what could be going on? Thanks.

I believe the issue is that shuffle defaults to True in model.fit, which will call .shuffle(len(sequence)) internally. (Prefetching all of the elements.) It does, however, strike me that for a Sequence this is not very efficient. (Since we can just directly index said sequence) I am already working on the data adapter; I will update it accordingly.

@LaCandela Could you check with latest TF version and let us know if the problem still persists or not.    Thanks!

@LaCandela  Any updates regarding this issue? Thanks!

@LaCandela  Closing due to lack of recent activity. Please update the issue when new information becomes available, and we will reopen the issue. Thanks!


Fix accessor creation when the first Realm opened is read-only

When a read-only Realm is missing tables, we can't read the column order from the table, so we were creating accessors where every property was using column 0, and then reusing those accessors for Realms opened later which do have the tables. This fixes the problem by explicitly checking _column in isEqualToProperty: rather than relying on the order of the properties in the array, and as an optimization skips creating the accessors for missing tables.
Fixes https://secure.helpscout.net/conversation/67013100/364/ and hopefully the other reports of column type mismatches.
@alazier

Is this something users have been hitting? If unsere, can you ping the users who have hit the column assertion and ask if they are using real-only realms? Would be nice to know if this issue is resolved or not.

I linked the helpscout thread where a user hit this and gave sample code...

I thought we had multiple users hitting the col_type_StringEnum assertion. Is this not the case?

We've had several people report similar problems, yes.

:+1:


Adding the ability to wait on a coroutine promise

@jeremeamia

_next_coroutine is recursive, but does it avoid increasing the recursion stack?

For most cases: yes. It handles the case where recursion can occur by checking if a promise will be immediately invoked, and in that case, it just keeps sending to the generator without recursing. When a promise is not resolved, it will be resolved outside of the function. A then is added to each of these promises to resume the coroutine.
$p1 = new Promise\Promise(function () use (&$p1) {
    echo "Resolve: " . xdebug_get_stack_depth() . "\n";
    $p1->resolve('skip me');
});
$p2 = new Promise\Promise(function () use (&$p2) {
    echo "Resolve: " . xdebug_get_stack_depth() . "\n";
    $p2->resolve('hello!');
});
$co = Promise\coroutine(function() use ($p1, $p2) {
    echo "Coroutine: " . xdebug_get_stack_depth() . "\n";
    yield $p1;
    yield $p2;
});

$co->wait(); // hello!

Outputs:
Coroutine: 15
Resolve: 20
Resolve: 20

The only case I see that would be recursive is try/catching.

Got it. cool. :ship:


nil check for testing if it was a cql package was incorrect.

NOTE changes also need to be made in bonnie.

it looks like the  build error here is related to cql4bonnie updates in HDS. when it points to the old HDS, tests pass fine. When it points to the cql4bonnie HDS, tests do not pass.


Options form cut off at 450px

Browser
MacOS
Version 68.0.3440.106 (Official Build) (64-bit)
Also
Version 69.0.3497.92 (Official Build) (64-bit)
Screenshot

I see, this looks fine when "open_in_tab": false (see below). The above screenshot is when "open_in_tab": true. Perhaps conditionally include CSS based on the options, since the README says both are supported?


Add LocalLLM

LocalLLM
Query LLM models locally through Ollama
By submitting this pull request I confirm I've read and complied with the below requirements 🖖
Requirements for your pull request

You have read and understood the Contribution Guidelines.
This pull request has a title in the format Add [Name of Plugin].

✅ Add GEmojiSharp
❌ Update readme.md
❌ Adding GEmojiSharp
❌ Added GEmojiSharp

Your entry should include a short description of the plugin. The first character should be uppercase and the description should end in a dot. It should be an objective description and not a tagline or marketing blurb.

✅ - [GEmojiSharp](…) - Find and copy GitHub Emoji to the clipboard.
❌ - [GEmojiSharp](…) - A GitHub Emoji plugin for PowerToys Run.
❌ - [GEmojiSharp](…)

The link text should be the Name of the plugin found in the plugin.json file. The URL should be the link to the GitHub repo, or a link to an anchor in the readme.

✅ - [GEmojiSharp](https://github.com/hlaueriksson/GEmojiSharp) - …
❌ - [GEmojiSharp.PowerToysRun](…) - …
❌ - [GEmojiSharp](https://github.com/hlaueriksson/GEmojiSharp/releases/download/v1.0.0/GEmojiSharp.PowerToysRun.1.0.0.zip) - …

Your entry should be added at the bottom of the appropriate category.

Hi @Darkdriller
Thanks for your contribution!
Nice plugin!
I have tried it and it works great.
I created two issues with feedback in your repo.
I hope you have time to take a look.

Hi @hlaueriksson.
I resolved the conflicts.

You must end the description with a period (.), otherwise the repo will not pass the linter. Can you update the PR?

Thanks for your contribution 🚀

Hi @Darkdriller
I just wanted to let you know that this list is now included in the main awesome list.
And that means that you can add a badge to your repo 😎


Fix Octocoral Density in Habitat

Funk with Octocoral Density in
https://marinebon.org/fk-esr-info/habitat.html:

@superjai identified this problem as being more widespread and related to me having deleted the old data files in commit rm old data · marinebon/fk-esr-info@33bb6bf.
For each file, you'll need to:

Upload the CSV before commit rm old data · marinebon/fk-esr-info@33bb6bf into fknms_esr_indicator_data - Google Drive.
Publish the data as a CSV under the File menu for each and update the icons to data | fk-esr-info - Google Sheets with that CSV publish URL. Please also add the editable Google Sheet Link to the cell's CSV url, so from the icons to data | fk-esr-info - Google Sheets you'll get a preview of the sheet + click to edit it, all the while the differing cell's value is the functional data URL.
Edit each of the page.html manually to update the csvURL.

Finally, you'll need to regenerate all modals, using _make_infographic_modals.R that uses _infographic_modal_template.html.


docs: Add inline comment explaining behavior options

@alikhajeh1 What do you think about explaining the behavior option in the example?

Great idea! I tweaked the wording, feel free to edit/merge.


some free joint not save authentifiers and messages are missing

seems that the nonseiral unit is not delete from units table
@GTnull


affinity, podAntiAffinity, podAntiAffinityTopologyKey added

Thanks for your PR @krakazyabra, I'll review it in a while

sovled in https://github.com/kvaps/kube-linstor/commit/bbf92bd7121a1a12eefb175dd36544b96a0fbdb0


Create keys in other keystore than 'default'

It appears that all keys created using redstone will be placed in the 'default' key ring.  Can a parameter be added to the create_key method which specifies which key ring to place the key in?  It looks like this is specified via the X-Kms-Key-Ring HTTP header for both KeyProtect and Hyper Protect Service APIs.
https://cloud.ibm.com/apidocs/key-protect#createkey
https://cloud.ibm.com/apidocs/hs-crypto#createkey
Thanks!

Hi @rtclauss, yes, there is only CRUD for the KeyRing resources themselves. None of the Key resource action methods have been updated with the key ring support yet.
The key ring header actually affects a lot more than just Key create. In the go SDK we actually have a header set for pretty much every call, and we could do something similar here.


Linux support PR

After a *few* hours, I finally made it run on Linux under Wine. Haven't tested if it compiles, but I believe it does (and I also have no energy to test it further).
You have to make it executable before running it:
chmod +x ./PySilon.sh

Run it and follow the steps. It handles everything so there is no need to install anything else first.
Here is a video on how to configure it for the first time:
(you may want to add this video to the wiki as a tutorial on how to use it)
https://github.com/mategol/PySilon-malware/assets/52631245/2c3a86e1-8613-4c33-aa99-5627deb679e4
There are some extra commits because I synced the main branch to my fork while I was working on it. Don't worry about those

looks wild, why did you close the PR?

Forgot to mention, venv does not work on Wine so the installation of pip requirements is done normally. (you can see in the vid, when the script initializes the virtual environment, wine does not recognize the program)


Update mdoc_2.13, sbt-mdoc to 2.2.19

Updates

org.scalameta:mdoc_2.13
org.scalameta:sbt-mdoc

from 2.2.18 to 2.2.19.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scalameta" } ]

labels: library-update, semver-patch

Superseded by #198.


Improve cleaning up header tables in METviewer pruning

Describe the Enhancement
The query generated in findHeaderIdsForDeletion method of PruneDbManagerMysql class stoles the Aurora database. Need to find better way how to clean up header tables
Time Estimate
Issues should represent approximately 1  day of work.
Relevant Deadlines

NONE.

Funding Source

NONE.

Define the Metadata
Assignee

[ ] Tatiana Burek
[ ]  no scientist required

Labels

[ ] https://github.com/NCAR/METviewer/labels/component%3A database https://github.com/NCAR/METviewer/labels/component%3A server
[ ] https://github.com/NCAR/METviewer/labels/priority%3A medium

Projects and Milestone

[ ] https://github.com/NCAR/METviewer/milestone/9

Define Related Issue(s)
Consider the impact to the other METplus components.

[ ] None

Enhancement Checklist
See the METplus Workflow for details.

[ ] Complete the issue definition above.
[ ] Fork this repository or create a branch of develop.
Branch name: feature_<Issue Number>_<Description>
[ ] Complete the development and test your changes.
[ ] Add/update unit tests.
[ ] Add/update documentation.
[ ] Push local changes to GitHub.
[ ] Submit a pull request to merge into develop, listing the <Issue Number> in the title.
[ ] Iterate until the reviewer(s) accept and merge your changes.
[ ] Delete your fork or branch.
[ ] Close this issue.

Because of this feature, the pruning process can hang and make the database not responsive.
I commented out the code where removing empty header records is performed


SSL enabling for a WLS domain

Test result @ http://wls-jenkins.us.oracle.com/job/weblogic-kubernetes-operator-javatest/3697/
using https://github.com/oracle/weblogic-kubernetes-operator/tree/12-12-2019_develop_OWLS-76552 and https://github.com/qvdang/docker-images

I can't remember if there was some reason we were waiting to review this.  @mriccell, do you know?

@rjeberhard There are some failures when running the full integration tests with this PR.  I'll get back to this task hopefully this week to make some further updates to resolve those. Thanks.


deb: add Ubuntu 24.04 (Noble Numbat)

NOTE:
16.52
16.52 The following packages have unmet dependencies:
16.58  cmake : Depends: cmake-data (= 3.28.3-1build5) but it is not going to be installed
16.59 E: Unable to correct problems, you have held broken packages.

arm64v8/ubuntu:noble cause an  error.
I'll take a look it later.

arm64v8/ubuntu:noble cause an error.

It seems that this issue was resolved.

memo: missing ubuntu:24.04 lxc image was blocker before, check it again.

It is strange that some container images were not found...

still ubuntu:24.04 image is not published yet.

It is very weird, the container images for CentOS/7, AmazonLinux 2 and AmazonLinux 2023 - CGroup V1 platform
were not available.
Thus, from ubuntu-20.04 host, these container can't be launched.
Here is the log on ubuntu-20.04 host:
lxc image list images:
Run test: list images
  To start your first container, try: lxc launch ubuntu:20.04
  Or for a virtual machine: lxc launch ubuntu:20.04 --vm

  +-------+-------------+--------+-------------+--------------+------+------+-------------+
  | ALIAS | FINGERPRINT | PUBLIC | DESCRIPTION | ARCHITECTURE | TYPE | SIZE | UPLOAD DATE |
  +-------+-------------+--------+-------------+--------------+------+------+-------------+
  + echo ::endgroup::

Empty image lists.
In contrast to ubuntu-20.04 host, centos/7 images are listed on ubuntu-latest (CGroup V2).
(centos/7 requires cgroup v1, but it was listed on it )
  +------------------------------------------+--------------+--------+-------------------------------------------+--------------+-----------------+------------+-------------------------------+
  | centos/7 (3 more)                        | a6930d4481f8 | yes    | CentOS 7 amd64 (20240531_0019)            | x86_64       | VIRTUAL-MACHINE | 405.45MiB  | May 31, 2024 at 12:00am (UTC) |
  +------------------------------------------+--------------+--------+-------------------------------------------+--------------+-----------------+------------+-------------------------------+
  | centos/7 (3 more)                        | f4c23689f2c8 | yes    | CentOS 7 amd64 (20240531_0019)            | x86_64       | CONTAINER       | 85.45MiB   | May 31, 2024 at 12:00am (UTC) |
  +------------------------------------------+--------------+--------+-------------------------------------------+--------------+-----------------+------------+-------------------------------+
  | centos/7/cloud (1 more)                  | 061551e59b88 | yes    | CentOS 7 amd64 (20240531_0019)            | x86_64       | CONTAINER       | 99.48MiB   | May 31, 2024 at 12:00am (UTC) |
  +------------------------------------------+--------------+--------+-------------------------------------------+--------------+-----------------+------------+-------------------------------+
  | centos/7/cloud (1 more)                  | d745955c6b5c | yes    | CentOS 7 amd64 (20240531_0019)            | x86_64       | VIRTUAL-MACHINE | 417.75MiB  | May 31, 2024 at 12:00am (UTC) |

lxc image info images:centos/7 on ubuntu-latest shows:
Fingerprint: f4c23689f2c89ca2c6d6088e7fb8c02b34fb3b39700d757e3844041562c3df25
  Size: 85.45MiB
  Architecture: x86_64
  Type: container
  Public: yes
  Timestamps:
      Created: 2024/05/31 00:00 UTC
      Uploaded: 2024/05/31 00:00 UTC
      Expires: never
      Last used: never
  Properties:
      architecture: amd64
      variant: default
      os: CentOS
      release: 7
      type: squashfs
      serial: 20240531_0019
      description: CentOS 7 amd64 (20240531_0019)
      requirements.cgroup: v1
  Aliases:
      - centos/7/default
      - centos/7/default/amd64
      - centos/7
      - centos/7/amd64
  Cached: no
  Auto update: disabled
  Profiles: []

It seems that ubuntu 20.04 host lxc images seems completely broken.

As #646 was resolved, rebased with master.

Fixed now.


Elfeed fails to open entry

Cambridge nixed their RSS feeds for their journals, so I have them email journal updates to an email address which then auto-generates an RSS feed. Here's the feed link: https://zapier.com/engine/rss/1846337/alex/ (yes, I've tried to get Cambridge to send plain-text emails but they seem to ignore that preference setting.)
However, trying to open an entry from that in elfeed results in url-expand-file-name: Wrong type argument: char-or-string-p, nil.

Elfeed was mistakenly setting the base URL to "nil" which was then
confusing shr (shr should really be more graceful about it) when the
link was relative (which it's not supposed to be). Should be fixed now.
Though because, as you probably noticed, the feed is a mess, the link
Elfeed was trying to insert still won't make sense, but at least it will
display the content.

That was super fast! Much appreciated.
Yes, the feed is an absolute mess. I keep  trying to get them to send plain text in the hopes that it's better but they don't seem to honor that setting. For now, it's enough just to be notified when they publish something new.
Thanks again!


Cant install rabbitmq via kitematic

Steps to reproduce the behavior

...install rabitMQ via kitematic
...Error message "context canceled"

This problem has been solved by kitematic v0.17.7


[test] Add some negative tests to tizen_capi

There are some case where there is a positive test but no negative test
Self evaluation:

Build test:   [X]Passed [ ]Failed [ ]Skipped
Run test:     [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: Hyunil<EMAIL_ADDRESS>

:memo: TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #1905. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://ci.nnstreamer.ai/.


M1 not working

I installed nightlight from the GH repo with homebrew, bu the workflow still gives me a message saying it can't open because of unverified developer. Nightlight works on command line though.
Do you have any tips on what I can do to make this work? Am I missing something obvious?
Thanks in advance!
message when using keyword: "“nightlight” cannot be opened because the developer cannot be verified."
commands working in terminal: % nightlight toggle

Can you try out some of the stuff suggested in #5 and see if any of it fixes your problem? Thanks.

Yes! it worked, thank you very much! Apologies for not seeing the other message! Thanks for your help!


refactor(rust): refactoring common create logic around Kafka producer and consumer

Refactor(rust): refactor common create logic around Kafka producer and consumer
Current behavior
Kafka producer and consumer have a common logic to resolve a node with slight variations
https://github.com/build-trust/ockam/issues/5087
Proposed changes
Refactored to avoid the code duplication
Checks

[x] All commits in this Pull Request are signed and Verified by Github.
[x] All commits in this Pull Request follow the Ockam commit message convention.
[x] I accept the Ockam Community Code of Conduct.
[x] I have accepted the Ockam Contributor License Agreement by adding my Git/Github details in a row at the end of the CONTRIBUTORS.csv file in a separate pull request to the build-trust/ockam-contributors repository. The easiest way to do this is to edit the CONTRIBUTORS.csv file in the github web UI and create a PR, this will mark the commit as verified.

@lper1582 looks like the email you used to sign the CLA is different from the one you used to sign your commits.
I've squashed your commits into a single one, so you should git pull --rebase before pushing your changes again. If you have trouble resigning the commit, we can always close this PR and open a new one with a fresh commit. Let me know if you need any help!

@adrianbenavides Created a pr for my email changes #234. Will that work with the current PR if we re-run the workflow?

@adrianbenavides Created a pr for my email changes #234. Will that work with the current PR if we re-run the workflow?

Looks like that did the trick, thanks @lper1582 !


Kubeadm included snippets are published and indexed.

This is a...

[ ] Feature Request
[x] Bug Report

Problem:
This problem applies to several kubeadm topics, but I'll illustrate with one example.
The kubeadm init topic has some authored content along with an included snippet that is generated. The generated snippet is here.
The problem is that the snippet is getting published as a standalone topic:
https://kubernetes.io/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/
Also, the standalone snippet topic is indexed and can be found through search.
Search for this:
kubeadm site:kubernetes.io inurl:kubeadm/generated
This problem was not created by the Hugo migration. The 1.9 docs have the same problem.
https://v1-9.docs.kubernetes.io/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/
Proposed Solution:
If possible, find a way for the snippets to not be published.
If that is not possible, remove the snippets from the search index.
Page to Update:
https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-init/
https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-join/
...
https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-version/

/cc @fabriziopandini
/cc @Bradamant3
/cc @chenopis

@steveperry-53
as discussed yesterday with @Bradamant3 , we are leaning towards removing all authored content from the reference pages like the one for kubeadm init and moving that to existing pages for creating a cluster with kubeadm.
ref: https://github.com/kubernetes/kubeadm/issues/899

The problem is that the snippet is getting published as a standalone topic:
https://kubernetes.io/docs/reference/setup-tools/kubeadm/generated/kubeadm_init/

still not sure why this is happening.
/remove-lifecycle stale
/assign


firewall/logs: normal view "Act" column needs a brush-up

The X-is clickable, but counter-intuitive.

@8191 what do you think, close this then? :)

@8191 ping :)

I'm not totally convinced with the current solution (neither the "normal" nor the "dynamic" view), since it's not completely intuitive. But I cannot imaging a better solution neither... :confused:

True. Ok, let's leave this open.

Was discussed internally, need to close without further sense of direction in favour of a new ticket, as soon as a direction is proposed.


documenation for bulkimport

I've updated this in my current branch

I think this is addressed between Albert's update and my updates

updates in #671


The “Bad Request” error appears

After the successful authentication during normal work unfortunately the “Bad Request” error appears
Bad Request
Your browser sent a request that this server could not understand.
Size of a request header field exceeds server limit.
Cookie
/n

Unfortunately I have log only the error level (and not in the debug).
I do not understand why I see in the log “oidc_clean_expired_state_cookies: state has expired”
I supposed that state is used only for the authentication.
ssl_error_log.zip

We have in our deployment CGI application that deployed in Apache and Java application deployed in Tomcat.
Therefore we have separate configuration for root and for the application.
Do you see any problems with the configuration below (the full is attached):
<Directory />
    Deny from all
	Allow from localhost
    Order Deny,Allow
    AuthType openid-connect
    Require valid-user
	AllowOverride None
    Satisfy any
    Options None FollowSymLinks
</Directory>
ProxyPass        /application1 http://localhost:8080/application1
ProxyPassReverse /application1 http://localhost:8080/application1

<Location /application1>
    Deny from all
    Order Deny,Allow
    AuthType openid-connect
    Require valid-user
    Satisfy any
    Options None FollowSymLinks
     RequestHeader set X-Remote-User  %{OIDC_CLAIM_preferred_username}e
</Location>

conf.zip

I don't necessarily see a problem with that config but it may depend on the client. Most often one would see state cookies pile up if there's a large number of parallel unauthenticated requests coming from a Javascript client.

Any update on this?

We have reproduced the same problem in UI but we have different error in log.
[Wed Mar 29 13:19:20 2017] [debug] protocol.c(705): [client <IP_ADDRESS>] (28)No space left on device: Failed to read request header line Cookie: KC_RESTART=eyJhbGciOiJIUzI1NiIsImtpZCIgOiAiZGJkNjRlOWI
tYzcwOC00ZGI5LThlOWYtZGZkNTU2ZTE2NzEyIn0….

Please find the attached log in the debug level.
We will happy for the clarification regarding the RP behavior:
How RP consider these configurations?
As the same applications?
Do we have same tokens for both configurations?
I mean if client access “application1” URLs will it affect the token on the root configuration?
<Directory />

    AuthType openid-connect

</Directory>

ProxyPass        /application1 http://localhost:8080/application1
ProxyPassReverse /application1 http://localhost:8080/application1
<Location /application1>

    AuthType openid-connect

</Location>

ssl_error_log.zip

The log shows that after the user logs out, a number of requests comes in to Javascript resources:
/application1/javax.faces.resource/richfaces-event.js.faces
/application1/javax.faces.resource/tufin/ui/js/blank.js.faces
/application1/js/modal.js
/application1/js/jquery.blockUI.js
/application1/js/jquery.jscrollpane.min.js
/application1/js/jcarousel.min.js
/application1/js/jtooltip.min.js
/application1/js/tufin.js?build=0
/application1/js/tufin.unsaved.js?build=0
/application1/javax.faces.resource/tree.js.faces?ln=org.richfaces

each of these requests generates a redirect to the Provider with a unique state cookie, hence the Cookie header limit is exceeded. Maybe those resources are loaded from the loggedout page, maybe there's another tab open that keeps polling those resources.
So either you should reorganize your Javascript code so it does not load those resources after logout, or you can use OIDCUnAuthAction 401 on the /application1/js and /application/javax* paths, see https://github.com/pingidentity/mod_auth_openidc/wiki/Cookies#cookie-exceeds-size-limit to prevent those redirects to happen.

Thanks,
How RP consider our configurations?
As the same applications?
Do we have same tokens for both configurations?
I mean if client access “application1” URLs will it affect the token on the root configuration?

yes, there's only one session, created by a single login

We will happy for couple of clarifications related to https://github.com/pingidentity/mod_auth_openidc/wiki/Cookies#tldr

If we use file storage (OIDCCacheType file) should we apply X-Requested-With: XMLHttpRequest?
Can we apply OIDCUnAuthAction 401 using LocationMatch like in the example below?

<LocationMatch "\.(js)$">
 OIDCUnAuthAction 401
</LocationMatch

the cache/storage type is independent of the interpretation of theX-Requested-With: XMLHttpRequest header
yes, but I believe the order of Location/LocationMatch directives matters

Thanks,
I guess OIDCUnAuthAction 401 should be after AuthType openid-connect.
Correct?

Unfortunately, the problem is back again.
We have added to the configuration the following:
<LocationMatch "\.(js|faces)$">
 OIDCUnAuthAction 401
</LocationMatch>

What we missed in our configuration?
The logs are attached.
ssl_error_log.zip


Make the factories act like factories

Instead of instantiating all plugins up front, instantiate them on demand.
Fixes #467

Looks like the events factory actually does want singletons and not a true factory...

Codecov Report

Merging #474 (7cb1cea) into main (a3992b8) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #474   +/-   ##
=======================================
  Coverage   99.92%   99.92%
=======================================
  Files         266      266
  Lines       15134    15134
=======================================
  Hits        15123    15123
  Misses         11       11

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a3992b8...7cb1cea. Read the comment docs.


Manifest parse failure

On the latest build of swift (downloaded via the python script), with a full uninstall and reinstall, the following error is occurring:
C:\SourceCache\redacted\desktop_client\shared\redacted >swift build
C:\SourceCache\AutoML\ redacted\shared\redacted: error: manifest parse error(s):
<unknown>:0: error: compiled module was created by an older version of the compiler; rebuild 'Swift' and try again: C:\Library\Developer\Platforms\Windows.platform\Developer\SDKs\Windows.sdk\usr\lib\swift\windows\Swift.swiftmodule\x86_64-unknown-windows-msvc.swiftmodule

C:\SourceCache\redacted\desktop_client\shared\redacted >

That is very confusing.  The VS2019 builds use the just built Swift.  It seems like the SDK and the toolchain are out of sync.  Could you provide the build number that you used?  (I am trying to get a newer snapshot built, so I'm not 100% certain if I have something stupid in the script or if there is an issue in the build or the installer packaging).  I guess this is a good reason to try to figure out how to embed the version information into the build itself.

C:\SourceCache\AutoML\desktop_client\shared\automl_core>swift --version
compnerd.org Swift version 5.3-dev (LLVM 8d6dfcd39f3bc7d, Swift 410b88ef07a1d18)
Target: x86_64-unknown-windows-msvc

Yeah, I really need to get the build number embedded into that.  Okay, for the mean time, the URL would be handy I guess :-(.

How do I actually find that? I just used the downloader script and have since closed my command prompt. I'm a macOS guy, sorry haha. I ran the python script about 10 minutes before posting this bug if that helps. If necessary I can also just upload the install exe somewhere like Google drive.

Just drop the --download flag and it will print the URL :)

The problem is the installer has most likely since changed because the CI has run since then and I was using the --latest-artifacts flag. I uploaded the installer to Google Drive. If the artifact link from the latest flag is at all helpful, it's this

So I just ran the latest installer (the one that I posted in that last comment) and running swift build doesn't crash with the manifest parse issue, but rather hangs. Should I file a separate issue for that?

I think that there's something stale left or the support files are out of date.  Could you re-copy the support files and remove the .build directory and then run swift build again?

Hmm yeah I did both of those and it's still hanging

That is very interesting.  I think that we should split that off into a separate issue.  I don't think that this is really the right avenue for the s-p-m issue - I think that the bugs.swift.org is probably better.  In any case, I've heard of one other case where the builder hangs, but I've not been able to figure out where the race condition it (I suspect that the process creation is going wrong and we aren't catching the error).  It is unclear whether it is in s-p-m or llbuild.  Would you mind running procmon and catching the CreateProcess calls and seeing if any of them are failing?

Here's the procmon logfile in the PML format. I'm not a Windows guy so I was probably filtering wrong, but there didn't seem to be any CreateProcess calls (I doubt that's true but which is why I'm guessing I did it wrong)
Logfile.zip

I see a number of calls to swiftc being made (which is normal during the startup process).  If you run with -v you should see those invocations.  What is interesting is that a file is being written for the stdout capture to redirect stdout, which is odd as I don't expect that.  But, I don't really see why the process would hang.

Running with -v gave me:
C:\SourceCache\AutoML\desktop_client\shared\automl_core>swift build -v
C:\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr\bin\swiftc.exe -print-target-info

I wonder if there is something waiting on the -print-target-info somehow.  Being able to get a reproduction would be rather useful, it seems like there is something in the s-p-m side that is racing in some cases (e.g. with the other instance, running the swift build procmon actually caused the problem to disappear).

Attached is an example project that I was able to have the same issue occur on. Tested on this build
reproduction.zip

So running swift build with the release version posted to the blog gives me this:
C:\SourceCache\redacted\desktop_client\shared\redacted>swift build
error: unable to invoke subcommand: C:\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr\bin\swift-build ()

I did a full uninstall and reinstall. Additionally, with the release, the REPL just immediately terminates with no output

That is 100% expected.  swift-package-manager is not supported in the 5.3 release, so that wont work.  It has to be from the snapshots.  The REPL issue on the release is a known thing, there was a missed patch.  Unfortunately, I found some follow up issues due to the layout with LLDB but those can be worked around.

I tried the reproduction that you posted - please note that this is not going to work off the bat - it is using a number of projects which aren't fully ported or at all (e.g. vapor).  That said, it doesn't hang for me, I've attached a log of what I saw with swift build.
log.txt

Note that this is largely what I expected - the problem isnt the package but rather the environment itself - there is some race condition in spm or llbuild that is getting exposed, and just needs to be tracked down.  Unfortunately, that isn't really possible to diagnose without a trace of what exactly is going on.

Ok so something was just actually ruined on this machine environment. It got to the point that the C compiler wasn't working, let alone swift. I just wiped windows, installed a new copy of VS, Swift, git, etc. There are still dependency  issues  with the included packages, but running swift build -Xlinker /INCREMENTAL:NO -v did at least work

Just reference to the same problem i met, thanks.
https://github.com/compnerd/swift-win32/issues/327#issuecomment-747231939


Add support for 128-bit primitives

Rust 1.26 is stabilizing i128 and u128.  We should add support for for these, to the same extent that other primitive types are supported.  This should be a conditional feature, rather than forcing all of num-bigint up to the very latest compiler -- e.g. rust-num/num-traits#60 has a new feature "i128".

[x] convert to and from 128-bit integers #44
[ ] implement operators with 128-bit operands
[ ] implement BigDigit = u64 with DoubleBigDigit = u128

still conditional on new enough rustc, and probably only for specific target CPUs

I will likely work on BigDigit = u64 in the next days, as I have seen significant speed increases when using this in other code.

FWIW I did this on a local branch (before finding this issue) and I noticed non-negligible performance improvements at least for modpow which is what I was benchmarking. It would be nice if it detected the native word size to automatically pick the type for BigDigit (maybe using target_pointer_width?).

How about the division benchmarks?  That's the main thing that worries me with DoubleBigDigit = u128 -- from what I've seen, LLVM never optimizes this to x86's pseudo-128-bit DIV with RDX:RAX.  There are division tests in benches/bigint.rs and benches/factorial.rs, so I'm guessing it's those factorial_div_* ones where you noticed a slowdown.
I'll want to see full benchmarks before we merge anything, not just cherry-picked impressive ones, but we'll get to that in your PRs.

Wild idea: It might be possible to take a hybrid approach, using u64 and u128 for most operations, but casting the digits down to [u32] (with big-endian fixups) for the raw div_rem_digit and div_wide.

This is the current state: https://gist.github.com/dignifiedquire/b7d442bea7bc2490bffd32ae31081f8 I have on my machine.
I am actually seeing speed ups in the division benchmarks, as far as I can tell.
I think hybrid could work, but I am not convinced it is needed/actually faster on supported architectures.

Your gist link is 404 from here.
The benefits to other operations involved in the division algorithm might outweigh the slowdown I'm expecting in raw u128 division.  The single-digit divisors in factorial_div_* will probably be more sensitive to that low-level performance.

ah yes factorial_div are the ones with the slow down, this will need some work to figure out to ensure no regression happens there
link should be fixed now


Terminal result can't be seen, but have response.

OS/Web Information

Web Browser: safari 15.0
Local OS: macOS big sur 11.6
Remote OS: Debian 10.2
Remote Architecture: amd64
code-server --version: 3.12.0

Steps to Reproduce
1.open terminal in code-server web page.
2.type any cmd.
Expected
Show cmd result.
Actual
Terminal responded, but can't be seen.Seem like transparent font color.
Logs
➜  ~ code-server --log debug
[2021-09-28T09:11:51.597Z] info  code-server 3.12.0 b37ff28a0a582aee84a8f961755d0cb40a4081db
[2021-09-28T09:11:51.598Z] info  Using user-data-dir ~/.local/share/code-server
[2021-09-28T09:11:51.615Z] info  Using config file ~/.config/code-server/config.yaml
[2021-09-28T09:11:51.615Z] info  HTTP server listening on http://<IP_ADDRESS>:8080
[2021-09-28T09:11:51.615Z] info    - Authentication is enabled
[2021-09-28T09:11:51.615Z] info      - Using password from ~/.config/code-server/config.yaml
[2021-09-28T09:11:51.615Z] info    - Not serving HTTPS
[2021-09-28T09:11:54.765Z] error vscode is not running Error: vscode is not running
    at VscodeProvider.send (/usr/lib/code-server/out/node/vscode.js:121:19)
    at VscodeProvider.sendWebsocket (/usr/lib/code-server/out/node/vscode.js:117:14)
    at async /usr/lib/code-server/out/node/routes/vscode.js:205:5
[2021-09-28T09:11:54.767Z] error vscode is not running Error: vscode is not running
    at VscodeProvider.send (/usr/lib/code-server/out/node/vscode.js:121:19)
    at VscodeProvider.sendWebsocket (/usr/lib/code-server/out/node/vscode.js:117:14)
    at async /usr/lib/code-server/out/node/routes/vscode.js:205:5
[2021-09-28T09:11:55.180Z] debug forking vs code...
[2021-09-28T09:11:55.463Z] debug setting up vs code...
[2021-09-28T09:11:55.467Z] debug vscode got message from code-server {"type":"init"}
[2021-09-28T09:11:56.130Z] debug vscode got message from code-server {"type":"socket"}
[2021-09-28T09:11:56.132Z] debug protocol Initiating handshake... {"token":"008ffa59-ba67-45af-ac75-7a5b12eb3a9a"}
[2021-09-28T09:11:56.179Z] debug protocol Handshake completed {"token":"008ffa59-ba67-45af-ac75-7a5b12eb3a9a"}
[2021-09-28T09:11:56.180Z] debug management Connecting... {"token":"008ffa59-ba67-45af-ac75-7a5b12eb3a9a"}
[2021-09-28T09:11:56.180Z] debug vscode 1 active management connection(s)
[2021-09-28T09:11:56.598Z] debug got latest version {"latest":"3.12.0"}
[2021-09-28T09:11:56.598Z] debug comparing versions {"current":"3.12.0","latest":"3.12.0"}
[2021-09-28T09:11:56.697Z] debug vscode got message from code-server {"type":"socket"}
[2021-09-28T09:11:56.698Z] debug protocol Initiating handshake... {"token":"7f29da65-23a4-431b-878a-0a064eb99e19"}
[2021-09-28T09:11:56.751Z] debug protocol Handshake completed {"token":"7f29da65-23a4-431b-878a-0a064eb99e19"}
[2021-09-28T09:11:56.752Z] debug exthost Connecting... {"token":"7f29da65-23a4-431b-878a-0a064eb99e19"}
[2021-09-28T09:11:56.752Z] debug exthost Getting NLS configuration... {"token":"7f29da65-23a4-431b-878a-0a064eb99e19"}
[2021-09-28T09:11:56.752Z] debug vscode 1 active exthost connection(s)
[2021-09-28T09:11:56.752Z] debug exthost Spawning extension host... {"token":"7f29da65-23a4-431b-878a-0a064eb99e19"}
[2021-09-28T09:11:56.756Z] debug exthost Waiting for handshake... {"token":"7f29da65-23a4-431b-878a-0a064eb99e19"}
[2021-09-28T09:11:57.275Z] debug exthost Handshake completed {"token":"7f29da65-23a4-431b-878a-0a064eb99e19"}
[2021-09-28T09:11:57.275Z] debug exthost Sending socket {"token":"7f29da65-23a4-431b-878a-0a064eb99e19"}
[2021-09-28T09:12:54.753Z] debug 2 active connections

Screenshot
Safari
https://user-images.githubusercontent.com/10864310/135058885-8c923626-a4b0-4892-bf98-9f4bb0a24304.mov
Chrome

Notes
Although Safari will have problems, Chrome will not, but I still hope this problem can be fixed.
This issue can be reproduced in VS Code: Yes
Thank you for your effort.

Although Safari will have problems, Chrome will not, but I still hope this problem can be fixed.
Thank you for your effort.

Thanks for reporting! This is a known issue which can be solved by turning of GPU acceleration for the terminal in settings. See #3824 for details.
Let us know if you run into any more issues running on iPad, we’re hoping to fix most of these soon. We also have a page dedicated to the iPad where we include guides and known issues.


SBT warning

SBT project import
[warn] Unable to reparse com.github.etaty#rediscala_2.11;1.8.0-SNAPSHOT from sonatype-snapshots, using Sun Nov 20 18:06:02 EAT 2016
what is that and how it can be fixed?
scalaVersion := "2.11.8"
sbt.version=0.13.13

I have no idea.
I have released 1.8.0 yesterday
Could you try again with?
libraryDependencies += "com.github.etaty" %% "rediscala" % "1.8.0"

@etaty no warnings with new version. Thank you! I think this issue can be closed.


executor_supervisor: fix Mix env check

This was a funny one to debug

Pull Request Test Coverage Report for Build 82a0cbb2934523c477ada2f64c93b9dbbaa88f8c-PR-327

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 83.482%

Totals

Change from base Build b20cd04b2a44cb21df35e98f323fef91e4368dbf:
0.0%

Covered Lines:
1400

Relevant Lines:
1677

💛  - Coveralls


cutting

it cuts before it finishes print what it is going to write.

This should be fixed in the upcoming version 3.
It includes a way to configure the number of lines you want to feed before cutting. And it has default values for a whole bunch of populair printers.


context construction relies on environment variables present in process

It would be ideal if the value could be pulled from the .env file if present otherwise try to read from the process.env to allow users who don't dump variables into process space the ability to get a twilio client object.
https://github.com/dkundel/twilio-run/blob/9a1a19be961647f1dcb2761be869d2611cbc391e/src/route.js#L18

This was a misunderstanding on my part.

You should be using context.getTwilioClient() for that behavior actually :)
Cheers,
Dominik