internal/poll: data race in (*FD).SetBlocking

What version of Go are you using (go version)?
go version go1.7.6 linux/amd64
go version go1.8.3 linux/amd64
go version go1.9.2 linux/amd64
go version go1.10 linux/amd64
go version devel +7974f0815e Mon Mar 19 21:51:23 2018 +0000 linux/amd64
Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (go env)?
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/twmb/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/twmb/go"
GORACE=""
GOROOT="/home/twmb/go/go"
GOTMPDIR=""
GOTOOLDIR="/home/twmb/go/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build795784354=/tmp/go-build -gno-record-gcc-switches"

What did you do?
https://play.golang.org/p/WS8xLLK7EcX
What did you expect to see?
starting
done

What did you see instead?
starting
==================
WARNING: DATA RACE
Read at 0x00c420112c90 by goroutine 73:
  net.(*netFD).dup()
      /home/twmb/go/go/src/net/fd_unix.go:307 +0x40
  net.(*conn).File()
      /home/twmb/go/go/src/net/net.go:294 +0x4d
  main.main.func2()
      /home/twmb/testing/junk.go:29 +0x3a

Previous write at 0x00c420112c90 by goroutine 75:
  internal/poll.(*FD).destroy()
      /home/twmb/go/go/src/internal/poll/fd_unix.go:72 +0xa4
  internal/poll.(*FD).decref()
      /home/twmb/go/go/src/internal/poll/fd_mutex.go:211 +0x55
  internal/poll.(*FD).Close()
      /home/twmb/go/go/src/internal/poll/fd_unix.go:93 +0x6b
  net.(*netFD).Close()
      /home/twmb/go/go/src/net/fd_unix.go:184 +0x60
  net.(*conn).Close()
      /home/twmb/go/go/src/net/net.go:200 +0x60
  net.(*TCPConn).Close()
      <autogenerated>:1 +0x43
  main.main.func3()
      /home/twmb/testing/junk.go:35 +0x3e

Goroutine 73 (running) created at:
  main.main()
      /home/twmb/testing/junk.go:28 +0x13f

Goroutine 75 (finished) created at:
  main.main()
      /home/twmb/testing/junk.go:34 +0x16b
==================
==================
WARNING: DATA RACE
Read at 0x00c4204078af by goroutine 296:
  internal/poll.(*FD).Close()
      /home/twmb/go/go/src/internal/poll/fd_unix.go:99 +0x91
  net.(*netFD).Close()
      /home/twmb/go/go/src/net/fd_unix.go:184 +0x60
  net.(*conn).Close()
      /home/twmb/go/go/src/net/net.go:200 +0x60
  net.(*TCPConn).Close()
      <autogenerated>:1 +0x43
  main.main.func3()
      /home/twmb/testing/junk.go:35 +0x3e

Previous write at 0x00c4204078af by goroutine 263:
  internal/poll.(*FD).SetBlocking()
      /home/twmb/go/go/src/internal/poll/fd_unix.go:121 +0x8f
  net.(*netFD).dup()
      /home/twmb/go/go/src/net/fd_unix.go:316 +0x7d
  net.(*conn).File()
      /home/twmb/go/go/src/net/net.go:294 +0x4d
  main.main.func2()
      /home/twmb/testing/junk.go:29 +0x3a

Goroutine 296 (running) created at:
  main.main()
      /home/twmb/testing/junk.go:34 +0x16b

Goroutine 263 (running) created at:
  main.main()
      /home/twmb/testing/junk.go:28 +0x13f
==================
done

This is different from https://github.com/golang/go/issues/24481 because one of the two races is present back in old versions of Go as well. The first (destroy + dup) is old. The second is likely from 0f3ab149ec4.

net.Conn methods are explicitly safe for concurrent use, but File is not strictly a method of net.Conn.
Ideally File should be safe for concurrent use too, but it's not obvious to me what kind of thorny side-effects that would bring.

(CC @ianlancetaylor for polling internals.)

As an aside, https://github.com/golang/go/issues/24481 is more specific to SetBlocking being racy. This issue has an additional race on *FD reference counting between a simultaneous File and Close.

Ah, ok.

Let's use #24481 for SetBlocking and focus this one on the reference counting:
==================
WARNING: DATA RACE
Read at 0x00c420112c90 by goroutine 73:
  net.(*netFD).dup()
      /home/twmb/go/go/src/net/fd_unix.go:307 +0x40
  net.(*conn).File()
      /home/twmb/go/go/src/net/net.go:294 +0x4d
  main.main.func2()
      /home/twmb/testing/junk.go:29 +0x3a

Previous write at 0x00c420112c90 by goroutine 75:
  internal/poll.(*FD).destroy()
      /home/twmb/go/go/src/internal/poll/fd_unix.go:72 +0xa4
  internal/poll.(*FD).decref()
      /home/twmb/go/go/src/internal/poll/fd_mutex.go:211 +0x55
  internal/poll.(*FD).Close()
      /home/twmb/go/go/src/internal/poll/fd_unix.go:93 +0x6b
  net.(*netFD).Close()
      /home/twmb/go/go/src/net/fd_unix.go:184 +0x60
  net.(*conn).Close()
      /home/twmb/go/go/src/net/net.go:200 +0x60
  net.(*TCPConn).Close()
      <autogenerated>:1 +0x43
  main.main.func3()
      /home/twmb/testing/junk.go:35 +0x3e

Goroutine 73 (running) created at:
  main.main()
      /home/twmb/testing/junk.go:28 +0x13f

Goroutine 75 (finished) created at:
  main.main()
      /home/twmb/testing/junk.go:34 +0x16b
==================

This is apparently a race between the File method and the Close method.  I'm likely missing something but I don't see why a real program would have such a race.  This is not the same as a race between Read and Close, where Read blocks for some period of time and it might be reasonable for the program to call Closewhile some other goroutine is blocked inRead.  Neither FilenorClose` block, so there is no good reason to call them concurrently.
That said this is easy enough to fix if we think it's worth fixing.  We just need to copy the dup method from net/fd_unix.go to internal/poll/fd_unix.go, change the net one to call the new one, and have the new one call incref/decref, like Shutdown or SetBlocking.

For realistic context, we have a program and we want to check the tcp queue depth of long lived connections occasionally. We will be updating it to use SyscallConn, but historically, we had to use File to get the file descriptor that we could run sys_ioctl on through syscall. Rather than keeping around some mysterious *os.File, we called .File on the connection whenever we wanted to check the depth. It's lifetime was managed concurrently, though, and this race could happen.
I would be satisfied if documentation warned of the issue -- fixing this in the runtime guts is not strictly necessary.

@ianlancetaylor says that this will be solved by #24481, which is fine. Note that this wouldn't otherwise be worth fixing necessarily: race between File and Close seems worth reporting, since the code is profoundly confused. If we can't report it, oh well.


Using Fine-tuning to train the model on a new database

Hello!
@githubharald
Please can you guide me how can I use fine-tuning for this project?
Thanks in advance!

sorry but i really don't have the time to provide personal guidance.
this repo is provided as it is, if you need more features, you will have to implement them yourself and go through the code yourself.


dataset production

hi,I am sorry to bother you, but i want to kown how do you make a data set? nsfw set maybe easy to classify,but the types of sfw images are too complex and huge, you can't include all the categories of sfw image.

This project does not use a training dataset. I just use the pretrained NSFW detector and generate adversarial images.


Create sudofun.toml

Description
All previous genesis validators should name their PRs "Update {validator_alias}.toml for tesntet-15" and provide links to previous PRs merged.
If this is an UPDATE for a previous genesis validator
Checks are made against your net_address. If this has changed since the previous testnet, make sure you provide links of previous prs merged from your previous git username.
Thanks in advance!
Checklist before merging

[ ] Only one toml is added in this PR
[ ] The file being added is indeed a .toml file
[ ] The toml being added is to the correct folder, and only to the correct folder
[ ] The eth_hot_key and eth_cold_key are present
[ ] The email, discord, elements telegram, and twitter fields are present and valid

The transactions.toml file for 651823c was deemed invalid with error Invalid signatures: Signatures could not be verified

The transactions.toml file for 651823c was deemed invalid with error Invalid signatures: Signatures could not be verified

The transactions.toml file for 651823c was deemed invalid with error Invalid signatures: Signatures could not be verified

The transactions.toml file for 651823c was deemed invalid with error Invalid signatures: Signatures could not be verified

The transactions.toml file for 651823c was deemed invalid with error Invalid signatures: Signatures could not be verified

The transactions.toml file for 651823c was deemed invalid with error Invalid signatures: Signatures could not be verified

The transactions.toml file for 651823c was deemed invalid with error Invalid signatures: Signatures could not be verified


Highlight matched search terms everywhere

Fix #259

Coverage increased (+0.002%) to 99.597% when pulling 455b07cc4312da682a6fdd16c22feec895cea24b on highlight into fcee5c5df195543e83cb9a879dc47a2a27ff9cb1 on master.


media.playVideo listener does not fire

[ x] I tried searching Solar2D forums for the solution.

Describe the bug
media.playerVideo's listener is not called.
To Reproduce
Steps to reproduce the behavior:

Invoke media.play("video file", function() print ("Done") end)

Target platform and where build was made:

Device: Corona Simulator 2021.3642, iPhone XS
OS: 10.15.7

Additional context
I have tried both mp4 and mov files. The videos play fine, but the callback never seems to fire
whole test project or build.settings
media.play.callback.zip

Looking at the syntax, I see showControls is required in the call. Is it possible that's causing the issue?
media.playVideo( "movie.m4v", true, onComplete )

Does it work with it? :)

@sekodev You are right! I missed that when simplifying from my original code. Reviewing the code I can see the actual issue I had was that the listener does not fire if the baseSource is set to system.ResourcesDirectory. In my case that was in there because I was thoughtlessly changing where the videos were loaded from. I guess it might still be worth investigating as it could cause issues when programmatically switching were a video is loaded from

Going to go ahead and close this issue, since the docs state that showControls is required and
media.playVideo("small.mp4",false, function() 	print ("Done") 	display.newCircle(10,10,1000) end)
or
media.playVideo("small.mp4",true, function() 	print ("Done") 	display.newCircle(10,10,1000) end)
both seem to work :)


Allow tabs for indentation in text edit widget

Hi,
it seems like the Tab key is not captured in the text edit widget, could that be ?
Like, when I put some debug output in the event handling loop, it seems like there's no event emitted at all ... just the outer frame flickers ...
Is there a way to capture the tab key ?
Best,
N

Egui uses the tab key to move focus between TextEdit widgets. You can try it in the web demo by clicking the "Hello World!" text edit field and pressing tab. The focus will move to the next field.
What do are you trying to accomplish? Do you have a text edit field where you would like the Tab key to insert a \t char instead of moving focus?

Exactly that ... I use a customized TextEdit widget for a (toy) code editor ... having the tab key would be somewhat essential for that ...

Hello. I just created this draft PR #246.
How should I render the TAB?

render 4 spaced instead of TAB
render a quad that occupies the spaces of 4 regular spaces?

I'm still investigating how the font rendering is done, but I decided to create this draft PR to be able to get some early reviews, suggestions and hints.

I kept hammering at the tabs as space idea, where a TAB is replaced by 4 Space chars, but I have two problems and I don't know how to fix one of them.
With this commit, https://github.com/emilk/egui/pull/246/commits/94c180179c49ed678536915d7af46f54eb5988bb, I took the approach of keeping TAB in the text string, but when we compute the TextEdit::content_ui I replace all tabs by 4 Space chars. I do this because I wanted to handle the case when the TextEditis initialized with a string that already contains TAB. Later I noticed that I actually mutate the original string. This means the TABs from the original string are gone and I'm not sure if this is what we want. For this issue I'm looking into replacing the TABs with Spaces without modifying the original string when we create a multi line TextEdit or when we render it in TextEdit::content_ui. I hope it makes sense what I'm saying, if not let me know and I will try to reword it.
Now onto the problem I have no idea how to fix. When I detect a TAB key I insert it in the string and then I advance the cursor, but the advancement of the cursor works only if I press TAB in the middle of the string. If I press it at the end it will render 4 Space chars, but it will not advance the cursor.
After work I will play a bit more with it.

I'm back the only thing left to do is to actually render TAB as a "space" with the size of n spaces, but I think I need to do some API breaking changes to the font.rs because Rust does not allow default values for parameters.
This commit has more details: https://github.com/emilk/egui/pull/246/commits/ba12f9e81f044e37d81e54376074cc8aa35e7a72

The simplest approach is to always use the \t tab character, and simply say one tab = 4 spaces. I just pushed a commit that does that: https://github.com/emilk/egui/commit/cbe6faa83be8d73e760d643656c7a90e39dda282. With that there is no need for any special handling in labels and TextEdit.
Then the only thing left is handling the focus issue. I think your lock_focus method is good, but there has to be a way to break out of the lock. In VS Code I notice that I can move the focus with option+tab (on Mac) even when in a text field. On Windows maybe that's ctrl-tab ?

That moment when I am... damn that's so easy :). I will rebase with your changes and update the PR.
For me on Fedora, but on Windows too, alt+tab switches between apps. From my knowledge option key is the same as the alt key.
On Fedora, ctrl+tab switches between VS Code tabs, forward and ctrl+shift+tab goes trough the editor tabs backwards and my guess on Windows is doing the same thing.
I'll try to figure out how it is done.

Found it for VS Code, ctrl+M activates a "tab moves focus" mode.
Should me come up with a similar convention?

Maybe we can keep it simple for now.
I think the "tab-as-character" behavior should be opt-in, i.e. TextEdit::multiline(text).code_editor(true) or something. (this could maybe also turn on monospace and other code-like behavior).
In most cases I think we would want tab/shift-tab to move focus even for multiline text (as in: most multiline text field do not need tab characters), so the locking-in is much less of an issue. A user can always press ESC to escape the locked in editing.

@emilk, with your help and suggestions I think I have the final version of the PR #246. When you have time to review it please do so. I'm really excited about this infinitesimal small contribution I did. :smile:

A small update. For the shift+tab shortcut you suggested I finished it for when the editor is in tab-as-spaces mode.
Later today I will finish when the editor is in tab-as-tabs mode. I have to say I used VS Code as inspiration on how to render the tabs.
Just a remark. I've noticed there is some alignment issues. I'll try to figure out at the end, but if you suspect anything let me know.

And the detailed view from VS Code.

When I render a tab I'm following your suggestion of getting the GlyphInfo for space and then doing a glyph_info.advance_width *= tab_size computation. I suspect the alignment issue comes from the fact that when we have separate space chars there is some space inserted between the characters.

My PR is ready for review again. I'm curious what do you think.


Configure container security context to restrictive defaults

Purpose

$subject
Support OpenShift 4.x

Codecov Report
Patch coverage has no change and project coverage change: +1.69 :tada:

Comparison is base (ee87a4a) 25.10% compared to head (04121bd) 26.79%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1394      +/-   ##
==========================================
+ Coverage   25.10%   26.79%   +1.69%
==========================================
  Files         258       63     -195
  Lines       29908    10684   -19224
  Branches     4828      246    -4582
==========================================
- Hits         7508     2863    -4645
+ Misses      22134     7688   -14446
+ Partials      266      133     -133

Flag
Coverage Δ

adapter
23.91% <ø> (ø)

config-deployer-service
?

idp-domain-service
80.10% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
see 195 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Handle closing modal

In the latest semantic ui update, closeModal is deprecated. Checking the code, i found this
// if controller handles approves, they must manually call view.execute('hide')

How do i access view from controller, and run execute on it. AFAIK, ember only works in one direction. You can only bubble upward, but you can't call view nor send it events. Is there something I'm missing here?

If you need to override a method for the modals standard approve or deny, you can create an action to represent that.
actions:
   approve: (view) ->
       # execute your code
       # when finished
       view.execute('hide')

That makes sense, but what about actions outside approve and cancel scope? In my case I have an extra button that should delete an object, then close modal. Is there any way to handle such thing?

If you look at the code for cancel on the semantic model, once it starts closing you can't stop it like you can with approve and deny. We do fire a hide action you can plug into to do some clean up, but you can't stop the action it's self.
To manually close outside of approve and deny just pass the view in from the template.

Nevermind

then you still have access to the view, which has the semantic modal mixin and the execute function.

One more comment. The modal is unique compared to other semantic modules. Usually we can just dispose of the html and not worry about it, but with modal we have ensure we go through semantic first and then through ember so semantic properly executes its code and visually looks correct. The only way to do that is by first executing the events on semantic and then once it's callbacks are finished, then dispose of the outlet. It's a little awkward in the traditional ember workflow, but it was the only way we could find to probably execute.

Awesome. It worked! Thanks a lot :)


Cocoapods support

First of all, great work!! Greatly appreciate you guys making flare available for swift! Do you guys intend to support cocoapods I. The future? Couldn’t find anything. Would really streamline the process of integrating flare into the projects. Thanks in advance for your answer and keep up the good work :)

Hey @kalmicka, thanks for the support!
Right now we're trying to reach feature-parity with our other Runtimes, so we should have a big update with support for meshes, deformations, blend modes and more, in the near future.
We surely intend to add support for CocoaPods, as well as Carthage and possibly SPM, as @artrmz also suggested!

+1 for swift package manager. It's the future!

Mhm, seems like there is no point so support anything besides SPM since it will be built-in into new Xcode.

Hello when to support SPM？


docs: removed duplicate in cancelNotification API

Deleted "removes removes" duplicate in cancelNotification API
The cancelNotification API removes removes any displayed notifications... to The cancelNotification API removes any displayed notifications...

Related issue: https://github.com/invertase/notifee/issues/411

Thank you!

You're welcome @mikehardy ^^


Fullscreen doc instructions out of date?

jsPsych/docs/markdown_docs/features/fullscreen.md
Should the example code be updated to include "timeline" parameter instead of "experiment_structure" within jspsych-init?

Thanks for catching this.


Do not add precompiled bridging header to a Compile job's display inputs

"Display inputs" are used to convey to the user/client the set of input files that a given compile job will build, e.g. in parseable-output.
The driver is relying on the presence of the bridging PCH in displayInputs because the printActions code relies on displayInputs, when they are present. Unlike parseable-output, action printing is used for compiler debugging purposes only, and given that printActions is a best-effort emulation of the legacy driver behavior that doesn't really exist in the new driver, we can just as well get away with using primaryInputs and ensure compile job actions refer to the corresponding precompiled header, to keep the C++ tests passing (test/Driver/actions.swift & test/Driver/actions-dsym.swift).

@swift-ci please test


Incorrect UID read with Wiegand 32 bit device - anyone know protocol

We currently use a 522 reader and have been donated several Wiegand devices.
These output 32 bits when a card is read.
The displayed UID is totally different to the UID stored on the card.
I have investigated the code and have a fix but this is to the Wiegand.cpp library file not the esp-rfid code.
If we had not already been using a 522 reader this issue would never have arisen.
So my question is, does anyone have a reference to the Wiegand 32 bit protocol? I can't find one.
I am going to post a similar question in the github repository for the code.

Try to check if D0 and D1 are inverted.

They are okay, I originally had them inverted on my logic analyser. When I work through manually the errors occur when the last bit is read in.
Sent from my iPhone

On 18 Nov 2022, at 10:28, Enrico Vazzoler @.***> wrote:

Try to check if D0 and D1 are inverted.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.

hey @windy54 do you confirm this is something to do with the Wiegand library? Am I right?
I'm going to close this for now, but if you still need help on this one let me know. Maybe can you provide a couple of examples of RFID reads from the two sensors for the same card?

Hi,Yes I have edited the wiegand.cpp file, I will send the edits to you.CheersSteve GaleOn 23 Jan 2023, at 20:39, Matteo Giaccone @.***> wrote:
hey @windy54 do you confirm this is something to do with the Wiegand library? Am I right?
I'm going to close this for now, but if you still need help on this one let me know. Maybe can you provide a couple of examples of RFID reads from the two sensors for the same card?
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>


Allow styled display (again) in edit:complex-candidate

At some point the &style option to edit:complex-candidate was deleted, and the &display option does not allow a ui:text object (the output of styled) as its value. It would be nice to either have the &style option back or (preferably) allow ui:text objects as the value to the &display option.

Quite desirable as colored descriptions (similar to fish) is a bit more readable.

Gentle ping on this bug. I just noticed that in fact &display is documented as accepting a styled object, even though it does not (https://elv.sh/ref/edit.html#editcomplex-candidate):

The &display option controls how the candidate is shown in the UI. It can be a string or a styled text. If it is empty, $stem is used.

@xiaq nice!  works fine but is there a way to interpret strings instead of splitting things up like this?

Thanks for this! Now the comp:decorate function supports the &style argument again :) https://github.com/zzamboni/elvish-completions/commit/8e27f28b183c7b2ced1a2fe01e5599b6f5050fb8

@xiaq nice! works fine but is there a way to interpret strings instead of splitting things up like this?

Do you mean interpolating strings? That is intentionally unsupported and what you are doing looks fine to me.

@xiaq  Yes,  probably better this way anyway as it saves me trouble of having to escape brackets and such.


Drop aws_secretsmanager_caching from the codebase

Closes #16

My only concern would be potential performance impact, but I suspect releasing is the only way we can realistically see that.
We could try and build in our own caching layer, but I'm not sure that's worth the effort. I'd probably only do that if we had obvious performance issues from this change.

We never actually ever re-retrieve in any of our current internal workflows, it just gets fetched once when Django starts up, all the settings get populated and we never attempt to retrieve the secret again, so the cacher doesn't do anything for us, so this won't affect performance.


Consistent package names, gradle clean

This change is

Reviewed 86 of 86 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable


_sparse suffixes for L_p spaces in documentation

There is no _sparse suffixes in almost all L_p spaces in documentation ( https://github.com/nmslib/nmslib/blob/master/manual/spaces.md#lp-spaces )
Seems it should be l1_sparse, l2_sparse and linf_sparse

Indeed, thank you for noticing (added).


FIX for: _on_message() takes 2 positional arguments but 3 were given

Hello,
pip install certstream generates the following error:

To fix it I did the following steps (You may have to use pip and pip3 just in case):
pip uninstall certstream
pip uninstall websocket-client
git clone https://github.com/CaliDog/certstream-python.git && cd certstream-python

Now edit the requirements.txt file and change websocket-client>=0.56.0 to websocket-client==0.56.0
Now run: pip install -r requirements.txt
Everything should work now. If not try the following as well:
pip3 install -r requirements.txt
python3 setup.py install
Now you can just run the Python script with:
python3 certstream-python/certstream/cli.py

Note
I'm using Python 3.8.5

This method solved my problem, thank you!

Seeing the same error with python 3.9.1.
[ERROR:websocket] 2021-04-28 11:57:39,876 - error from callback <bound method CertStreamClient._on_message of <certstream.core.CertStreamClient object at 0x7fc84b42c640>>: _on_message() takes 2 positional arguments but 3 were given

Hi folks, this should be fixed with #42

Confirmed that it's working now


Add header options for starting workflow via CLI (#2862)

What changed?
This change exposes three new optional parameters for starting a workflow:

header_key
header_value
header_file

They allow setting header values in JSON format which are propogated
via workflow context. Values can be provides either directly or with a
file. If multiple keys and values are used, they separated by a space.
Count of keys and values must be equal, otherwise error is returned.

Why?
It was not possible to set header values via CLI when starting a workflow.

How did you test it?
Tested locally with:
./cadence --domain mickey workflow start --tasklist mini --workflow_type pluto --execution_timeout 1 --header_key='aaa bbb ccc ddd eee fff ggg hhh' --header_value='1 3.3 "text" true [] {} [1,2,3] {"inner":"a"}'

Verified that headers were set on local server:
header.fields.ggg.0     1
header.fields.ggg.1     2
header.fields.ggg.2     3
header.fields.hhh.inner a
header.fields.aaa       1
header.fields.bbb       3.3
header.fields.ccc       text
header.fields.ddd       true

Potential risks

Coverage increased (+0.4%) to 68.212% when pulling 13a673f2d1abbfc7f45be4c12fbc2030d80d7334 on cli-workflow-start-header-support into e2401c4199a5578ed10c2080a2b93d1c3379f879 on master.


Update email doesn't show which collective it's from

Unless they happen to name the collective in the text of the update, or you look at the email sender details and notice the subdomain address, there's no way to tell which collective an update is from when you receive it as an email. Could it say "Update from [Collective]" at the top maybe?

yes, it's extra confusing because it has the open collective footer

Fixed in https://github.com/opencollective/opencollective-api/pull/1244


Bug or feature - Confusing shapes of Tensor resulted from tf.decode_csv

reader = tf.TextLineReader()
key, value = reader.read(filename_queue)
parsed = tf.decode_csv(value, [[0.0]] * 24)
label = parsed[0]
weight = parsed[1]
feature = tf.pack(parsed[2:])
label_tensor, weight_tensor, feature_tensor = tf.train.shuffle_batch(
      [label, weight, feature],
      batch_size=200,
      capacity=capacity,
      min_after_dequeue=min_after_dequeue,
      num_threads=num_threads)
print("Parsed tensors: %s, %s, %s" % )

The shapes of label_tensor, weight_tensor and feature_tensor are (200, ), (200, ) and (200, 22) respectively.
If we put an extra comma at the end of line 4 and 5, like this:
label = parsed[0],
weight = parsed[1],

Then the shapes become (200, 1), (200, 1) and (200, 26) respectively.
So an extra comma determines the shape of the resulted tensor. I am not quite familiar with Python and am new to TensorFlow, but is it supposed so?

The extra comma is significant: it turns a tf.Tensor object (e.g. parsed[0]) into a single-element tuple (e.g. (parsed[0],)).  When you pass them to APIs that expect a single tf.Tensor, TensorFlow automatically packs lists and tuples of tf.Tensor objects into a single tf.Tensor with an additional dimension. Therefore, since parsed[0] is a scalar (a 0-dimensional tensor), parsed[0], is treated as a single-element vector (a 1-dimensional tensor).
The tf.train.shuffle_batch(..., batch_size=200, ...) method accumulates a batch of 200 labels, weights, and features, by packing together 200 randomly sampled parsed inputs. Batching 200 scalars gives a shape of (200,), whereas batching 200 single-element vectors gives a shape of (200, 1).
Hope this helps. Note that in future, queries like this one are better handled on Stack Overflow, so that they're more easily searchable.

Thanks Derek for the detailed explanation. My question is resolved.
On Thu, Dec 29, 2016 at 3:59 PM, Derek Murray<EMAIL_ADDRESS>wrote:

The extra comma is significant: it turns a tf.Tensor object (e.g.
parsed[0]) into a single-element tuple (e.g. (parsed[0],)). When you pass
them to APIs that expect a single tf.Tensor, TensorFlow automatically
packs lists and tuples of tf.Tensor objects into a single tf.Tensor with
an additional dimension. Therefore, since parsed[0] is a scalar (a
0-dimensional tensor), parsed[0], is treated as a single-element vector
(a 1-dimensional tensor).
The tf.train.shuffle_batch(..., batch_size=200, ...) method accumulates a
batch of 200 labels, weights, and features, by packing together 200
randomly sampled parsed inputs. Batching 200 scalars gives a shape of
(200,), whereas batching 200 single-element vectors gives a shape of (200,
1).
Hope this helps. Note that in future, queries like this one are better
handled on Stack Overflow, so that they're more easily searchable.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/issues/6554#issuecomment-269594672,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAOttIsfFLVZuPEVjC9IyaNm3-ljd51Nks5rM2hygaJpZM4LXVX-
.


itemMediaView image flicker when loading image

Describe the bug
itemMediaView image flicker when loading image.
Component
SDC library
To Reproduce
Steps to reproduce the behavior:

Have a questionnaire that loads image using itemMedia
Open it

Expected behavior
Should not flicker.
Video
blinking_imageview.webm
Smartphone (please complete the following information):

Device: [e.g. Pixel4a emulator]
Android version: [e.g. Settings -> About phone -> Android version]
Build number: [e.g. Settings -> About phone -> Build number]
Version [e.g. 22]

Additional context
N/A
Would you like to work on the issue?
Yes

I suspect it could be because of the reason in this comment which says

The flickering is caused by you loading images into the same view in quick succession.

Maybe the bind function is being called unnecessarily in quick succession? Not sure if that is solvable but can you investigate ? Thanks @FikriMilano

I think the easiest way is to revert back to:
imageAttachment.setImageBitmap(image)

I've tried to fix Glide itself by following these tickets below, but none works: bumptech/glide#755 bumptech/glide#527

why did we use glide in the first place?
i think what's happening is reflow caused by glide loading up the image after thee text is loaded.
a potential fix (i'm not saying it's better) to consider is reserving the space on screen so that the text doesn't get moved down - this would probably alleviate the blinking effect.
if we remove the use of glide here, should we remove it as a dependency? do we use it elsewhere?


#GH-49 Moved all Sentry configurations to build_properties.json and removed sentry.properties

Summary
Moved all Sentry configurations to build_properties.json and removed sentry.properties. This will make build_properties.json the single source of truth for all build time configuration values.
webapp/sentry.properties is now automatically generated, deriving from build_properties.json
Checklist

[x] make test Ran test cases and ensured they are passing
[x] make check-style Ran style check and ensured both webapp and server components comply the style guides

This fixes #49


[WIP] Swift 2.0

Fix #156
New PR with the fixes from @ashfurrow after the transition to an organization (Old was #162 )
Done:

Currently using Quick and Nimble swift-2.0 branches
AlamoFire swift-2.0 branch as well
Moya Core is compiling
Minimal Swift 2.0 migration (not reworked; only made it compile)
@garethknowles migrated the code for RxSwift

TODO:

[ ] Migrate ReactiveCocoa to swift2 branch

[ ] Generate a podspec for swift2 branch first

Worth noting that this branch is currently not compiling on my computer with the following message:
Module file was created by an older version of the compiler; rebuild 'Security' and try again: /Applications/Xcode-7.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/iphonesimulator/x86_64/Security.swiftmodule

After uninstalling and re installing Xcode 7 ß5 the project is compiling with Swift 2.0 now :sunglasses:

Fantastic! Let me know when this is ready for review :tada:

Will do; just need to finish the ReactiveCocoa podspec and in theory that will make it compile 100%; next step would be to «translate» to Swift 2.0 and take advantage of stuff like guard and/or throws, etc

@Moya/contributors given the new repo organization would it make more sense to close this PR (again) and start 3 different ones; 1 Swift 2.0 for the core, and 2 for each Reactive extension repo? (maybe 3 because there's a shared reactive core) ?

For now, why don't we PR against this one. I'd rather not make swift-2 support depend on getting Carthage builds working. Shouldn't be difficult to transition then.

@Moya/contributors I'm almost done with this BUUUUT
After creating a podspec for ReactiveCocoa https://github.com/esttorhe/ReactiveCocoa/blob/swift2/ReactiveCocoa.podspec and correctly configuring it i'm facing one last issue.
When building ReactiveCocoa scheme on Demo.xcworkspace everything works as expected; but if I try to build Pods or Moya I'm getting this error:
/Users/<username>/Library/Developer/Xcode/DerivedData/Demo-dwrrownhydordrffokwamvlwfmga/Build/Products/Debug-iphonesimulator/ReactiveCocoa.framework/Headers/ReactiveCocoa-Swift.h:94:9: 'ReactiveCocoa/ReactiveCocoa.h' file not found

And here's the generated file:
// Generated by Apple Swift version 2.0 (swiftlang-7<IP_ADDRESS> clang-700.0.65)
#pragma clang diagnostic push

#if defined(__has_include) && __has_include(<swift/objc-prologue.h>)
# include <swift/objc-prologue.h>
#endif

#pragma clang diagnostic ignored "-Wauto-import"
#include <objc/NSObject.h>
#include <stdint.h>
#include <stddef.h>
#include <stdbool.h>

#if defined(__has_include) && __has_include(<uchar.h>)
# include <uchar.h>
#elif !defined(__cplusplus) || __cplusplus < 201103L
typedef uint_least16_t char16_t;
typedef uint_least32_t char32_t;
#endif

typedef struct _NSZone NSZone;

#if !defined(SWIFT_PASTE)
# define SWIFT_PASTE_HELPER(x, y) x##y
# define SWIFT_PASTE(x, y) SWIFT_PASTE_HELPER(x, y)
#endif
#if !defined(SWIFT_METATYPE)
# define SWIFT_METATYPE(X) Class
#endif

#if defined(__has_attribute) && __has_attribute(objc_runtime_name)
# define SWIFT_RUNTIME_NAME(X) __attribute__((objc_runtime_name(X)))
#else
# define SWIFT_RUNTIME_NAME(X)
#endif
#if defined(__has_attribute) && __has_attribute(swift_name)
# define SWIFT_COMPILE_NAME(X) __attribute__((swift_name(X)))
#else
# define SWIFT_COMPILE_NAME(X)
#endif
#if !defined(SWIFT_CLASS_EXTRA)
# define SWIFT_CLASS_EXTRA
#endif
#if !defined(SWIFT_PROTOCOL_EXTRA)
# define SWIFT_PROTOCOL_EXTRA
#endif
#if !defined(SWIFT_ENUM_EXTRA)
# define SWIFT_ENUM_EXTRA
#endif
#if !defined(SWIFT_CLASS)
# if defined(__has_attribute) && __has_attribute(objc_subclassing_restricted)
#  define SWIFT_CLASS(SWIFT_NAME) SWIFT_RUNTIME_NAME(SWIFT_NAME) __attribute__((objc_subclassing_restricted)) SWIFT_CLASS_EXTRA
#  define SWIFT_CLASS_NAMED(SWIFT_NAME) __attribute__((objc_subclassing_restricted)) SWIFT_COMPILE_NAME(SWIFT_NAME) SWIFT_CLASS_EXTRA
# else
#  define SWIFT_CLASS(SWIFT_NAME) SWIFT_RUNTIME_NAME(SWIFT_NAME) SWIFT_CLASS_EXTRA
#  define SWIFT_CLASS_NAMED(SWIFT_NAME) SWIFT_COMPILE_NAME(SWIFT_NAME) SWIFT_CLASS_EXTRA
# endif
#endif

#if !defined(SWIFT_PROTOCOL)
# define SWIFT_PROTOCOL(SWIFT_NAME) SWIFT_RUNTIME_NAME(SWIFT_NAME) SWIFT_PROTOCOL_EXTRA
# define SWIFT_PROTOCOL_NAMED(SWIFT_NAME) SWIFT_COMPILE_NAME(SWIFT_NAME) SWIFT_PROTOCOL_EXTRA
#endif

#if !defined(SWIFT_EXTENSION)
# define SWIFT_EXTENSION(M) SWIFT_PASTE(M##_Swift_, __LINE__)
#endif

#if !defined(OBJC_DESIGNATED_INITIALIZER)
# if defined(__has_attribute) && __has_attribute(objc_designated_initializer)
#  define OBJC_DESIGNATED_INITIALIZER __attribute__((objc_designated_initializer))
# else
#  define OBJC_DESIGNATED_INITIALIZER
# endif
#endif
#if !defined(SWIFT_ENUM)
# define SWIFT_ENUM(_type, _name) enum _name : _type _name; enum SWIFT_ENUM_EXTRA _name : _type
#endif
typedef float swift_float2  __attribute__((__ext_vector_type__(2)));
typedef float swift_float3  __attribute__((__ext_vector_type__(3)));
typedef float swift_float4  __attribute__((__ext_vector_type__(4)));
typedef double swift_double2  __attribute__((__ext_vector_type__(2)));
typedef double swift_double3  __attribute__((__ext_vector_type__(3)));
typedef double swift_double4  __attribute__((__ext_vector_type__(4)));
typedef int swift_int2  __attribute__((__ext_vector_type__(2)));
typedef int swift_int3  __attribute__((__ext_vector_type__(3)));
typedef int swift_int4  __attribute__((__ext_vector_type__(4)));
#if defined(__has_feature) && __has_feature(modules)
@import ObjectiveC;
@import Foundation;
@import Foundation.NSURLSession;
#endif

#import <ReactiveCocoa/ReactiveCocoa.h>

#pragma clang diagnostic ignored "-Wproperty-attribute-mismatch"
#pragma clang diagnostic ignored "-Wduplicate-method-arg"

/// Wraps an Action for use by a GUI control (such as NSControl or UIControl), with KVO, or with Cocoa Bindings.
SWIFT_CLASS("_TtC13ReactiveCocoa11CocoaAction")
@interface CocoaAction : NSObject

/// The selector that a caller should invoke upon a CocoaAction in order to execute it.
+ (SEL __null_unspecified)selector;

/// Whether the action is enabled.
///
/// This property will only change on the main thread, and will generate a
/// KVO notification for every change.
@property (nonatomic, readonly) BOOL enabled;

/// Whether the action is executing.
///
/// This property will only change on the main thread, and will generate a
/// KVO notification for every change.
@property (nonatomic, readonly) BOOL executing;

/// Attempts to execute the underlying action with the given input, subject to the behavior described by the initializer that was used.
- (IBAction)execute:(id __nullable)input;
+ (BOOL)automaticallyNotifiesObserversForKey:(NSString * __nonnull)key;
@end

/// Wraps a dynamic property, or one defined in Objective-C, using Key-Value Coding and Key-Value Observing.
///
/// Use this class only as a last resort! MutableProperty
///  is generally better
/// unless KVC/KVO is required by the API you're using (for example,
/// NSOperation
/// ).
SWIFT_CLASS("_TtC13ReactiveCocoa15DynamicProperty")
@interface DynamicProperty : RACDynamicPropertySuperclass

/// The current value of the property, as read and written using Key-Value Coding.
@property (nonatomic, getter=rac_value, setter=setRac_value:) id __nullable value;

/// Initializes a property that will observe and set the given key path of the given object. object must support weak references!
- (nonnull instancetype)initWithObject:(NSObject * __nullable)object keyPath:(NSString * __nonnull)keyPath OBJC_DESIGNATED_INITIALIZER;
@end

@interface NSNotificationCenter (SWIFT_EXTENSION(ReactiveCocoa))
@end

@interface NSURLSession (SWIFT_EXTENSION(ReactiveCocoa))
@end

@interface RACCommand (SWIFT_EXTENSION(ReactiveCocoa))
@end

@interface RACDisposable (SWIFT_EXTENSION(ReactiveCocoa))
@end

@class NSDate;

@interface RACScheduler (SWIFT_EXTENSION(ReactiveCocoa))
@property (nonatomic, readonly) NSDate * __nonnull currentDate;
@end

@interface RACSignal (SWIFT_EXTENSION(ReactiveCocoa))
@end

#pragma clang diagnostic pop

The faulting line being 94:
#import <ReactiveCocoa/ReactiveCocoa.h>

Any ideas why it works when building it separately works but not when trying to build Pods or Moya ?

Whoa! Esteban, how did you make the RAC 3 podspec? I could update the existing one on CocoaPods trunk if that would help.

@ashfurrow I forked ReactiveCocoa and then added the .podspec to the swift2 branch.
Even created a custom tag just to test it but sadly I can't make it work inside the Pods project.
If you can update the podspec I would give it a try with your version

I'll see what I can do.

@esttorhe It looks like RAC 3 beta 9, as well as a newer release candidate, are both on CocoaPods Trunk: https://github.com/CocoaPods/Specs/blob/master/Specs/ReactiveCocoa/3.0-beta.9/ReactiveCocoa.podspec.json
So you should be able to use them.

Wouldn't that fail if I try to use that podspec but pointing to swift2 branch?

AHHH, right >.<
I don't see anything official from ReactiveCocoa about Swift 2 – normally, I'd just point it at a branch of theirs in the Podfile. They have a swift2 branch, but I dunno if that'll work.

@esttorhe could you create a new PR to Moya's swift-2.0 branch? We can merge it then.

@ashfurrow :scream: dear lord; this was pointing to master !?
hopefully third time is the charm :sparkles:
I'll create the PR right now and will close this one.


Support Ingestibles - Consume sound

Also bump VCPkg baseline, couldn't find the one in main
Added .vs dir to gitignore for visual studio users (though I love intellij)

Looks good, thanks.


add images

Here is how to work on your slides with the hide-able text
```{card}

![alt text](path/to/image)
+++
description that will be hidden in presentation mode
```


pip3 location?

When I attempt to install I get an error:
root@zones:~/rpi2home-assistant# make raspbian_install
# check OS version
Your operating system seems to be OK for rpi2home-assistant
# install python venv
python3 -m venv /root/rpi2home-assistant-venv
/root/rpi2home-assistant-venv/bin/pip3 install .
/bin/bash: line 1: /root/rpi2home-assistant-venv/bin/pip3: No such file or directory
make: *** [Makefile:37: raspbian_install] Error 127

pip3 is installed but not the venv:
root@zones:~/rpi2home-assistant# whereis pip3
pip3: /usr/bin/pip3 /usr/share/man/man1/pip3.1.gz

I'm using debian 12.7:
root@zones:~/rpi2home-assistant# cat /etc/debian_version
12.7

hey @tfcarlin ,
can you copy-paste what you have inside /root/rpi2home-assistant-venv/bin ?
I do have a pip3 there:
root@myhost:~/rpi2home-assistant-venv/bin# ls -l
total 56
-rw-r--r-- 1 root root 2034 Aug  7 23:04 activate
-rw-r--r-- 1 root root  960 Aug  7 23:04 activate.csh
-rw-r--r-- 1 root root 2240 Aug  7 23:04 activate.fish
-rw-r--r-- 1 root root 9033 Aug  7 23:04 Activate.ps1
-rwxr-xr-x 1 root root  241 Jun 25 00:21 f2py
-rwxr-xr-x 1 root root  241 Jun 25 00:21 numpy-config
-rwxr-xr-x 1 root root  242 May 29 14:38 pinout
-rwxr-xr-x 1 root root  243 May 29 14:38 pintest
-rwxr-xr-x 1 root root  246 May 29 14:38 pip
-rwxr-xr-x 1 root root  246 May 29 14:38 pip3
-rwxr-xr-x 1 root root  246 May 29 14:38 pip3.11
lrwxrwxrwx 1 root root    7 May 29 14:38 python -> python3
lrwxrwxrwx 1 root root   16 May 29 14:38 python3 -> /usr/bin/python3
lrwxrwxrwx 1 root root    7 May 29 14:38 python3.11 -> python3

Also try running
 apt install python3.11-venv
and then relaunch make raspbian_install

Very strange...but based on this (https://stackoverflow.com/questions/77200905/pip-not-installed-in-virtualenv-bin-folder) I recreated the venv and pip3 is now present. Ugh. Thanks for the quick reply!!!


demo now fails

The demo now fails.  In console, I see the following error.
Uncaught TypeError: Cannot read property 'numberOfItems' of undefined      svg.js:662

@timelyportfolio Is this happening when using latest version of svg.js?

I will debug further, but the screenshot was from http://jillix.github.io/svg.connectable.js/ in Windows 7 Chrome Canary.  Will let you know.

@timelyportfolio Thanks for heads-up! Looks like a solved svg.js issue: https://github.com/wout/svg.js/issues/409


handle shift+click for list that are not focused fixes #122402

fix list shift+click when an element isn't initially focused

Reproduction steps, https://github.com/microsoft/vscode/issues/122402#issuecomment-828494840.  Test that doing this does focus the elements as expected.  Can test with either the terminal tabs or file explorer.
This PR fixes #122402.

The fix looks good, but it didn't catch every case in the issue, namely https://github.com/microsoft/vscode/issues/122402#issuecomment-828508026
Because of that, I've decided to go ahead with a different fix: https://github.com/microsoft/vscode/commit/edf85f1711d6f59c5d51bb66a54e86603e339136
Thanks anyway! 🍻


Dnsmasq running inside container.

This needs to wait for a packer build for being merged.
We might want wait for https://github.com/Capgemini/Apollo/issues/129 and https://github.com/Capgemini/Apollo/issues/392

Just need to wait for https://github.com/Capgemini/Apollo/pull/441 to merged now as the ami ids here include changes in that PR


Wait for all requests to load(complete Page load)

It would be nice to have waitUntilCompletePageLoad(or something with a better name) method that waits for all the requested resources to load for the current page(i.e all ajax calls).
Context
I am currently testing the load times of pages in a web application. We make a lot of ajax calls the the page loads. What I need is the time from page load to the time the "loading" spinner stops. It could probably be implemented using waitUntilVisible over multiple selectors. But this is complicated. For the purposes of page load test. I only need the total time.
Implementation
We have a version that uses waitFor and tracks the resource  requested/resourceReceived counts to  make sure we load the full page before timing it. I wonder if it makes sense to bake it into Casperjs itself?
Thoughts?
I could do a pull request if you think it makes sense.

@n1k0
Can you show me how to implement watiFor with a event listener?
Code snippet here with comment on what I want to achieve with waitFor found here:
https://gist.github.com/acegreen/17a7e81630235d9012cd


pull request for e2e test at 12/27/2016 00:56:12

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Allow CouchDB by_db metrics to be disabled.

What does this PR do?
This PR introduces the enable_per_db_metrics flag to the CouchDB integration config. It's set to true by default, preserving the existing behaviour of the integration, but disabling it will prevent the integration from listing all of your CouchDB databases every time the check runs.
Motivation
We found that the CouchDB integration causes our CouchDB cluster to use a lot of CPU when enabled.

We suspect this is caused by calls to /_all_dbs, which take several seconds to complete with the number of databases we have in our cluster (~200k).
Additional Notes
This is my first PR to this repo, and I've done my best to following the CONTRIBUTING.md and developer documentation. I was only able to get the CouchDB 1.6 tests running by explicitly setting the platform in the docker-compose.yaml. I believe this should be safe, and will allow people with recent Apple machines to run these tests.
Also, apologies for some of the noise in the diff. I used Ruff, as per the README.md, but it seems it hasn't been run on this integration before so lots of quotes have been changed from single to double.
Review checklist (to be filled by reviewers)

[x] Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
[x] Changelog entries must be created for modifications to shipped code
[ ] Add the qa/skip-qa label if the PR doesn't need to be tested during QA.

I'm not sure what's wrong with the CHANGELOG.md changes I've made, I believe I've followed the format at https://datadoghq.dev/integrations-core/guidelines/pr/ correctly.

@hestonhoffman thanks for your initial review! I've committed your suggestions and updated the branch to be in line with master. 🙏

@iliakur Done and done. Sorry for the hassle, should be much easier to review now. 🙂

@samwho run ddev test -fs chouch to fix the linting issue.

Ah, thank you! I hadn't noticed this. I'm on a half day today, will fix this later this afternoon (UK time).
Thanks for the fast turnaround on this change.

@samwho thanks again for the contribution and seeing it through!

@iliakur thank you for your review! Will this be included in the next datadog-agent release?

@iliakur thank you for your review! Will this be included in the next datadog-agent release?

Yea, that would be 7.52.0, which should drop second half of March. If you can't wait until then, you could build the wheel locally and install it using the agent integration command.

Oof, a little bit longer than we were hoping for 😅 Is the release process/cycle documented somewhere? I'm a little bit confused that this change wouldn't be in the next patch/minor release, given that it is backwards compatible.
Nice to have the option to build locally if we need to, thank you for letting me know about that.


Unreachable Code in Color Palette Helper

The file ColorpaletteHelper, method get_palette has code that appears to be unreachable, namely that to convert to hsv.   The config default is used instead.

Code deleted


Clarification b/w IMPLEMENTATION and SETUP guide

Updating readme so that newcomers don't find the setup guide confusing.
[^1]: Development Testing
[^2]: Browser and device support
[^3]: Accessibility Target

@gasman Hey, I got what you were saying. What I meant was....there are two types of setup a user might want to do
The first one is to USE the Wagtail to create their website, (where the person is a user)
The second one is to set up the CODEBASE on their PC in order to contribute. (where the person is contributer)
I myself found it very difficult to set up the source code.
So, keeping that in mind I've proposed these changes.
Please give me further feedback if necessary.

Thanks, I understand now!
During initiatives like GSoC and Hacktoberfest, people might be arriving at the project with the intention to contribute, but that's a relatively rare case - the vast majority just want to use it (and will maybe contribute later on if there's a specific thing they want to fix), and for those people, even mentioning the option of contributing is a distraction at this point in the readme. There's a "Contributing" section further down the readme addressing the people who do want to contribute.
I'd be happy to go for your first change ("To get started with Wagtail" -> "To get started with using Wagtail") - that one seems like a good idea, since the meaning stays exactly the same for 'normal' users, while giving GSoC / Hacktoberfest people extra information to help steer them to the right place 👍 . I think the rest should be kept as the original version.

@gasman Good good. I've reverted the 2nd part of the changes. It should be fine now.


Scrcpy app for android

Is there a way to easily port to android? I want to remote into a phone that has no display but from my phone running android,not a PC on Linux or Windows. (Via wireless debugging It works in Windows and Linux but I want to use another phone not a PC)
I tried Termux running Ubuntu 20.04 and I was able to install scrcpy via apt install  but adb won't start I get "adb server didn't ACK adb failed to start daemon"

See #100 (from a very old version of scrcpy).

Yup seems like his app is what I want.I did try it but I just found out that it doesn't work on android 11 likely because because of the port after the IP address


Support for more GPIO ports via MCP23017

Your plugin is awesome!
An idea to make it even better would be to add support for GPIO multiplication chips like the MCP23017 (connected via i2c) to be able to use more of the devices you already support.
Otherwise there is just a limited number of GPIO to use.

I don't support any I2C nor SPI device at the moment.  I think the real-time clock on the RaspBee II uses I2C, but other than that I have no I2C devices.  I know pigpio supports I2C, but I have no experience developing against it.  Unless there's a really cool device that I just have to investigate, I won't be looking into it - too many other interesting things to pursue.

Otherwise there is just a limited number of GPIO to use.

Recent Raspberry Pi models expose 27 GPIO pins (GPIO 2 through 27) on the 40-pin header, available for user-defined purposes.  If you have a need for more than that, I would look at getting another Raspberry Pi Zero 2, before messing with port extenders.  If you have a specific device with an embedded port extender, it's a different story, of course.  But then it's really about supporting that device.
I started Homebridge RPi because I wanted to monitor the Pi's hardware, to figure out the right cooling solution.  Hence the early support for the FanSHIM.  That's still my main usage, even though I've switched to passive cooling.
Once I got the FanSHIM's button and fan supported through pigpio, it was easy enough to expose other simple devices connected to a single GPIO input or GPIO output pin.  I'm happy to add more of those binary sensors and actuators, using whatever HomeKit services available.  I'm hesitant to expose more complex devices, especially when I won't be using them myself.  I won't be exposing custom-built devices, using multiple GPIO pins.  I simply don't have the bandwidth to maintain and support these.
I played a bit with PWM for a dimmable LED and servo, but don't like that I need to poll them for the current state.  The FanSHIM includes a chainable LED (even through it doesn't expose the pins to chain another LED), and the Blinkt! uses the same.  The P9813 has a very similar protocol, and I could add support in a couple of hours, earlier today.  I don't partially like them either, as they don't report their current state at all.  The DHTxx temperature/humidity sensor was fun, but about the limit of what I can support.  I still have some NeoPixel LEDs lying around, but I doubt I can support them without root privileges.

I started Homebridge RPi because I wanted to monitor the Pi's hardware, to figure out the right cooling solution. Hence the early support for the FanSHIM. That's still my main usage, even though I've switched to passive cooling.

OK now I think I understand why you have so much log entries about temperature and CPU load... Actually as I use it in my house to activate lights and monitor sensors and switches, I would be happy to have the temperature and load logged less frequently. Would you be ok to allow the heartbeat in Eve be set to up to an hour? this way the events happening in the night would still appear in the morning instead of being over written in the log file.
Still love the plugin though. I have ordered a new Pi 3 with more GPIO as mine is an old one with 26 pins only...


Shown files don't match ls command in the terminal

Describe the bug
The "files" menu is showing this:

But the terminal ls -la command outputs this:

Even the npm start command works.
However, after I save, exit, and reopen, the terminal will only list the files that are shown in the menu. This makes the project lose relevant files.
Link to the blitz that caused the error
https://stackblitz.com/edit/stackblitz-starters-whhznj?file=package-lock.json
Steps to reproduce

Create Node project
cd ..
rm -rf stackblitz-starters-whhznj (use project ID)
npx @angular/cli@14 new stackblitz-starters-whhznj (use project ID)
cd stackblitz-starters-whhznj (use project ID)
ls

Expected behavior
The "files" menu should show the files generated by the CLI app.
When I save and reopen the project, the "files" menu should reflect the VM's file system too.
Parity with Local

[X] I have run the project in my local machine and I could not reproduce the issue.

Screenshots
No response
Platform
Browser name  = Firefox
Full version  = 121.0
Major version = 121
navigator.appName = Netscape
navigator.userAgent = Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:121.0) Gecko/20100101 Firefox/121.0
performance.memory = undefined
Hash = f7aa08df

Additional context
No response

Hey @LuisTovar0 👋 ! Thanks for reporting. I can confirm this is an issue. I'll create an internal ticket to track this.

I can now confirm this issue is fixed. It happened to be I was working on that area of the codebase and found the bug quite quickly. Let us know if there's still something you run into.


Update google-cloud-bigquery to 1.127.5

Updates com.google.cloud:google-cloud-bigquery from 1.122.2 to 1.127.5.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.google.cloud", artifactId = "google-cloud-bigquery" } ]

labels: library-update, semver-minor

Superseded by #190.


fix/discuss recommended workflows

I am surprised to see the following in our current recommendations:

Ocropy nlbin instead of one of the Olena algorithms
slow skimage binarize/denoise processors instead of Olena/Ocropy
only Tesseract deskewing (no Ocropy)
region clipping after Ocropy page segmentation (not necessary)
Ocropy line segmentation after Ocropy page segmentation (redundant)
line clipping after Ocropy line segmentation (not necessary)
Tesseract line segmentation without resegmentation or line clipping (to remove bbox overlaps)
Tesseract vs Calamari recognition (should be exchangeable regardless of workflow up to that point)

EDIT (thanks @jbarth-ubhd for reminding me): also

region clipping after region deskewing (impossible!)

Do these choices have some empirical grounding (measuring quality and/or performance on GT)?

if olena binarization, I would recommend wolf:

But in Step 5 ocropy-deskew is mentioned ( but not "recommended" in Step 9 (?) )

Hmm.. we are talking about https://ocr-d.de/en/workflows ?

Hmm.. we are talking about https://ocr-d.de/en/workflows ?
The "Recommendations" at the end of that page?

Yes!

But in Step 5 cis-ocropy-deskew is mentioned ( but not "recommended" in Step 9 (?) )

I don't mind that it is not recommended in step 9, because IMHO on the region level orientation is more important than skew, and can differ between region and page, whereas skew is usually uniform across a page (otherwise you usually need dewarping anyway).
The above list really only concerns the overall workflow configuration recommendations.

if olena binarization (other methods tested long before), I would recommend wolf

Thanks again for that test suite and for paying attention in general on that front, which is not appreciated enough IMO. We've briefly discussed this in the chat already, and I would like to elaborate on some open points:

The example image is interesting and helpful to see what's going on, but it is also misleading, because it is highly artificial: Algorithms with local thresholding are quite sensitive to highly localized contrast/brightness changes, esp. if they are discontinuous. But realistically, except for the special case of inverted text, these would be spread more widely and continuously across the page. Even more so for noise, which usually appears equally across the page, which is why raw denoising typically measures noise levels globally.
Binarization does not (have to) stand alone. If we know contrast/brightness is far from normal, and noise is perceptible, then we would run normalization and raw denoising before anyway. Of course some algorithms are more robust against either of those than others. But if we want a fair competition, we should eliminate them (because we can).
Most algorithms have 2 degrees of freedom: the window size (influencing locality; dependent on pixel density) and threshold level (influencing stroke weight). One should allow optimising for them, or at least representing different choices for them. For example, ocrd-skimage-binarize and ocrd-olena-binarize (since v1.1.11) do already set the window size automatically based on a DPI rule of thumb by default. (But this requires having correct DPI annotated.) Niblack is one of those algorithms which is extremely sensitive to the correct window size.
As in point 2, but after binarization: Some algorithms produce noise which can be easily removed with a simple binary denoiser. So for a fair comparison all methods should enjoy that benefit.

So I think we need a different artificial test bed. And then we also need a set of true images of various appearances and qualities.

PS2: One could argue "preprocess low-contrast first", but how to do this without knowing noise levels to skip etc... I think this is the primary task for binarization

I disagree. IMO there should be specialisation and modularisation. So binarization processors can concentrate on its core problem, and others can try to solve related ones. If a processor chooses to do 2 or 3 steps in one, fine (we've seen this elsewhere), but we should always have the option to freely combine. And that in turn means we must do it for a fair evaluation, too.

1. The example image is interesting and helpful to see what's going on, but it is also misleading, because it is highly artificial: Algorithms with local thresholding are quite sensitive to highly localized contrast/brightness changes, esp. if they are discontinuous.

The word "Wiener" is just a snippet from a 1238 × 388 px image, but I now realize that I forgot to set the DPI.

2. Binarization does not (have to) stand alone. If we know contrast/brightness is far from normal, and noise is perceptible, then we would run normalization and raw denoising before anyway. Of course some algorithms are more robust against either of those than others. But if we want a fair competition, we should eliminate them (because we can

We have images with weak contrast, partly with black surrounding background. Then we must do preprocessing after the clipping step, too, I assume.
Will redo with DPI set.

With imagemagick convert -contrast-stretch 1%x7% # assumption: black on white and 300 DPI:

The word "Wiener" is just a snippet from a 1238 × 388 px image, but I now realize that I forgot to set the DPI.

Wow! They rarely come in as pristine in quality as this!

We have images with weak contrast, partly with black surrounding background. Then we must do preprocessing after the clipping step, too, I assume.

If you mean cropping instead of clipping, then yes. And contrast/brightness normalization afterwards.

With imagemagick convert -contrast-stretch 1%x7% # assumption: black on white and 300 DPI:

Thanks! Very interesting. You immediately see the class of algorithms that are most sensitive to level dynamics: Sauvola type (including Kim). They are usually implemented without determinining the r from the input. (For example, Olena/Scribo just uses #define SCRIBO_DEFAULT_SAUVOLA_R 128 in a uint8 space.)
But just to be sure: did you use the window size zero default for Olena (which should set it to the odd-valued number closest to DPI)? I would expect Niblack to look somewhat better...
Also, your contrast-stretch recipe is somewhat different to normalize as used by Fred's IM script textcleaner or by one of ocrd_wrap's presets. I wonder what made you consider increasing the white-out tolerance up to 7%. Perhaps we can get to an OCR-specific optimum preset here?

cropping instead of clipping

yes

did you use the window size zero default for Olena

I did not add any parameter -- defaults only.

I wonder what made you consider increasing the white-out tolerance up to 7%

from here: https://sourceforge.net/p/localcontrast/code/ci/default/tree/ctmf.c , line 362.
7/1 = white/black ratio when using "Stempel Garamond" Font (black on white) with reasonable leading.

from here: https://sourceforge.net/p/localcontrast/code/ci/default/tree/ctmf.c , line 362.
7/1 = white/black ratio when using "Stempel Garamond" Font (black on white) with reasonable leading. This is what I am used to do when I don't know better. The base idea is that black + white pixels are gaussian distributed and far enough apart, so I crop "equally" on both ends.

Oh I see. So IIUC your logic goes:

take an average Antiqua-style (modern serif) font,
look at its white/black ratio when set with an average leading
assume this will be the minimum white/black ratio, because between text blocks there will be even more white
expect input to be a binary distribution of the signal (non-normalized) plus some Gaussian noise (but not large enough to make both peaks coincide)
generally, try to normalize the dynamic range by stretching the histogram to the full dynamic before thresholding
but try to make better use of the dynamic range in the interesting area between both history peaks (pushing them further apart) by clipping the extremes beyond the peaks
since both peaks must be expected to be of different heights, with at least as much more white than black as in the artificial consideration above, clip 7 times more of the brightest pixels to absolute white than of the darkest pixels to absolute black

So you could have used convert -contrast-stretch 0.5%x3.5% by the same argument, right?
Also, I'd like to verify that ratio point for concrete scans, because on average I assume initials, separators, ornaments and images will increase the share of black. (I'll make a coarse measurement on a real corpus to check.)

Yes, this the idea.
Some interesting binarization with much more equations someone else pointed me to: https://arxiv.org/pdf/2007.07350.pdf

Some interesting binarization with much more equations someone else pointed me to: https://arxiv.org/pdf/2007.07350.pdf

Yes, we've briefly discussed that in the Lobby. Here is the implementation. Unfortunately does not combine with existing local thresholding algorithms yet.

So I think we need a different artificial test bed.

As a first step, @jbarth-ubhd could you please change your code to do each point in your matrix (i.e. noise columns, brightness rows) on the full image, and only tile it in the final summary image for visualisation?

Yes, this is was I've done. Process the "full" 1238 × 388 px image (from PDF, 300 DPI, DIN A7) and extract the word "Wiener" for compact comparison.

Yes, this is was I've done. Process the "full" 1238 × 388 px image (from PDF, 300 DPI, DIN A7) and extract the word "Wiener" for compact comparison.

Oh, I see! Then I misunderstood you above. In that case you can scratch my point 1 entirely.
So how about running ocrd-skimage-normalize in comparison to your contrast-stretch, and running ocrd-skimage-denoise-raw even before that, and running ocrd-skimage-denoise after binarization? That would be points 2 and 4. Finally, point 3 would be running with different window sizes, say 101 (i.e. smaller than default, more localized) and 401 (i.e. larger), and thresholds, say k=0.1 (i.e. heavier foreground) and k=0.4 (i.e. lighter foreground). If you point me to your implementation, I can help with a PR/patch...

https://digi.ub.uni-heidelberg.de/diglitData/v/various-levels-black-white-noise.tgz . Feel free to anything with it. Sorry no docs. "gen" generates various bXXXwXXXnX.ppm from Beethoven-testtext.pgm downsampling 25%. Sorry width+hight hard coded in gen.c++. Convert it to .tif . In methods/ run do.cmd, afterwards montage.pl . I could do it on monday.

I think show-through can't be binarized correctly in all cases. What, if this was a blank page and the whole reverse page showes through? Perhaps we could build some statistics over all pages of a book so we can estimate the average minimum (local) contrast, but what then when a page had weak ink...

I think show-through can't be binarized correctly in all cases. What, if this was a blank page and the whole reverse page showes through? Perhaps we could build some statistics over all pages of a book so we can estimate the average minimum (local) contrast, but what then when a page had weak ink...

sbb-binarize:

Would be interesting to see how sbb-binarize copes with normalized and with cropped images. But the message is already clear: Good neural modelling is superior.

I think show-through can't be binarized correctly in all cases. What, if this was a blank page and the whole reverse page showes through?

I don't think this is really an issue practically. We will need (and have) page classification anyway, and having a class empty page beside title, index, content (or whatever) should not be difficult.

Perhaps we could build some statistics over all pages of a book so we can estimate the average minimum (local) contrast, but what then when a page had weak ink...

Good idea, but robust heuristic binarization needs to be locally adaptive, so it might be difficult to go global even across the page. Perhaps some algorithms are more suited for this than others. And certainly quality estimation will build on such global statistics.

Just for completeness: ocropus-nlbin with -n; not normalized before:

Just for completeness: ocropus-nlbin with -n; not normalized before:

Should be the same as ocrd-cis-ocropy-binarize, right?

(venv) xx@yy:~/ocrd_all> find . -name "*.py"|egrep -v '/venv/'|xargs grep -3  percentile_filter
./ocrd_anybaseocr/ocrd_anybaseocr/cli/ocrd_anybaseocr_binarize.py-            # if not, we need to flatten it by estimating the local whitelevel
./ocrd_anybaseocr/ocrd_anybaseocr/cli/ocrd_anybaseocr_binarize.py-            LOG.info("Flattening")
./ocrd_anybaseocr/ocrd_anybaseocr/cli/ocrd_anybaseocr_binarize.py-            m = interpolation.zoom(image, self.parameter['zoom'])
./ocrd_anybaseocr/ocrd_anybaseocr/cli/ocrd_anybaseocr_binarize.py:            m = filters.percentile_filter(
./ocrd_anybaseocr/ocrd_anybaseocr/cli/ocrd_anybaseocr_binarize.py-                m, self.parameter['perc'], size=(self.parameter['range'], 2))
./ocrd_anybaseocr/ocrd_anybaseocr/cli/ocrd_anybaseocr_binarize.py:            m = filters.percentile_filter(
./ocrd_anybaseocr/ocrd_anybaseocr/cli/ocrd_anybaseocr_binarize.py-                m, self.parameter['perc'], size=(2, self.parameter['range']))
./ocrd_anybaseocr/ocrd_anybaseocr/cli/ocrd_anybaseocr_binarize.py-            m = interpolation.zoom(m, 1.0/self.parameter['zoom'])
./ocrd_anybaseocr/ocrd_anybaseocr/cli/ocrd_anybaseocr_binarize.py-            if self.parameter['debug'] > 0:
grep: ./ocrd_olena/repo/olena/dynamic-use-of-static-c++/swig/python/ltihooks.py: Datei oder Verzeichnis nicht gefunden
--
./ocrd_cis/ocrd_cis/ocropy/common.py-        warnings.simplefilter('ignore')
./ocrd_cis/ocrd_cis/ocropy/common.py-        # calculate at reduced pixel density to save CPU time
./ocrd_cis/ocrd_cis/ocropy/common.py-        m = interpolation.zoom(image, zoom, mode='nearest')
./ocrd_cis/ocrd_cis/ocropy/common.py:        m = filters.percentile_filter(m, perc, size=(range_, 2))
./ocrd_cis/ocrd_cis/ocropy/common.py:        m = filters.percentile_filter(m, perc, size=(2, range_))
./ocrd_cis/ocrd_cis/ocropy/common.py-        m = interpolation.zoom(m, 1. / zoom)
./ocrd_cis/ocrd_cis/ocropy/common.py-    ##w, h = np.minimum(np.array(image.shape), np.array(m.shape))
./ocrd_cis/ocrd_cis/ocropy/common.py-    ##flat = np.clip(image[:w, :h] - m[:w, :h] + 1, 0, 1)

@jbarth-ubhd I'm not sure what you want to say with this. But here's a comparison of both wrappers for old ocropus-nlbin:

anybaseocr-binarize
cis-ocropy-binarize

OCR-D wrapper only formally correct
OCR-D wrapper adequate

exposes all params as is
exposes only relevant ones, controls for others (e.g. zoom via DPI)

original code without changes
includes some fixes:

* pixel-correct image size

* robustness against NaN

* zoom and plausibilise sizes relative to DPI

* opt-in for additional deskewing and/or denoising

* opt-in for grayscale normalization

Should be the same as ocrd-cis-ocropy-binarize, right?

Didn't know if orcropus-nlbin is the same as cis-ocropy-binarize, so tried to find out and found some lines that look very similar.

Should be the same as ocrd-cis-ocropy-binarize, right?

Didn't know if orcropus-nlbin is the same as cis-ocropy-binarize, so tried to find out and found some lines that look very similar.

Oh, now I got it. (Sorry, have never found the time to summarise my ocropy/ocrolib changes and re-expose the old ocropus CLIs from it.)
My question was actually whether the picture looks any different when you use the OCR-D wrapper.


Fix rounding bug in lookup_glyph_id and fix a corner case.

An attempt to fix the bug described here: https://github.com/raphlinus/font-rs/issues/32
Additionally, found a corner case where it underflows when the code point searched for is smaller than any of the code points supported by the font.

Hmmm, https://docs.microsoft.com/en-us/typography/opentype/spec/cmap#format-4-segment-mapping-to-delta-values doesn't say anything about rounding up, and integer division usually truncates.
What font and code point are you testing with?

Oops, now there is a case where 1 divided by 2 doesn't go to 0 so the loop never terminaties. Fixing that.
I'm using https://fonts.google.com/specimen/Kosugi+Maru and the code point it fails with is 赤, U+8d64.

https://github.com/raphlinus/font-rs/pull/34 fixes this with more robust and easily-understandable logic so I'm closing this.


Uses GITHUB_API_TOKEN for API rate limiting

Please create and add a GITHUB_API_TOKEN to .travis.yml
See https://asdf-vm.com/#/plugins-create?id=github-api-rate-limiting

I think this PRno longer applies, as another PR with more changes (including this one) is already merged. But thanks for the PR and sorry for the delay! ❤


qt_gui_core: 1.1.2-2 in 'foxy/distribution.yaml' [bloom]

Increasing version of package(s) in repository qt_gui_core to 1.1.2-2:

upstream repository: https://github.com/ros-visualization/qt_gui_core.git
release repository: https://github.com/ros2-gbp/qt_gui_core-release.git
distro file: foxy/distribution.yaml
bloom version: 0.10.0
previous version for package: 1.1.2-1

qt_dotgraph
* add API to set edge tooltip (#237 <https://github.com/ros-visualization/qt_gui_core/issues/237>)

qt_gui
* allow hide title in standalone (#235 <https://github.com/ros-visualization/qt_gui_core/issues/235>)

qt_gui_app

No changes

qt_gui_cpp

No changes

qt_gui_py_common

No changes

Holding for coordinated merge...

@clalancette What was this deb inc for?

@clalancette What was this deb inc for?

Re-release with bloom 0.10.0 to get the fix for https://github.com/ros2/console_bridge_vendor/issues/19 in.

Re-release with bloom 0.10.0 to get the fix for ros2/console_bridge_vendor#19 in.

But none of the packages in this repo depend on it?

But none of the packages in this repo depend on it?

I'm not sure exactly why it was needed, but without the re-release it was finding libconsole_bridge.so.0.4 (and not libconsole_bridge.so.1.0, which is what it should have been finding).

I'm not sure exactly why it was needed, but without the re-release it was finding libconsole_bridge.so.0.4 (and not libconsole_bridge.so.1.0, which is what it should have been finding).

I highly doubt that this deb inc addressed the problem. I assume you also released packages which directly depend on console_bridge(_vendor) which are dependency of qt_gui_core. Rer-releasing those already triggers a rebuild of qt_gui_core which should have solved the problem.

I highly doubt that this deb inc addressed the problem. I assume you also released packages which directly depend on console_bridge(_vendor) which are dependency of qt_gui_core. Rer-releasing those already triggers a rebuild of qt_gui_core which should have solved the problem.

I thought the same thing, but it didn't work.  Yesterday I re-released urdf, urdfdom, and kdl_parser into Rolling.  When everything finally finished rebuilding, those 3 packages (and their dependencies) were using libconsole_bridge.so.1.0, but qt_gui_cpp was still using libconsole_bridge.so.0.4.  This morning I then re-released qt_gui_core into Rolling, and it solved the problem.
I didn't actually wait in between releases on Foxy here, but I assume the same thing would have happened.

I would strongly suggest to figure out the actual cause (why didn't it work, what is the dependency hierarchy connecting any package in this repo to console_bridge, etc.) to ensure the mental model matches reality.


Performance: Skip metric index if no glob

Speeds up check for metric existence by the carbon plugin.

This change is

+@wdauchy please have a look, could be a good opportunity to dive in the project ( -@natbraun  to balance)

Review status: 0 of 1 files reviewed at latest revision, all discussions resolved.

Comments from Reviewable

Review status: 0 of 1 files reviewed at latest revision, 1 unresolved discussion, some commit checks failed.

biggraphite/drivers/cassandra.py, line 449 [r1] (raw file):

        }
        try:
            metrics_names = [r[0] for r in self.__session.execute(query)]

is it really needed here?

Comments from Reviewable

Review status: 0 of 1 files reviewed at latest revision, 1 unresolved discussion, some commit checks failed.

biggraphite/drivers/cassandra.py, line 449 [r1] (raw file):
Previously, wdauchy (William Dauchy) wrote…
> is it really needed here?

you mean replacing `.name` with `[0]` ? No it's needed, I can do it in another change if you want, but it'd be a one line change...

Comments from Reviewable

Review status: 0 of 1 files reviewed at latest revision, 1 unresolved discussion, some commit checks failed.

biggraphite/drivers/cassandra.py, line 449 [r1] (raw file):
Previously, unbrice (Brice Arnould) wrote…
> you mean replacing `.name` with `[0]` ? No it's needed, I can do it in another change if you want, but it'd be a one line change...

I meant: it's *not* needed, only a small refactoring on the way

Comments from Reviewable

Reviewed 1 of 1 files at r1.
Review status: all files reviewed at latest revision, 1 unresolved discussion.

Comments from Reviewable


marketing card button on accounts overview page needs content from the cta_text key in the json output

On the accounts Dashboard page - the marketing card button needs the cta_text content from the json endpoint:
https://kfg6bckb-dev.webengine.zesty.io/-/accountsdashcards.json

"marketing": {
"title": "Integrate your Favorite Technology with Zesty",
"description": "The Zesty.io Marketplace features top integrations and tools, such as Live Editor and Layouts, to help you create and distribute content faster.",
"cta_text": "Go to Marketplace",
"link_type": "external",
"url": "https://www.zesty.io/marketplace"
}

Thanks @bltarkany will work on this


"backward pass is invalid for module in evaluation mode" with deepspeed stage 3

Bug description
Hello,
After 2.2 Lightning upgrade and only with deepspeed stage 3, we experience a crash backward pass is invalid for module in evaluation mode. Most likely is caused by the recent changes in train/eval mode switching.
What version are you seeing the problem on?
v2.2
How to reproduce the bug
use deepspeed stage 3 strategy

Error messages and logs
  File "/pip-ai-experimental_torch/site-packages/torch/autograd/__init__.py", line 251, in backward
    Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
  File "/pip-ai-experimental_torch/site-packages/torch/autograd/function.py", line 288, in apply
    return user_fn(self, *args)
  File "/pip-core_deepspeed/site-packages/deepspeed/runtime/zero/parameter_offload.py", line 170, in backward
    ctx.pre_backward_function(ctx.module)
  File "/pip-core_deepspeed/site-packages/deepspeed/utils/nvtx.py", line 15, in wrapped_fn
    ret_val = func(*args, **kwargs)
  File "/pip-core_deepspeed/site-packages/deepspeed/runtime/zero/parameter_offload.py", line 447, in _run_before_backward_function
    self.pre_sub_module_backward_function(sub_module)
  File "/pip-ai-experimental_torch/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/pip-core_deepspeed/site-packages/deepspeed/runtime/zero/parameter_offload.py", line 524, in pre_sub_module_backward_function
    assert sub_module.training, "backward pass is invalid for module in evaluation mode"
AssertionError: backward pass is invalid for module in evaluation mode

Environment
torch 2.1.2
More info
No response

@olegsinavski This is unavoidable unfortunately. You must have a module somewhere in LightningModule that's in eval mode, and deepspeed makes this absurd decision to not let you backward (even though train/eval mode has nothing to do with gradient computation).
Maybe you have a HFModel.from_pretrained() or similar that defaults to eval mode.

Yes, I do actually have HFModel.from_pretrained()! Could you please elaborate why it's unavoidable? From the perspective of Lightning user, it does look like a regression in the behavior of 2.2: on 2.1 it works fine, but on 2.2 stage 3 doesn't work (the stage 2 still works). Since it worked on 2.1 Lightning, there was something that made it work?
I just checked that .from_pretrained() produces model with model.training == False for both stage 2 and stage 3. Why would it work for Lightning with stage 2 then?

Could you please elaborate why it's unavoidable?
Since it worked on 2.1 Lightning, there was something that made it work?

HuggingFace models loaded with .from_pretrained() are by default in eval mode, because they are expected to be used for inference. To train, you would need to call .train() on them. This is special to HuggingFace, because by default in PyTorch if you create any nn.Module, it will be in training mode.
In Lightning 2.1 and prior, Trainer called model.train() at the beginning of .fit(), which is why your HuggingFace model was silently converted to training mode. While in your case this might have been ok, this automatic conversion had one big problem: If you had frozen layers in eval mode that you didn't want to train, the Trainer would silently convert them to train mode anyway and there would be no way for the user to know that. Overriding that behavior required implementing a special hook in Lightning, but this was practically not discoverable by users. In 2.2, we removed this silent behavior, so users don't fall into this trap. Now, if you train a model in Lightning and have part of it frozen (e.g. you have a feature extractor that you don't train), then the training mode you set will be respected. I consider this an important fix, it has personally bothered me for a long time. Especially when users would compare their results trained with Lightning vs. raw PyTorch, and saw a discrepancy in the losses/accuracies due to this silent conversion that was happening.
I think one thing we could do, for visibility, is have include a column in the ModelSummary (the table that prints by default at the beginning of fit) that shows the training mode. This could help users sanity check this better.

Ok, thank you for the explanation! Totally makes sense! (although I personally prefer "require_grad=False" for frozen bits).
The last question - does it mean that that with stage 2, it also in val mode, its just stage 2 doesn't have that check so it doesn't crash?
I guess the reason stage 2 still works for me is because I don't have any dropout/batch_norms that change behavior depending on training flag... So everyone needs to call train() after .from_pretrained().
If I understand correctly, some users might have silent bugs if the used .from_pretrained() with dropout and batchnorms after the 2.2 upgrade. But I guess the new behavior is less "state-mutating" and hence safer on average

Just wanted to report that I'm getting
  File "/app/wayve/ai/language/vlm/train_docker.binary.runfiles/pip-core_deepspeed/site-packages/deepspeed/runtime/zero/partitioned_param_coordinator.py", line 204, in reset_step
    raise RuntimeError(f"still have inflight params "
RuntimeError: still have inflight params [{'id': 691, 'status': 'INFLIGHT', 'numel': 262144000, 'ds_numel': 262144000, 'shape': (32000, 8192), 'ds_shape': (32000, 8192), 'requires_grad': False, 'grad_shape': None, 'persist': False, 'active_sub_modules': set(), 'ds_tensor.shape': torch.Size([32768000])}, {'id': 692, 'status': 'INFLIGHT', 'numel': 67108864, 'ds_numel': 67108864, 'shape': (8192, 8192), 'ds_shape': (8192, 8192), 'requires_grad': True, 'grad_shape': None, 'persist': False, 'active_sub_modules': set(), 'ds_tensor.shape': torch.Size([8388608])}, {'id': 693, 'status': 'INFLIGHT', 'numel': 0, 'ds_numel': 8388608, 'shape': (0,), 'ds_shape': (1024, 8192), 'requires_grad': True, 'grad_shape': None, 'persist': False, 'active_sub_modules': set(), 'ds_tensor.shape': torch.Size([1048576])}, {'id': 694, 'status': 'INFLIGHT', 'numel': 0, 'ds_numel': 8388608, 'shape': (0,), 'ds_shape': (1024, 8192), 'requires_grad': True, 'grad_shape': None, 'persist': False, 'active_sub_modules': set(), 'ds_tensor.shape': torch.Size([1048576])}, {'id': 695, 'status': 'INFLIGHT', 'numel': 0, 'ds_numel': 67108864, 'shape': (0,), 'ds_shape': (8192, 8192), 'requires_grad': False, 'grad_shape': None, 'persist': False, 'active_sub_modules': set(), 'ds_tensor.shape': torch.Size([8388608])}, {'id': 696, 'status': 'INFLIGHT', 'numel': 234881024, 'ds_numel': 234881024, 'shape': (28672, 8192), 'ds_shape': (28672, 8192), 'requires_grad': False, 'grad_shape': None, 'persist': False, 'active_sub_modules': set(), 'ds_tensor.shape': torch.Size([29360128])}, {'id': 697, 'status': 'INFLIGHT', 'numel': 234881024, 'ds_numel': 234881024, 'shape': (28672, 8192), 'ds_shape': (28672, 8192), 'requires_grad': False, 'grad_shape': None, 'persist': False, 'active_sub_modules': set(), 'ds_tensor.shape': torch.Size([29360128])}]

now sporadically.. I don't think anything else has changed for me except upgrading lightning to 2.2. Could it be that train/val broke something else? seems like deepspeed tickets on that issue are all pointing to training/val mode.. The same indirect evidence is here https://github.com/Lightning-AI/pytorch-lightning/issues/17523

No, it has nothing to do with stage 2. What I explained applies in general in the Trainer for all strategies, not just deepspeed.

But I guess the new behavior is less "state-mutating" and hence safer on average

Yes that's the goal. I wasn't aware that huggingface .from_pretrained() is in eval mode, but retrospectively it makes sense. Since Lightning is a general Trainer, this is unfortunate for huggingface users to make that extra step but I believe in the long-term it's better and more flexible for everyone.
Regarding huggingface+lightning, there are a few other things that need to be documented. We need a page in the docs "Using HuggingFace models with Lightning" that explains these things, including the .train() thing we talked about here.

Does it mean I should do something like this
    def on_train_start(self, trainer, pl_module):
        self.llm.train()

    def on_train_end(self, trainer, pl_module):
        self.llm.eval()

The docu should also be fixed https://lightning.ai/docs/pytorch/stable/notebooks/lightning_examples/text-transformers.html.

Does it mean I should do something like this
def on_train_start(self):
    self.llm.train()

def on_train_end(self):
    self.llm.eval()

I don't think this this is necessary. I would call model.train() directly after loading the model from huggingface. You can check with assert statements in the train/val/test/predict step that lighting does the switching of train states automatically.


Get image information

Is your feature request related to a problem? Please describe.
No response
Describe the solution you'd like
The method should return image informaiton :

format
size
dimension
exif

or any information imagemgick can extract
Describe alternatives you've considered
No response
Additional context
No response

You can already get format, dimension and exif. And I will not be going to add size because you can only read from a byte array at the moment and then you can just do array.length to get the size. More information will be added when I have time to work on this project. Feel free to open an issue if you are missing a specific set of information that is available in ImageMagick but not in this library.

will dig the code and wait for doc.
Thks for your work


[WIP] Loading upfront

Investigating a way to avoid needing to have all the loaders be synchronized in multithreaded mode. Playing around with trying to do all of the loading upfront, so we could synchronize just that chunk. If it stays relatively neat, ideally this would be something we could opt into only in multithreaded mode, where it wouldn't need to be repeated per-JRuby.

I think this doesn't cover 3x functions yet.

We're taking a different direction with this for now.


Add quotes around paths in shell script call

Summary

Without this the RNTester app on iOS can’t be built in a folder that has a space in the path.
Changelog

[iOS] [FIXED] - RNTester app builds in a path that contains a space
Test Plan

Clone react-native into a folder where the path has a space

Follow the instructions at https://github.com/facebook/react-native/blob/master/RNTester/README.md to get the RNTesterPods.xcworkspace

Open the workspace and build the project. Path to workspace was /Users/richard/UX Cam/Code/Test Projects/ReactNative/react-native/RNTester/RNTesterPods.xcworkspace

See error message - '/Users/richard/UX' is the first part of my path before the space

/Users/richard/Library/Developer/Xcode/DerivedData/RNTesterPods-ctjcmrlwzntpssgmcjydmovoztwl/Build/Intermediates.noindex/RNTesterPods.build/Debug-iphonesimulator/RNTester.build/Script-68CD48B71D2BCB2C007E06A9.sh: line 3: /Users/richard/UX: No such file or directory

Is this a problem in the projects that the React Native cli generates or only RNTester?
Are you able to create a new project in a folder with a space using react-native init and successfully launch it?

A fresh project with react-native init builds fine. The equivalent build stage on that project uses relative path addressing:

I'm updating my PR to remove the unneeded 'PROJECT_ROOT' variable setting.


Past Files Table

Description
Addresses STEAM 977, adding a Past Files component. This component includes a table that displays the file name, file size, when the file was added, and a JSON viewer for uploaded mCODE bundles.
(Feature) Demo/Screenshots
Table:
https://user-images.githubusercontent.com/53443041/198703408-0c66055d-be85-49da-bac8-93557373c838.mov
Reference from Figma:

Important Changes

Adds a table to PastFiles.js capable of displaying metadata of uploaded JSON files, viewing these files, and deleting these files. Note that these files are hardcoded right now, but should be passed as props once this component is actually hooked into the rest of the web app.
Adds the PastFiles component to App.js for easy testing.

Testing Recommendations

Ensure that table styling is consistent with the Style Guide, Figma reference, and generally is appealing to the user.
Ensure that code structure is sensible and maintainable. I tried to use tailwind classes for styling where possible, but in instances where the Style Guide specified a font size value that was not included a tailwind class, I included it as custom style. If you see anything that could or should be styled as a tailwind class rather than a custom style, please point it out!
Ensure file deletion and toggling the JSON viewer works as expected.

Checklists
Submitter:

[x] This PR describes why these changes were made.
[x] This PR is into the correct branch.
[x] This PR includes the correct JIRA issue reference.
[x] Code diff has been reviewed (it does not contain: additional white space, not applicable code changes, debug statements, etc.)

@ACCT1 :

[ ] Code is maintainable and reusable, reuses existing code and infrastructure where appropriate, and accomplishes the task’s purpose
[ ] You have tried to break the code
[ ] Tested all recommendations listed in the "Testing Recommendations" section. The application behaves as expected with this PR.

@Dtphelan1 commit 665b9b7 removed all of the custom style props, all styling is now done through tailwind classes. Some things aren't exactly the same (the shade of gray used in the table for instance) but it's a pretty close approximation. Let me know if you think anything should be tweaked further!


[WIP] Send digest emails less frequently. Fixes #6114

I didn't see any tests for digests in test_management_commands.py so if other tests need to be updated let me know.

@phansen01 this looks great; I think we don't have any tests for this, unfortunately.
I think what we'll want to do eventually is refactor most of the code of this management command to live in a new function in zerver/lib/digest.py, and then unit-test that function.  If you're interested, that could be a great project to work on after this.
In the meantime, have you tested this manually @phansen01?  The change is simple enough that I'm OK merging with new unit tests, but someone should have actually run it :).
@rishig FYI.

Yeah ran a manual test just now - seems good to go.

Great, will merge shortly, after a bit of copy-editing of the comments.  Thanks for fixing this!


Add support for replicates

Support added for demuxing a sample-sheet w/replicates into n sample-sheets according to their respective plate-quadrant (since replication is plate-level).
Support added to metapool package to make GenPrepFileJob handle this use-case fairly transparently.
WIP support for handling pre-prep files left out (amplicon), but is simpler and derived from the sample-sheet implementation.

This PR includes changes from https://github.com/biocore/metagenomics_pooling_notebook/pull/131 as well. The number of files to compare should go down once 131 is merged. It has already been approved.

Closing. Superseded by manual rebase: https://github.com/biocore/metagenomics_pooling_notebook/pull/133


Add Path interpolate with pre-allocated jsi object

We tested it on @benawad example and it seemed have a substantial performance impact.

I think I have a simpler solution (as I mentioned a while back) that doesn't require us to create a new factory method for this. If you can provide me with the example you used for testing this and the expected numbers I'll do some tests.

super nice! What's missing now is how to avoid the mode="continuous" due to the path stability.
There are ways we could do this but maybe one way to go directly to the "final" solution you have built (declarative interpolators).

Yes, I know - not gotten there yet :) :)

maybe a top level prop in Canvas where we can pass values to be registered in the dep manager? not sure

maybe a top level prop in Canvas where we can pass values to be registered in the dep manager? not sure

On web this example leak in an even more dramatic fashion, so we might need to add dispose to al JsiSkInstance and review its usage in our examples? (and document it). And we should probably deprecate the current version of interpolate.

when is this gonna be merged ?

@chrfalch I would like to reopen this PR.
In conjunction with the swap method, we could update some one of the documentation example to have an appealing fast path interpolate.

@chrfalch I would like to reopen this PR.
In conjunction with the swap method, we could update some one of the documentation example to have an appealing fast path interpolate.


How to set Minimum/Maximum Time?

The question says it all.  The Min/Max Date works great but can't seem to get it to handle Time.
Your Environment

flatpickr version used: 4.5.1
Browser name and version: Chrome, but irrelevant
OS and version: Windows 10.0.17134

NEVERMIND.  Tried it Without the Date part, just using my options.dateFormat of H:i and that did the trick.  Carry on.


Merge HomePage (Current name)

This is just to merge what I've been working on with the new master branch. From what we talked on this might be changed to the "schedule upload" page or something of that sort but it is currently named HomePage.

You should set it to merge into addregister instead XD

I have many merge conflicts to fix, luckily I'm experienced with this.


Could not change user's password

Hello!
It seems like I found some unexpected behavior with changing password when profile editing does not require current password to be provided.
I implemented an alternative solution from here.
Now when I'm trying to change user's password it redirects me to root with message 'Your account has been updated successfully.' even with invalid password and password confirmation (either when there is password but no confirmation or when 'password confirmation' is not the same as 'password'). But in real password is not being changed. I could not change password even if provided password and confirmation are correct.
You can check the code here. Rails v-4.2.6, Devise v-3.4.1
Sorry for my broken English.
Regards, Ignat

You used updating user without password. So, i think password wasn't changed in this case. Please insert password field in permit line. Ex:  params[:user].permit(:first_name, :last_name, :address, :password)

@vietntframgia thanks for your reply.
That's my fail. I get it.
I looked at the docs now and it says that update_without_password method never allows a change to the current password.
Don't you think that Wiki is little bit confusing and it would be better to notice about this effect?