Make structure files available for communication with other tools

In order to use e.g. MDAnalysis in MDsrv, the structure file is requested. Please make it callable / available on request (like it is possible for remote trajectories).

feature request for the mdrsv datasource (https://github.com/arose/ngl/blob/master/src/datasource/mdsrv-datasource.js)

@j0kaso can you please describe again what we need here, on the server as well as in NGL (in the mdsrv datasource). Thanks!

MDsrv needs the trajectory and the structure file when MDAnalysis should be used. So far, the trajectory file loaded through the data source is callable by the MDsrv (via the GET/POST request as here: https://github.com/arose/ngl/blob/master/src/trajectory/remote-trajectory.js), the "location"/path of the structure file is not. Both are needed (and could come from a different location). I assume the path of the structure file would be needed to be saved (maybe difficult for drag'n'droped structures?). Afterwards, MDsrv needs to GET this location and forward it to MDAnalysis. Hope this helps.

Can you suggest an url you can consume in mdsrv that includes the required info?

similar to traj: "/traj/numframes//path:filename" --> "/struc//path:filename"
I assume it should be fine with this (could also think that further information as number of atoms, ... could also be of interest in the future and with regards to further analysis)

doesn't the structure path have to be send with every traj request?

right, otherwise it would be mixed up with other trajs... so just the info after the traj?:
/traj/frame/<int:frame>/<root>/<path:filename>/<root2>/<path:filename2>

or as a GET param?

the traj information is send via GET so it would be easier to continue with this. still together, otherwise it would be mixed up with other structures (thanks for the reminder)

changed the mdsrv datasource such that any query string from the original file path gets appended to requests for frames
original path: file://cwd/data/md.xtc?foo=bar
frame path: traj/frame/42/cwd/data/md.xtc?foo=bar

you can put anything in the query string
can you work with that @j0kaso ?

see f0c7d126ab0456f5e352b11c5fead1dcd1a6ca68

works - thanks for including it!


Minor documentation issue?

Hi,
First up - awesome work, I just started looking into this project after working with GraphQL-php for some time. I really like the OOP type declaration approach.
I come across an issue in the getting started document, which I think might just be outdated documentation?
public function build(TypeConfigInterface $config)
{
    $config
        ->addField('title', new NonNullType(new StringType()), [
            'description'       => 'This field contains a post title',
            'isDeprecated'      => true,
            'deprecationReason' => 'field title is now deprecated',
            'args'              => [
                'truncated' => new BooleanType()
            ],
            'resolve'           => function ($value, $args) {
                return (!empty($args['truncated'])) ? explode(' ', $value)[0] . '...' : $value;
            }
        ])
        ->addField('summary', new StringType())
        ->addField('likeCount', new IntType());
    $config->addArgument('id', new IntType());
}

public function resolve($value, $args, $info)
{
    return [
        "title"     => "Title for the latest Post",
        "summary"   => "Post summary",
        "likeCount" => 2
    ];
}

inline documentation says it is ObjectTypeConfig and not TypeConfigInterface.

addFields only takes 2 arguments, how is it configured then?

AbstractObjectType does'nt not contain a resolve() to override, and if I define a resolve function here, it is not called.

Hi, @larswolff, thanks for the comment, it is an outdated piece, I will check everything again to make sure it's all up to date.
To answer your question I'll have to explain the latest changes we've done here.

We now have types and fields classes. GraphQL Type can't contain resolve function. It's just a type. So we made changes to conform this architecture piece.
As a result of 1st change – we now add fields with two options:

->addField('fieldName', array $fieldConfig)
->addField(new FieldClass())

You can find working example in our examples folder – https://github.com/Youshido/GraphQL/tree/master/examples, those are tested every time before publishing a new version. Also, there's a link to gitter chat where you can ask any questions and we'll try to support you asap.

This part of documentation was updated and there's a new version of documentation that's coming along with new features.


Cloudflare Status code 524

What is the issue?
I'm running into the following error message:
Ollama call failed with status code 524. Details: <bound method ClientResponse.text of <ClientResponse(http://chat.tamdu.com/api/generate) [524 None]>
<CIMultiDictProxy('Date': 'Wed, 17 Apr 2024 07:30:15 GMT', 'Content-Type': 'text/html; charset=UTF-8', 'Content-Length': '7080', 'Connection': 'keep-alive', 'Report-To': '{"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v4?s=YbviAMSSM3LYJsWVzIaJrWEbptcEzL9UKo8lBnELb9SXTbsiBItX%2BEpnXCNzVCALyxSNNoyoTtuD49XA5Jvk7lWVXbDXX6KoblL84ABMzmC61tkChe8BxvzdInZqFg%2FJpQ%3D%3D"}],"group":"cf-nel","max_age":604800}', 'NEL': '{"success_fraction":0,"report_to":"cf-nel","max_age":604800}', 'X-Frame-Options': 'SAMEORIGIN', 'Referrer-Policy': 'same-origin', 'Cache-Control': 'private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0', 'Expires': 'Thu, 01 Jan 1970 00:00:01 GMT', 'Server': 'cloudflare', 'CF-RAY': '875ab2ff4bf8963f-SJC', 'alt-svc': 'h3=":443"; ma=86400')>
What did you expect to see?
No response
Steps to reproduce
No response
Are there any recent changes that introduced the issue?
No response
OS
Linux
Architecture
x86
Platform
WSL
Ollama version
0.1.31
GPU
Other
GPU info
No response
CPU
Intel
Other software
No response

Hi @zhaopengme, we've been working with cloudflare and the issue should hopefully be resolved. Can you confirm?

any tips on how to solve this? @zhaopengme


(fix): Mark intermittent test failure as xfail

Feature or Bugfix

Bugfix

Detail

Mark intermittent test failure as xfail

Relates

#1451

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

AWS CodeBuild CI Report

CodeBuild project: GitHubCodeBuild8756EF16-4rfo0GHQ0u9a
Commit ID: b806475dd90127a34c971b4eb3ff3f7238efa19a
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository


Add Abseil as an external dependency for testing.

The latest version of RE2 depends on Abseil. We will have to start depending on Abseil if we are going to continue to update RE2.
For now, we cannot simply add Abseil as another source dependency because RE2 uses find_package to get Abseil, and we would prefer to avoid that if possible. See https://github.com/google/re2/issues/427.
In the short term we should,

[x] Disable the auto roller or RE2.
[x] Wait for RE2 to respond to the issue above.
[x] Add Abseil as a dependency, and update RE2
[ ] Restart the autoroller for RE2.

Latest revision e0936b6 fails to build with this warning :
CMake Error at external/CMakeLists.txt:106 (add_subdirectory):
  add_subdirectory given source "abseil_cpp" which is not an existing
  directory.

Posting here seemed a better choice then opening  a new issue.

Did you rerun git-sync-deps to checkout abseil in "external" directory?

No, didn't need that command at all sofar.
I recentIy took over maintenance for the spirv-tools-git  archlinux aur package.. Until now the only external component needed was SPIRV-Headers sources which was switched to be taken from /usr .
I did re-read the readme and noticed it mentions the urls & folder locations for several external components under "getting the source" .
Will add them & abseil and report back.

Are you building the spirv-tools tests? If you are not building the tests, then you should not need abseil.

I was building the tests, disabled now by adding -D SPIRV_SKIP_TESTS=ON .
The build now finishes succesfully.
Is running the tests important for end users ?

Is running the tests important for end users ?

That is a call you will have to make. You are distributing the package.

Added tests to the aur package, thanks for the help.


Unable to display Chinese interface

I tried changing the “language” setting in the config.json file to “zh_tw”, “ZH_TW”, “zh_TW”, “ZH_tw”, “ZHTW” and “zhtw” but the interface still does not display in Chinese. Is there a way to use the Chinese interface?

Hi @doggycxj ,
You can just use the command !noalbs lang zh_tw to change the language.

Thank you very much


Alternative tmux command (i.e. byobu)

I didn't see any mention in the docs of being able to specify the tmux command.
tmuxinator lets you specify the command with
tmux_command: byobu
Is tmuxp compatible with byobu (which supports the same interface as tmux)? And if so, how is it specified? If not, consider this a feature request ;).

If you run tmuxp from within a byobu session, and select `Yes' when it asks you to switch, all is well.

It could be worth documenting if you like. If you bumped into it, I'm sure other byobu users could find it helpful. :)

I would like to call attention to this issue again. Although it is true that manually selecting Yes within a byobu session allows to run tmuxp in byobu there are a couple of caveats:

You have to manually interact with the system to create the byobu session and them answer to the prompt with a yes (although this could be solved using, for example, "yes | tmuxp load test.yml"). If you are using tmuxp to launch tasks in remote systems in a automated way this is problematic.
This creates two different tmux/byobu sessions, one with the single tmuxp command and the other launched with tmuxp.
It would be nice to have the option to use directly byobu with tmuxp if possible.

@nudomarinero @kmactavish what do you use byobu for? what does it give you over tmux/screen?

You have to manually interact with the system to create the byobu session and them answer to the prompt with a yes (although this could be solved using, for example, "yes | tmuxp load test.yml"). If you are using tmuxp to launch tasks in remote systems in a automated way this is problematic.
This creates two different tmux/byobu sessions, one with the single tmuxp command and the other launched with tmuxp.

@nudomarinero  @kmactavish Can you give me more info about how you specifically work with byobu / tmuxp?
I'm still not sure if this is an issue with byobu or tmux, or if its a bug or not

TLDR byobu looks nicer than default tmux. To use it, have a byobu session running somewhere before loading with tmuxp (don't have to load it from within to get most functionality). Official support looks tricky.
byobu gives you nice looking decorations with no configuration:

rather than the (semi stock) tmux:

As long as you start the tmux server with byobu before loading with tmuxp (i.e. have one byobu session running), you don't have to select yes after loading, and all you lose is some coloring (the decorations and key mappings are there).
It's definitely not a bug, just a mild incompatibility between tmuxp and byobu. I tried replacing the tmux command in tmuxp with byobu, but all of the parsing of command returns was messed up because byobu didn't strip ANSI color codes. It seemed like an endeavour to offer official support, so I was happy with the workaround.

P.S. If someone wanted to try to get it working, I would be happy. Might be an easy fix if I missed something obvious. You just change the tmux command to byobu in a couple of spots, and should start to see the weird ANSI behaviour.

My plan is to automatically launch commands in several remote systems within byobu (or tmux if that is not possible) sessions using tmuxp. I am developing a pipeline and some steps of the commands can fail. It is very useful for debugging purposes to enter a byobu session and see what happened, etc. I prefer tmux or byobu to screen and, for me, byobu is like tmux with sugar and nice key mappings than I am used to.
I agree this is not a bug. I thought that it could be something easy to implement given that byobu is build on top of tmux. However, after the comment of @kmactavish it is clear that it may not be easy.

The following command allows you to initialize a byobu session using tmuxp even when there is no running byobu session:
byobu new-session tmuxp load --yes SESSION_TO_LOAD

As you can see, this command creates a temporary byobu session that runs tmuxp and attach it immediately. The temporary session exists immediately after tmuxp loads the session. This workaround is a bit ugly on rendering but seems to always to work.

Thanks for posting this.  I was able to get my session to load in byobu by replacing s/tmux/byobu-tmux/ in tmux_cmd:__init__ libtmux/common.py:187.  I didn't see any side effects, other than slow load times.
about 6 seconds after replacing tmux_cmd to use byobu-tmux
$ time tmuxp load my_session
user	0m5.996s

< 1 second with regular tmux
$ time tmuxp load my_session
user	0m0.355s

< 1 second with attaching to the byobu session
$ time byobu new-session tmuxp load --yes my_session
user	0m0.062s


Em and Maria mentoring meeting notes

These are our first peer mentoring meeting notes.

Meeting note have been copied to another folder.


36 - Release readiness for v1.1.1

Changes Done

Modify README for yarn start
Add description field in package.json
Add API methods to the README

LGTM


rust -> edition 2021 and version 1.56.0

Codecov Report

Merging #198 (eb974f1) into master (f891be5) will decrease coverage by 0.00%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #198      +/-   ##
==========================================
- Coverage   88.52%   88.52%   -0.01%
==========================================
  Files          73       73
  Lines       17536    17534       -2
==========================================
- Hits        15524    15522       -2
  Misses       2012     2012

Impacted Files
Coverage Δ

dlog/plonk/src/index.rs
88.73% <0.00%> (-1.55%)
:arrow_down:

dlog/plonk-15-wires/src/index.rs
97.14% <0.00%> (+0.91%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f891be5...eb974f1. Read the comment docs.


ENH: Make Elastix and Transformix GetName() members const-correct

Fixed the location of the const keyword of GetName() member function declarations, allowing to retrieve the name from a const ElastixImageFilter or TransformixImageFilter object.

Honestly I wonder if these member functions are very useful. They just return "ElastixImageFilter" or "TransformixImageFilter", respectively.  So they might even be declared static. Maybe Kasper (@kaspermarstal) still knows their purpose...?

The method are in all the other SimpleITK messages. Mostly use in printing debug, and warning messages.


DMN Editor does not display properties of edges

In the attached screenshot we see selected edge, however the opened properties panel displays properties global properties of the diagram itself. The properties panel should in this situation either:

show edge properties
or be empty

If we check check the diagram source code, there is no edge with ID shown in the properties panel from the screenshot.
dmn-source-code

@jomarko Great finding! I believe we could keep it blank as the prior editor in a first interaction. I took a look into the DMN spec [1] to check the edge properties, on section 13.4.5 we can see that DMNEdge specializes DMNDiagramElement, enabling the possibility to add the DMNStyle to it. Also, all edges inherits from DMNElement, meaning the edge could have other optional properties, like the description and label.
[1] https://www.omg.org/spec/DMN/1.5/PDF


Verifier hangs on cfsslproc.communicate

Environment

OS / version:  Ubuntu 20.04.4 LTS (GNU/Linux 5.13.0-37-generic x86_64)
Processor architecture: x86_64  Virtual Box Version 6.1.32 r149290 (Qt5.6.2)
TPM Manufacturer: Software emulated TPM installed by ./keylime/installer.sh -s
Keylime version: 9a03bd4

Description
When starting up for the first time after a fresh install keylime_verifier appears to hang on cfsslproc.communicate
Expected behavior vs. actual behavior
With the verifier running, attempting to start the keylime_registrar results in it exiting with the following message:
Exception: It appears that the verifier has not yet created a CA and certificates, please run the verifier first.
More details in the next section.
Steps to reproduce problem
With the verifier running, I tried to start the keylime_registrar and got the following (although the  verifier is running):
2022-04-06 14:09:45.955 - keylime.registrar - INFO - Setting up TLS...
Traceback (most recent call last):
  File "/usr/local/bin/keylime_registrar", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/dist-packages/keylime/cmd/registrar.py", line 21, in main
    registrar_common.start(
  File "/usr/local/lib/python3.8/dist-packages/keylime/registrar_common.py", line 541, in start
    context, _ = web_util.init_mtls(section='registrar', generatedir='reg_ca', logger=logger)
  File "/usr/local/lib/python3.8/dist-packages/keylime/web_util.py", line 67, in init_mtls
    raise Exception(
Exception: It appears that the verifier has not yet created a CA and certificates, please run the verifier first

It seems the verifier is hung. Killing the verifier generates the following:
^CTraceback (most recent call last):
  File "/usr/local/bin/keylime_verifier", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/dist-packages/keylime/cmd/verifier.py", line 21, in main
    cloud_verifier_tornado.main()
  File "/usr/local/lib/python3.8/dist-packages/keylime/cloud_verifier_tornado.py", line 1105, in main
    context, mtls_options = web_util.init_mtls(logger=logger)
  File "/usr/local/lib/python3.8/dist-packages/keylime/web_util.py", line 57, in init_mtls
    ca_util.cmd_init(tls_dir)
  File "/usr/local/lib/python3.8/dist-packages/keylime/ca_util.py", line 151, in cmd_init
    cacert, ca_pk, _ = [ca_impl.mk](http://ca_impl.mk/)_cacert()  # pylint: disable=W0632
  File "/usr/local/lib/python3.8/dist-packages/keylime/ca_impl_cfssl.py", line 122, in mk_cacert
    stop_cfssl()
  File "/usr/local/lib/python3.8/dist-packages/keylime/ca_impl_cfssl.py", line 89, in stop_cfssl
    cfsslproc.communicate()
  File "/usr/lib/python3.8/subprocess.py", line 1015, in communicate
    stdout = self.stdout.read()
KeyboardInterrupt

So it looks like it is hung on cfsslproc.communicate(). I commented out cfsslproc.communicate() at line 89 of ca_impl_cfssl.py and the error goes away.
diff --git a/keylime/ca_impl_cfssl.py b/keylime/ca_impl_cfssl.py
index 5a0db9a..20932bb 100644
--- a/keylime/ca_impl_cfssl.py
+++ b/keylime/ca_impl_cfssl.py
@@ -86,7 +86,7 @@ def stop_cfssl():
     global cfsslproc
     if cfsslproc is not None:
         cfsslproc.kill()
-        cfsslproc.communicate()
+        #cfsslproc.communicate()
         os.system("pkill -x cfssl")
         cfsslproc = None

The registrar, verifier start without exceptions and I can register the agent using the tenant..

@sergio-correia any idea why you added cfsslproc.communicate() in https://github.com/keylime/keylime/pull/747?

@sergio-correia any idea why you added cfsslproc.communicate() in #747?

I don't remember exactly, but if I am not mistaken, it was fixing a leak reported by the cfssl (test_ca_impl_cfssl.py) unit tests.

Ok, but why can this hang after a SIG_KILL?

No idea yet. I will investigate this.

@sergio-correia no need to investigate this any further. We decided to drop support for CFSSL. See #1012


"Learn more about scanning the task output" should link directly to Problem Matchers

Problem
When running a task for the first time, you are given the following options:

Continue without scanning the task output
Never scan the task output for this task
Never scan the task output for shell tasks
Learn more about scanning the task output

The last option links to:
https://go.microsoft.com/fwlink/?LinkId=733558
Which redirects to:
https://code.visualstudio.com/docs/editor/tasks#vscode
But it would be slightly better to link directly to the section on "Problem Matchers":
https://code.visualstudio.com/docs/editor/tasks#_defining-a-problem-matcher
... because it's not entirely clear that's what's meant by "scanning task output"
Proposed Solution
There are other usages of the referral link at the top of a task definition template, so probably can't just update the redirect address.  Here are all the usages of ?LinkId=733558 in the repo
The option is created in the abstractTaskService.ts line here:
https://github.com/microsoft/vscode/blob/1.52.0/src/vs/workbench/contrib/tasks/browser/abstractTaskService.ts#L1035
{ label: nls.localize('TaskService.attachProblemMatcher.learnMoreAbout', 'Learn more about scanning the task output'), matcher: undefined, learnMore: true }

Which sets learnMore: true which, when clicked, calls the openDocumentation method here:
https://github.com/microsoft/vscode/blob/1.52.0/src/vs/workbench/contrib/tasks/browser/abstractTaskService.ts#L856-L858
private openDocumentation(): void {
    this.openerService.open(URI.parse('https://go.microsoft.com/fwlink/?LinkId=733558'));
}

Action Items
If it's not a requirement that all links go through fwlink - I can put in a PR to link to the subsection of the docs, but wanted to check here first

Looks like we have plenty of references directly to https://code.visualstudio.com. Please feel free to make a PR!


update v2 creation docs to use tfc

Current dependencies on/for this PR:

main

PR #5467   👈

This comment was auto-generated by Graphite.


MemoryAwareThreadPoolExecutor automatically calls Channel#setReadable(true)

Related to #214
I'm using OrderedMemoryAwareThreadPoolExecutor.
My pipeline:
... -> myhandler1 -> OrderedMemoryAwareThreadPoolExecutor -> ...
In myhandler1, I call setReadable(false), but OrderedMemoryAwareThreadPoolExecutor automatically calls setReadable(true) later. This resets my call.
In MemoryAwareThreadPoolExecutor.java, there's this line:
channel.setReadable(true)

You are right... We should only set it readable again if it was set non readable by the Executor itself. I will fix it later today

should be fixed.. could you retest with latest changes ?

I'm using Mac thus can't build Netty.
Is Netty SNAPSHOT published somewhere?

How about adding a simple README to
https://github.com/netty/netty
describing how to compile etc.?


[stdlib] Introduce _invariantCheck_5_1 for 5.1 and later assertions.

Inlinable and non-inlinable code can cause 5.1 code to intermix with
5.0 code on older OSes. Some (weak) invariants for 5.1 should only be
checked when the OS's code is 5.1 or later, which is the purpose of
_invariantCheck_5_1.
Applied to String.Index._isScalarAligned, which is a new bit
introduced in 5.1 from one of the reserved bits from 5.0. The bit is
set when the index is proven to be scalar aligned, and we want to
assert on this liberally in contexts where we expect it to be
so. However, older OSes might not set this bit when doing scalar
aligning, depending on exactly what got inlined where/when.

@swift-ci please test

@swift-ci please benchmark

Performance: -O

Regression
OLD
NEW
DELTA
RATIO

FlattenListFlatMap
3682
5265
+43.0%
0.70x (?)

Code size: -O
Performance: -Osize

Regression
OLD
NEW
DELTA
RATIO

DataAppendDataSmallToSmall
3000
3360
+12.0%
0.89x (?)

ObjectiveCBridgeStubURLAppendPathRef2
1260
1370
+8.7%
0.92x (?)

Code size: -Osize
Performance: -Onone
Code size: -swiftlibs

How to read the data
The tables contain differences in performance which are larger than 8% and
differences in code size which are larger than 1%.
If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.
Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview
  Model Name: Mac mini
  Model Identifier: Macmini8,1
  Processor Name: Intel Core i7
  Processor Speed: 3.2 GHz
  Number of Processors: 1
  Total Number of Cores: 6
  L2 Cache (per Core): 256 KB
  L3 Cache: 12 MB
  Memory: 64 GB


Add Monaka

Add Monaka to Data Structures / Algorithms
Project URL
https://github.com/naru-jpn/Monaka
Description
Convert custom struct and fundamental values to NSData.
Checklist

[x] Only one project/change is in this pull request
[x] Addition in chronological order (bottom of category)
[x] Supports iOS 7 or later
[x] Has a commit from less than 2 years ago
[x] Has a clear README in English

          1 Error

  :no_entry_sign:
  Found 1 link issue

Link issue by awesome_bot

Line
Status
Link

639
301
https://github.com/comyarzaheri/TuningFork redirects tohttps://github.com/comyar/TuningFork

  Generated by :no_entry_sign: danger

@dkhamsing what do you think? I think it fits more under /Core Data

@lfarah submitted category is fine

merging

@naru-jpn
Thank you for contributing to awesome-ios 😄


Write some more documentation for kilt

What type of PR is this?
/kind documentation
What this PR does / why we need it:
Document more of the functioning of kilt
NONE

Thanks for your pull request. Before we can look at it, you'll need to add a 'DCO signoff' to your commits.
:memo: Please follow instructions in the contributing guide to update your commits with the DCO
Full details of the Developer Certificate of Origin can be found at developercertificate.org.
The list of commits missing DCO signoff:

6666fba Write some more documentation for kilt

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Thanks for your pull request. Before we can look at it, you'll need to add a 'DCO signoff' to your commits.
:memo: Please follow instructions in the contributing guide to update your commits with the DCO
Full details of the Developer Certificate of Origin can be found at developercertificate.org.
The list of commits missing DCO signoff:

6666fba Write some more documentation for kilt

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To complete the pull request process, please assign admiral0 after the PR has been reviewed.
You can assign the PR to them by writing /assign @admiral0 in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To complete the pull request process, please assign admiral0 after the PR has been reviewed.
You can assign the PR to them by writing /assign @admiral0 in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: 4c0207dcb9d562a4340b1ffdba6db501049c2b7f

LGTM label has been added.  Git tree hash: 4c0207dcb9d562a4340b1ffdba6db501049c2b7f

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: fntlnz
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [fntlnz]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: fntlnz
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [fntlnz]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


Curious on expansion on this

I gotta be honest.  This is some AMAZING work you are doing here.  I am curious though how you found the patterns to extract this information.  I am wondering if we can extract even more information and add them somehow.  I know, in my case, I have an OVHD panel that has the ADIRS LEDs.  That is not exposed but would be Nice to add to realism.  Is there a hint or something you can provide that shows how you found these patterns and so forth?

You could also write me a PM on the various platforms instead of opening an issue ^^
I'm searching/scanning for the Values in Memory with a tool like CheatEngine (like the current Battery Voltage for example). Then some iterations of changing the current Value (through Cockpit-Interactions) and rescanning the found locations. In the end I have the (current!) Memory-Location where a Value is stored. I then need to find some kind of Pattern in that Memory-Region which always stays the same (like a string). That is also the Reason why finding the Information for "is dashed" on the FCU Displays is so difficult - finding the right bit in Gigabytes of Memory is near to impossible ;)
That Pattern is then what the Binary uses to find the Memory-Location of a Value: Find the Pattern and the Pattern-Address plus an Offset gives the current Memory-Location. It then just reads the Memory-Locations and stores the values formatted or directly to FSUIPC Offsets.
The Binary is / was only really needed for some Display-Contents like the FCU most importantly.
Mostly everything else can be read from the Lvars which Fenix provides, like the IR/ADR Korrys for example:

(Screenshot from my shared StreamDeck/PilotsDeck Profile)

Fenix have documented this since day one in their Knowledge-Base: https://kb.fenixsim.com/example-of-how-to-use-lvars

Hey there, no clue how to PM so I did this instead.


The question about the loss fuction

Dear author,
I have a problem about the loss function. As shown in the paper, we use clusters' probability distributions to calculate the loss, however, use the probability distributions of samples to calculate DR, in order to upgrade the threshold d. Why not use the same distributions to calculate the loss and DR?

Hi!
For the update of DR we don't actually use probability distributions. We convert the probability distributions of samples-to-clusters into hard assignments, and use them to measure the NMI between clusterings.
The reason is that, while our loss is applied in the distribution space, if we calculated DR there the confidence of sample assignments would be a factor in whichever similarity measure we chose and that would cause problems. For example: if we used cosine similarity as in the loss, the degree of similarity between clusterings would change a lot depending on the sharpness of cluster distributions (even without the "hard" assignment being any different). Therefore, any threshold a user would choose would have a different impact depending on the stage of training and assignment confidence. We found that using hard assignments to measure DR worked better, was much more stable, and also that it is much more intuitive for users
I'm not sure if that answers your question fully, please let me know if I've missed something.

Thanks, I fully get it!


LTD-625: Declaration changes at end of application

Change description
Declaration text changes
New text input to capture why user cannot disclose few details
Terms and conditions acceptance modified - user need to type "I AGREE" in the text input.

@saruniitr Mindful of the timing but can we please sneak in some tests?

I am surprised that we just changed caseworker/cases/views/main.py without affecting any tests. Might be that we are testing the view for 200 and we can cheaply add a few assert statements to that tests so that we are covered.


Problem of running test.py

It seems test.py will not extract recipe embedding features and image embedding features. The code is running for more than 4 hours, but nothing has been saved (still running). I have checked the test.py, it seems that no data fed into the model. I think there may be something wrong with the test function.
%----------------------------------------------------------------------
for i, (input, target) in enumerate(test_loader):
input_var = list()
for j in range(len(input)):
v = torch.autograd.Variable(input[j], volatile=True)
target_var = list()
for j in range(len(target)-2): # we do not consider the last two objects of the list
target[j] = target[j]
v = torch.autograd.Variable(target[j], volatile=True)
target_var.append(v.cuda() if not opts.no_cuda else v)
    # compute output

output = model(input_var[0],input_var[1], input_var[2], input_var[3], input_var[4])
%---------------------------------------------------------------------------------------------------
From the code, input_var is not updated and keeps empty. Could you help me address this?

"I have fixed the bugs in test.py. So the only problem now is the wrong test_lmdb. Could you please updated a new one thus I can download for testing as soon as possible."

I think that a similar issue was encountered in #6 but was fixed by @jingjing1. Perhaps they'd be willing to share their fix as a PR?

I think that a similar issue was encountered in #6 but was fixed by @jingjing1. Perhaps they'd be willing to share their fix as a PR?

Hi,jingjing1,
could you be willing to share your fix in test.py?
thanks very much.

Hi, DongAn,
I need to find the revised test.py file, and will send you soon.
2018-06-04 11:33 GMT+08:00 DongAn<EMAIL_ADDRESS>
Hi,jingjing1,
could you be willing to share your fix in test.py?
thanks very much.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/torralba-lab/im2recipe-Pytorch/issues/6#issuecomment-394225518,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIx6f70NbZftMcsmFdVW3vbEYTLYio8Lks5t5Kp0gaJpZM4TGUCO
.

Thank you very much！

Dear DongAn,
Add "input_var.append(v.cuda() if not opts.no_cuda else v)" right after
line 96 in test.py. And then run the scripts: (data_path is the path of
test_lmdb)
python test.py --model_path=model_e220_v-4.700.pth.tar --data_path=./
2018-06-04 12:14 GMT+08:00 陈静静<EMAIL_ADDRESS>
Hi, DongAn,
I need to find the revised test.py file, and will send you soon.
2018-06-04 11:33 GMT+08:00 DongAn<EMAIL_ADDRESS>
Hi,jingjing1,
could you be willing to share your fix in test.py?
thanks very much.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/torralba-lab/im2recipe-Pytorch/issues/6#issuecomment-394225518,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIx6f70NbZftMcsmFdVW3vbEYTLYio8Lks5t5Kp0gaJpZM4TGUCO
.

Dear DongAn,
Add "input_var.append(v.cuda() if not opts.no_cuda else v)" right after
line 96 in test.py. And then run the scripts: (data_path is the path of
test_lmdb)
python test.py --model_path=model_e220_v-4.700.pth.tar --data_path=./
2018-06-04 14:54 GMT+08:00 DongAn<EMAIL_ADDRESS>
Thank you very much！
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/torralba-lab/im2recipe-Pytorch/issues/6#issuecomment-394252458,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIx6f8LNOmeT0IIRfieOVdlVFxiH-gK7ks5t5NmvgaJpZM4TGUCO
.

Dear jingjing,
It works now,thank you very much！

Hi, did anyone also encounter this problem when running test.py? I'm not sure if all my configuration is correct or not. I installed pytorch in a python2.7 virtualenv and the unzipped files of test.tar are in data directory. Any help would be appreciated!
(pytorch-conda) kk118@graphic08:im2recipe-Pytorch$ python test.py --model_path=snapshots/model_e220_v-4.700.pth.tar
Traceback (most recent call last):
File "test.py", line 176, in
main()
File "test.py", line 30, in main
model = im2recipe()
File "/homes/kk118/im2recipe-Pytorch/trijoint.py", line 103, in init
resnet = models.resnet50(pretrained=True)
File "/vol/bitbucket/kk118/anaconda2/envs/pytorch-conda/lib/python2.7/site-packages/torchvision-0.2.1-py2.7.egg/torchvision/models/resnet.py", line 188, in resnet50
File "/vol/bitbucket/kk118/anaconda2/envs/pytorch-conda/lib/python2.7/site-packages/torch/utils/model_zoo.py", line 66, in load_url
return torch.load(cached_file, map_location=map_location)
File "/vol/bitbucket/kk118/anaconda2/envs/pytorch-conda/lib/python2.7/site-packages/torch/serialization.py", line 303, in load
return _load(f, map_location, pickle_module)
File "/vol/bitbucket/kk118/anaconda2/envs/pytorch-conda/lib/python2.7/site-packages/torch/serialization.py", line 459, in _load
magic_number = pickle_module.load(f)
cPickle.UnpicklingError: bad pickle data

Hi, nhynes,
thanks very much. It works now. In deed, the LMDB file does not contain images. So I need to give the path of testing images.

Hi @jingjing1
If by any chance you look at this, can you explain in a bit more detail what you did? If I understand correctly, you need the actual image files along with the test_lmdb to run testing, right?

Hi, did anyone also encounter this problem when running test.py? I'm not sure if all my configuration is correct or not. I installed pytorch in a python2.7 virtualenv and the unzipped files of test.tar are in data directory. Any help would be appreciated!
(pytorch-conda) kk118@graphic08:im2recipe-Pytorch$ python test.py --model_path=snapshots/model_e220_v-4.700.pth.tar
Traceback (most recent call last):
File "test.py", line 176, in
main()
File "test.py", line 30, in main
model = im2recipe()
File "/homes/kk118/im2recipe-Pytorch/trijoint.py", line 103, in init
resnet = models.resnet50(pretrained=True)
File "/vol/bitbucket/kk118/anaconda2/envs/pytorch-conda/lib/python2.7/site-packages/torchvision-0.2.1-py2.7.egg/torchvision/models/resnet.py", line 188, in resnet50
File "/vol/bitbucket/kk118/anaconda2/envs/pytorch-conda/lib/python2.7/site-packages/torch/utils/model_zoo.py", line 66, in load_url
return torch.load(cached_file, map_location=map_location)
File "/vol/bitbucket/kk118/anaconda2/envs/pytorch-conda/lib/python2.7/site-packages/torch/serialization.py", line 303, in load
return _load(f, map_location, pickle_module)
File "/vol/bitbucket/kk118/anaconda2/envs/pytorch-conda/lib/python2.7/site-packages/torch/serialization.py", line 459, in _load
magic_number = pickle_module.load(f)
cPickle.UnpicklingError: bad pickle data

Hi @JohnnyQIU0311 ,
I met the same problem, and I have solved it. I download the pre-trained model failed first time because of my limited disk space, when I run the second time, I met the problem, I solved it by deleting the incomplete model I download first and then the problem was solved. Hope it is helpful.


Fixes TileMap clear_layer not recreating internal CanvasItems

Fixes https://github.com/godotengine/godot/issues/63476
Fixes https://github.com/godotengine/godot/issues/54984

Thanks!


Create an admin login system

The admin login system will use and have the following:

MongoDB
JWT tokens (?)
Some basic UI that makes a post request to (our API) validate login

Done


WIP Add http timeout to reporter

Add http timeout to reporter

@batuhancaglayan there are reporter related failing tests


Synapse S360 Item #935323

MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Contribution checklist:

[x] I commit to follow the Breaking Change Policy of “no breaking changes
[x] I have reviewed the documentation for the workflow.
[ ] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

[x] Ensure to check this box if one of the following scenarios meet updates in the PR, so that label “WaitForARMFeedback” will be added automatically to involve ARM API Review. Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs, all “removals” and “adding a new property” no more require ARM API review.

Adding new API(s)
Adding a new API version
Adding a new service

[x] Please ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If there are following updates in the PR, ensure to request an approval from API Review Board as defined in the Breaking Change Policy.

[ ] Removing API(s) in stable version
[ ] Removing properties in stable version
[ ] Removing API version(s) in stable version
[ ] Updating API in stable version with Breaking Change Validation errors
[ ] Updating API(s) in preview over 1 year

Please follow the link to find more details on PR review process.

/azp run automation - sdk

/azp run automation - sdk

@kashyap-kunal , can you please resolve the conflicts and fix the CI check failures?

/azp run automation - sdk

/azp run automation - sdk

/azp run automation - sdk

/azp run automation - sdk

/azp run automation - sdk

/azp run automation - sdk

/azp run automation - sdk

@kashyap-kunal , please ping ARM review<EMAIL_ADDRESS>to get this PR reviewed by ARM.

/azp run automation - sdk

/azp run automation - sdk

/azp run automation - sdk

Based on the comments from Amogh and Kunal "The APIs here are all being proxied to SQL RP and thus we’ve added the corresponding Swagger on our side to be compliant with the completeness and correctness requirement"
Signing off from ARM side.

For breaking change please follow-up with the SDK team as newPath should not be a breaking change.


NugetPackageRoot not getting set after package restore.

I have a simple custom project file where I am using package reference to pull props & targets file from NuGet. This variable $(NuGetPackageRoot) is never set to global NuGet cache.  I am importing Common targets & props in my props & targets which I am trying to bring in via package reference.

MyProjName
32de6665-de71-4c85-aabc-d0696a45dbd9

PackageReference

Can you share the custom project file? Is it a sdk style project?
Why are you particularly interested in that Property?

myprojtype.zip

Attached the project file. This is a custom project type where I have my own props & targets file that I am pulling through a NuGet package & then importing it.

I figured that this is getting set if I import Microsoft.Common.Targets & Microsoft.Common.Props file in my project file. I had the targets imported in my props & targets file. I am planning to move them to proj file.


Fix FWEO-1094 - Use small QRCode when needed

Description
The goal of this PR is to fix https://ledgerhq.atlassian.net/browse/FWEO-1094, by automatically detecting that a QR Code (+description) is not fitting in the screen, and use a smaller size (132x132 instead of 264x264)
Changes include

[*] Bugfix (non-breaking change that solves an issue)
[ ] New feature (non-breaking change that adds functionality)
[ ] Breaking change (change that is not backwards-compatible and/or changes current functionality)
[ ] Tests
[ ] Documentation
[ ] Other (for changes that might not fit in any category)

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 59.85%. Comparing base (90d0dce) to head (03141ff).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #614   +/-   ##
=======================================
  Coverage   59.85%   59.85%
=======================================
  Files          12       12
  Lines        1684     1684
=======================================
  Hits         1008     1008
  Misses        676      676

Flag
Coverage Δ

unittests
59.85% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


[fuzz]: fix oss fuzz 32643, Config validation of lb_subset_config for server fuzz test

For an explanation of how to fill out the fields, please see the relevant section
in PULL_REQUESTS.md
Commit Message: In server fuzz test, misconfiguration of subset_selectors will cause the test to failed. When we set single_host_per_subset to be true and specify more than 1 key, envoy will throw unexpected exception. Since the correct configuration of subset_selectors is well documented, we can reject such misconfiguration to prevent the fuzz test failure.
Additional Description:
Risk Level: low
Testing: add config validation to server fuzz test.
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Deprecated:]
[Optional API Considerations:]

Could you please add the monorail bug as well?


Enable standard HTML 5 errors and set Error property automatically

[X] This is not a v0.x issue.
[X] I have searched the issues of this repository and believe that this is not a duplicate.

Expected Behavior 🤔
Should be able to set max, min, expression, maxlength, required on the control and when invalid should set the error prop.
Current Behavior 😯
Right now only required does anything and that only puts the * on the label.  None of the other HTML 5 validators do anything to the control itself even when inputProps are set.  Yes it blocks form post but it doesn't give visual cues.
Context 🔦
Right now the only way to do form validation with Material-UI is to use separate libraries, most of which add unwanted ceremony to something that should be trivial with these controls.  IT would be greatly desirable to only have to use material-ui and have the controls reflect their status properly automatically.

It's a common confusion. The TextField API is arbitrarily fragmented. What you're looking for is inputProps. Check out https://material-ui.com/api/text-field/#props

@eps1lon Ok, but it still doesn't seem to set the error prop and thus change the color and show an error message properly...

You need to handle form validation on your side. You might want to keep track of #15585


Access token cannot be accessed in latest chrome update

It seems that Google has changed the name of the property that holds the object with the access_token from Zi to uc.
So, in the following code snippet, authResult is undefined.
GoogleAuthService.prototype._setSession = function _setSession (response) {
  var profile = this.authInstance.currentUser.get().getBasicProfile();
  var authResult = response.Zi;
  this._setStorage(authResult, profile);
  this.authenticated = true;
};

Is anyone else experiencing this too?

I see there is an open pull request, #44, to replace the hardcoded Zi. I like this idea, any idea if we could implement this?

PR merged. Will deploy this weekend ✌

Hi, could you please push a new tag including this fix? Thanks!


Only count sub requirements towards parent when they are fulfilled

See 4 year cs degree specialization requirements. Only 3 sub requirement points count towards the total specialization points requirement.

The 'strict' keyword enables this behavior for a specific requirement, this is not enabled by default.


[wip] bitcoin politician automation

#36
How it works currently:
The entry point is the run.py file. Run with python run.py
You can check the current output at sample_house.md and sample_senate.md (should be reproducible when given the api key in member.py)
run.py first fetches all members of the current congress from https://api.congress.gov/v3 and creates a folder for each member inside /data with a json that details that member's descriptions.
Then the disclosures from both House and Senate is scraped from (disclosures-clerk.house.gov) and (efdsearch.senate.gov) respectively for a given year defined in run.py, and the disclosures are parsed and added into the json files of the members (for the ones that are directly parsable)
For the disclosures that contain pdfs/images, the images are saved inside the folder of the member.
The current outputted markdown files ignore image disclosures, hence why sample_house.md is empty.
There are two TODOs:

[ ] extract disclosure in json form from images so that it can be rendered into markdown
[ ] parse out only cryptocurrency related holdings from json
[ ] cleanup

For the first TODO, the code is already implemented in extract.py. I think a VLM (visual language model) is suited for this task. I've tried plain OCR and that was really bad, and I'm pretty sure SOTA for this stuff is just VLMs. Doing a few shot prompt asking for parsed json should do the trick. Problem is, I don't have an OpenAI API key to test and run this. Not too sure what to do here.
For the second TODO, it seems LLMs aren't the best for this... I tried gpt4 by asking if ARKK is related to cryptocurrency and it wasn't sure. (ARKK holds coinbase in it's portfolio), so I need to think of another way....

small note:
The interchangable_names.json is because often these members have/use different names when registering for member of congress and when uploading their disclosures.
For example, GONZALES, ERNEST
his name from https://api.congress.gov/v3 is "GONZALES, TONY" while his disclosures use "GONZALES, ERNEST".
I decided to get around this by writing a checker in run.py that asks if the two names are the same person which requires you to manually input the interchangeable name into that json...

https://gist.github.com/geohotstan/18ceeb2ee6a5fd965ac61c99ee6b7839
Here's the sample_senate.md since github diff doesn't like big files

Nice; making good progress!

For the second TODO, it seems LLMs aren't the best for this... I tried gpt4 by asking if ARKK is related to cryptocurrency and it wasn't sure. (ARKK holds coinbase in it's portfolio), so I need to think of another way....

To be clear, I don't think the determination of which tickers / asset names are considered "crypto" needs to be dynamic. I'm perfectly happy with it being a statically set list, because that won't require frequent updates.

Problem is, I don't have an OpenAI API key to test and run this.

I will gladly set up an OpenAI account with API access if you think that will help accomplish this goal. Just let me know the best way to securely share an API key with you.


EES-574 Support dynamic port for deps in integration tests

Right now it is not being used as travis allows static ports binding, but in GH Actions port is going to be dynamic, so it will be passed as env var to tests.
This PR adds dynamic port support to integration tests only, feature tests to be done.
One test marked as integration is simple unit test actually that does not require any deps, so build tag was removed.

+1


RFC: Reduce channel type duplication

Reduce Channel Type Duplication
We have a type named twilight_model::channel::Channel. It's an enum with three variants: groups, private channels, and guild channels. The group and private channels are fairly simple enough and contain Group and PrivateChannel structs, respectively. The guild channel variant is a little more complex and contains a GuildChannel enum, which in turn contains seven variants: category, text, voice, stage, news thread, private thread, and public thread.
Let's take a look at GuildChannel in specific here. We have seven channels, but in particular three of them are for threads. Of the others, two are similar: stage and voice channels. In fact, they're variants containing the same type! What this gives us is a total of four real "similar types" of guild channels: text, audio (stage and voice), threads, and categories. One would expect that there would be common elements between all of these. As a matter of fact, there are: they all contain a guild ID; a channel ID; a position; and an underlying type. Guild channels that aren't threads contain permission overwrites.
So, we've got a lot of duplication here between all of these types. What if we could break it down?
Current API
Our root level type looks like:
pub enum Channel {
    Group(Group),
    Guild(GuildChannel),
    Private(PrivateChannel),
}

This type in particular is mostly fine. Here's a diff of the Group and PrivateChannel types, with PrivateChannel as the base:
pub struct PrivateChannel {
+   pub application_id: Option<ApplicationId>,
+   pub icon: Option<String>,
    pub id: ChannelId,
    pub last_message_id: Option<MessageId>,
    pub last_pin_timestamp: Option<Timestamp>,
    pub kind: ChannelType,
+   pub name: Option<String>,
+   pub owner_id: UserId,
    pub recipients: Vec<User>,
}

These are some of the less common types, especially Group. Bots can't really work with groups, which leaves it as a part of our OAuth support.
Let's look at the GuildChannel type:
pub enum GuildChannel {
    Category(CategoryChannel),
    NewsThread(NewsThread),
    PrivateThread(PrivateThread),
    PublicThread(PublicThread),
    Text(TextChannel),
    Voice(VoiceChannel),
    Stage(VoiceChannel),
}

All these threads and these two Stage and Voice variants makes one wonder: are they really necessary? Particularly Stage and Voice? The same type - VoiceChannel - is stored in them. Looking at this at a glance, one would assume that when you have only the VoiceChannel type then you can't distinguish whether it's a stage or voice channel. That doesn't turn out to be the case, because all channels store a ChannelType enum looking like this:
pub enum ChannelType {
    GuildText,
    Private,
    GuildVoice,
    Group,
    GuildCategory,
    GuildNews,
    GuildStore,
    GuildNewsThread,
    GuildPublicThread,
    GuildPrivateThread,
    GuildStageVoice,
}

Because one can distinguish between stages and voice channels, we could merge them into an "audio channel".
Now let's take a look at the difference between guild text channels and public thread channels. The base is a guild text channel and the diff is how it compares to a public thread channel:
pub struct TextChannel {
+   pub default_auto_archive_duration: Option<AutoArchiveDuration>,
    pub guild_id: Option<GuildId>,
    pub id: ChannelId,
    pub kind: ChannelType,
    pub last_message_id: Option<MessageId>,
    pub last_pin_timestamp: Option<Timestamp>,
+   pub member: Option<ThreadMember>,
+   pub member_count: u8,
+   pub message_count: u8,
    pub name: String,
    pub nsfw: bool,
+   pub owner_id: Option<UserId>,
    pub parent_id: Option<ChannelId>,
-   pub permission_overwrites: Vec<PermissionOverwrite>,
-   pub position: i64,
    pub rate_limit_per_user: Option<u64>,
-   pub topic: Option<String>,
+   pub thread_metadata: ThreadMetadata
}

Looks like we've got quite a difference! Notably, permission overwrites and positions are something unique to non-thread channels, and threads contain a whole bunch of other stuff: the owner of the thread, number of members in the thread, when it archives, and so on.
With that in mind, let's now look at the difference between a public thread and a private thread, with a public thread as the base:
pub struct PublicThread {
    pub default_auto_archive_duration: Option<AutoArchiveDuration>,
    pub guild_id: Option<GuildId>,
    pub id: ChannelId,
+   pub invitable: Option<bool>,
    pub kind: ChannelType,
    pub last_message_id: Option<MessageId>,
    pub member: Option<ThreadMember>,
    pub member_count: u8,
    pub message_count: u8,
    pub name: String,
    pub owner_id: Option<UserId>,
    pub parent_id: Option<ChannelId>,
+   pub permission_overwrites: Vec<PermissionOverwrite>,
    pub rate_limit_per_user: Option<u64>,
    pub thread_metadata: ThreadMetadata,
}

We have invitable added, and now we're back to permission overwrites! News threads are the exact same as public threads.
We could keep this as-is. However, let's say Discord adds a field to stage channels that is not on voice channels. What do we do? We could add an optional field on VoiceChannel (or, possibly rename it to AudioChannel) that is never present for voice channels but always present for stage channels. That gets messy, and conflicts with why we have a bunch of different variants. We could split them, but then we have more duplication both for us and the user.
Solution 1: Merge types
Groups aren't something bots can be in and we only have it as part of our OAuth support. It's pretty low traffic. We could merge groups and private channels like this:
pub struct PrivateChannel {
    pub application_id: Option<ApplicationId>,
    pub icon: Option<String>,
    pub id: ChannelId,
    pub kind: ChannelType,
    pub last_message_id: Option<MessageId>,
    pub last_pin_timestamp: Option<Timestamp>,
    pub name: Option<String>,
    pub owner_id: Option<UserId>,
    pub recipients: Vec<User>,
}

That isn't so bad.  Less matching and duplication for users and removes a few types for us. Seems like a win-win.
Solution 2: Optional Children
We could instead use a design model where we have optional children on channels for subsets of those channel types. Let's take a look at the private channel from proposal 1 again:
pub struct PrivateChannel {
    // Only on groups
    pub application_id: Option<ApplicationId>,
    // Only on groups
    pub icon: Option<String>,
    pub id: ChannelId,
    pub kind: ChannelType,
    // On both
    pub last_message_id: Option<MessageId>,
    // On both
    pub last_pin_timestamp: Option<Timestamp>,
    // Only on groups
    pub name: Option<String>,
    // Only on groups
    pub owner_id: Option<UserId>,
    pub recipients: Vec<User>,
}

We've got fields that are only on groups, but never only on direct message channels. We could have a child struct for group data:
pub struct PrivateChannel {
    /// Additional fields that may be available if the private channel
    /// is a group.
    pub group: Option<Group>,
    pub id: ChannelId,
    pub kind: ChannelType,
    pub last_message_id: Option<MessageId>,
    pub last_pin_timestamp: Option<Timestamp>,
    pub recipients: Vec<User>,
}

pub struct Group {
    pub application_id: Option<ApplicationId>,
    pub icon: Option<String>,
    pub name: Option<String>,
    pub owner_id: UserId,
}

This makes it abundantly clear that these four fields are only possibly present for groups, and in the case of the owner_id field always present if it's a group.
Let's look at how this might look for guild channels:
pub struct GuildChannel {
    /// Data present when the channel is an audio channel.
    pub audio: Option<AudioChannel>,
    pub guild_id: Option<GuildId>,
    pub id: ChannelId,
    pub kind: ChannelType,
    pub name: String,
    pub parent_id: Option<ChannelId>,
    /// Present for channels that can have their permission overridden.
    pub permission_overwrites: Option<Vec<PermissionOverwrite>>,
    /// Present for channels that can have a customized position.
    pub position: Option<i64>,
    /// Data present when the channel is textual.
    pub text: Option<TextChannel>,
    /// Data present when the channel is a thread.
    pub thread: Option<Thread>
}

pub struct AudioChannel {
    pub bitrate: u64,
    pub rtc_region: Option<String>,
    pub user_limit: Option<u64>,
    pub video_quality_mode: Option<VideoQualityMode>,
}

pub struct TextChannel {
    pub last_message_id: Option<MessageId>,
    pub last_pin_timestamp: Option<Timestamp>,
    pub nsfw: bool,
    pub rate_limit_per_user: Option<u64>,
    pub topic: Option<String>,
}

This gives users only one enum to match, and keeps us open to some types of new conditional fields in the future!
pub enum Channel {
    Guild(GuildChannel),
    Private(PrivateChannel),
}

Looking at the GuildChannel type in particular, an advantage here is that the conditional channel fields and structs are not exclusive. A channel can both have thread data and textual data. As the Discord API evolves, this is an advantage over an enum that looks like this:
pub enum ChannelAdditions {
    Audio(AudioChannel),
    Text(TextChannel),
    Thread(ThreadChannel),
}

Solution 3: It's all just a channel anyway
We could instead just have one big Channel type:
pub struct Channel {
    /// Data present when the channel is an audio channel.
    pub audio: Option<AudioChannel>,
    /// Data present when the channel is in a guild.
    pub guild: Option<GuildChannel>,
    pub id: ChannelId,
    pub kind: ChannelType,
    pub name: Option<String>,
    /// Data present when the channel is textual.
    pub text: Option<TextChannel>,
    /// Data present when the channel is a thread.
    pub thread: Option<Thread>
}

pub struct AudioChannel {
    pub bitrate: u64,
    pub rtc_region: Option<String>,
    pub user_limit: Option<u64>,
    pub video_quality_mode: Option<VideoQualityMode>,
}

pub struct GuildChannel {
    /// ID of the guild if available
    pub guild_id: Option<GuildId>,
    /// Channels that have a parent (such as a thread or a channel within a
    /// category)
    pub parent_id: Option<ChannelId>,
    /// Present for channels that can have their permission overridden.
    pub permission_overwrites: Option<Vec<PermissionOverwrite>>,
    /// Present for channels that can have a customized position.
    pub position: Option<i64>,
}

pub struct TextChannel {
    pub last_message_id: Option<MessageId>,
    pub last_pin_timestamp: Option<Timestamp>,
    /// Present for channels that can be configured as being NSFW
    pub nsfw: Option<bool>,
    pub rate_limit_per_user: Option<u64>,
    pub topic: Option<String>,
}

What I like
I like solution 3. There's no nested enum matching involved like now (as of Twilight 0.8), and it's a flat structure. If you want textual information, you check if it's available. If you want thread information, you can check if that's available. Importantly, it's extensible. If Discord adds new fields, we can just put them there without worrying about how to work it in to the existing 11 channel types. It's just easy to use.

I dislike this. Common fields should instead be accessible through getters on the enum.

I have updated this issue with an RFC.

I personally prefer solution 4, since there's little to no difference in abstraction from having an Option<AudioChannel>::bitrate or an Option<bitrate>. It also allows us to simply #[derive(Deserialize)] instead of having to write custom deserialization, which slows down development time if we have to add a field.
An issue with Solution 3 is that it makes assumptions about what is present, as there might be a weird case where bitrate is missing from an AudioChannel, which will make deserialization fail. 4 is more flexible for changes that Discord could make at any time.

I agree with Cassandra: solution four seems like the only one capable of being in 1.0 (stable) due to Discord's lack of stability guarantee.
Solution three is awkward due to the nested optionals (Option<AudioChannel>::Option<rtc_region>), with flattened access being cumbersome: Channel.audio_channel.and_then(|audio_channel| audio_channel.rtc_region).
Solution one and two rely on field comments, to specify when they are included, like solution four but it doesn't have the same stability guarantee so they are in my mind strictly worse.
However, going with solution four necessitates new error types for missing (previously required) fields since Rust's type system is circumvented. Since this is highly useful I'd like for model moving between stable versions fast. I think there's a sweet spot for it following semver and offering these assumptions Discord doesn't promise. As long as breaking versions aren't released too fast (I think one every 3 months would be good) a surrounding ecosystem could flourish. (Yes, 3 months is a short cycle but Discord breaks regularly so devs would hopefully be used to it.)

I'm a fan of solutions 2 and 4, the only problem with 2 is a lot of potential custom deserialization that could easily break at any point.


small typo fix

This has been fixed in a PR I just merged so I'm going to close this. Thank you for pushing this fix tho!


[Upstream] depends: zeromq 4.3.5

First new point release of zeromq in two and a half years. Mostly bug fixes; the project also completed a relicense to the "Mozilla Public License".
See https://github.com/zeromq/libzmq/releases/tag/v4.3.5.

from https://github.com/bitcoin/bitcoin/pull/28627

Mist that sorry guys.


Review 00, 01, 02, 03 Readme, text plus ensure is working fine

Assigned to @dailymp

Daily sorry it was on hooks folder, the new samples


Support node

Node support has always been in mind, since the debugger features we use are in V8 it should be fairly straight forward altho, not entirely non trivial as things like the compiler pipeline would need to be overhauled.
Opening this issue for R&D, this is very unlikely to go in before 1.2, the plan has always been to make the chrome support rock solid before adding support for any more. (Electron, chromium etc, are just variants of Chrome)

Discussion was continued in #145


Bump base and template-haskell for GHC 9.6

Extend CI to 9.4 and 9.6; better cache keys, bump actions.
Workflow run at: https://github.com/andreasabel/vector-th-unbox/actions/runs/4442558688
Revision 4 published at: https://hackage.haskell.org/package/vector-th-unbox-0.2.2/revisions/
Fixes #11

#11

Thank you for your pull request.


non material nodes can now have characteristics stored in GUS

Simple fix allows non-material nodes to have Characterisics.
Loaded just one dataset at the moment, but everything is loaded correctly (except attributes of Genotypes and Phenotypes, which is the next issue to tackle).  Confirmatory SQL below and a PDF of the results attached.
 :-(
When I re-run the workflow another time, but this time with this bug in the CBIL parser fixed
https://github.com/VEuPathDB/CBIL/pull/1/commits/1d87295a82398aa979d5a18e5bb60996938fd7c5
and also with Phenotype and Genotype fields exported from Chado as Characteristics, not ISA-attributes, some of the assays don't get Characteristics loaded for them :-(  (have checked in raw Characteristics table also)
Attached is the output
loaded-chars-v2.pdf
similar to the previous attachment
check the PAN with name like 'site_b_gambiae_sl.dt.IR_WHO.perm%'
on page 3.  they have "null null null"

Something weird is going on.. :-(

duh - forgot to check the workflow logs - this step actually failed due to a missing ontology term.
I'm used to things not being committed to the database (in VB-land) if there is a cock-up along the way.

Ugh - actually it wasn't missing ontology terms - that was an old error message for when this particular workflow step was run in November...
Actually it's a spurious
ERROR: sqlldr returned exit status 512 at /home/bmaccallum/gus_home/lib/perl/ApiCommonData/Load/Plugin/InsertInvestigations.pm line 319.
which I've seen before from time to time
will raise on the appropriate slack channel

For posterity: InsertInvestigations plugin logs activity in WORKFLOW_DIR/steps/[step]/step.err. The plugin runs sqlldr which writes its own log in /tmp/[random string].dat.log (and *.dat, *.bad, *.ctl) where [random string] can be found in step.err. In this case it was trying to load rows with primary key (CHARACTERISTIC_ID) that already existed. This can happen if the step is not cleaned up after failure, or if multiple steps are simultaneously trying to load into the table. One can limit the number of steps of this type (stepLoadType) as per ReFlow documentation; in this case the step is in loadDataset.xml with type "plugin", so you would add "plugin=1" in WORKFLOW_DIR/config/loadThrottle.prop to run them one at a time.

All good now. Will merge this into master now.


Improving converters for Select/SelectV2 operations.

The converters for Select/SelectV2 operations have been improved to cover some additional DynamicShape mode test cases.

@bixia1 : This is a replacement for PR# which was unexpectedly closed when I tried to resolve the merge conflicts.


skip grouping level and show full shortcut

I have configured which-key without labeling my shortcuts because I don't really care, it is enough for them to show as the related command so I can look at them when I need to.
I find myself wanting to disable "layers" for shortcuts and show them directly at once.
I will explain myself.
I have shortcuts like \rml and \rtl that show this way after pressing <leader>r
m -> prefix
t -> prefix

instead I would like
ml -> ros message list
tl -> ros topic list

So: is it possible to disable a level globally or group based to get all the shortcuts that require more than 2 keys shown once the first key is pressed?

May be somehow related to #90

that enhancement would probably also do what I am asking for

Is there a known workaround to achieve this behavior?


Updated to work with v2 api

Hello. I updated the api url's to v2 endpoints and changed the "records" to "tickets", as pointed out in https://github.com/Andre0512/lidl-plus/issues/4

Hey, can't test at the moment but looks like @enteryournamehere research, thanks! 👍


Error: The class 'TextMessage' is abstract and can't be instantiated

I can't build the app because of this bug.
The classes 'TextMessage', 'FileMessage', 'ImageMessage' and 'CustomMessage' are abstract and cannot be initiated. See attached screenshot, package did not pass static analysis because of the issue.

Updated!


Content Suggestion: Satstreet

Link to Content: https://www.satstreet.com/
Briefly Describe Content: Satstreet is a new custodial solution for gifting Bitcoin. You can send as little as 1 sat to any email address.
Suggested Section: Custodial Bitcoin Wallets

Service is currently bitcoin-only, but per their terms, they appear to be open to adding altcoins:
"5. Satstreet Services
Satstreet provides a way to store, use and manage Bitcoin and any other supported Digital Assets designated by Satstreet. At present, Bitcoin is the only supported Digital Asset, although we reserve the right to add other assets over time."

Duplicate


Pokemon 95 style react

Hi, not sure what's the problem but you can check the Windows 95 UI components here https://react95.io


Edit bitrise CI to only test build validity

We should have a handy way of directly retrieving app artifacts without generating it ourselves.
Bitrise CI has already been configured, but is currently failing. The automatic testing we made is most likely too ambitious for the team resources, so we should adapt it so it only builds the app.
(as a bonus, we can see if theemre is a way to make a semi automatic playstore deployment, but chances are this would be too costly)

Now that I went through the release process again, it doesn't seem that useful to automate the deployment process.
I just changed the issue, the goal is to remove the cumbersome testing and only keep CI as a way to test if the app is correctly built.


重构neovim配置

Lazyvim + Nvchad
https://github.com/NvChad/NvChad/tree/v2.0

drop null-ls https://github.com/jose-elias-alvarez/null-ls.nvim

https://github.com/nvimtools/none-ls.nvim


Missing Table Binding

There doesn't seem to be a way to bind to a table. Previously there was the TableAttribute in the Microsoft.Azure.WebJobs.Extensions.Storage assembly that allowed direct access to an Azure Table.
Is there another way to achieve this? When using the TableInputAttribute in the following way
[Function("Bands")]
 public async Task<HttpResponseData> Execute(
            [HttpTrigger(AuthorizationLevel.Anonymous, "get")] HttpRequestData req,
            [TableInput("Requests", Connection = "AzureWebJobsStorage")] CloudTable table,
            FunctionContext executionContext)
        {
            var tableEntity = new PayloadTableEntity(req.ReadAsString());

            await table.ExecuteAsync(TableOperation.InsertOrMerge(tableEntity));
...
        }

This exception is thrown:
System.Private.CoreLib: Exception while executing function: Functions.Bands. System.Private.CoreLib: Result:
Failure
Exception: Microsoft.Azure.Functions.Worker.Diagnostics.Exceptions.FunctionInputConverterException: Error converting 1 input parameters
for Function 'Bands': Cannot convert input parameter 'table' to type 'Microsoft.Azure.Cosmos.Table.CloudTable' from type 'System.String'.
[2021-04-21T07:11:43.727Z]    at Microsoft.Azure.Functions.Worker.Context.Features.DefaultModelBindingFeature.BindFunctionInput(FunctionContext context) in D:\a\1\s\src\DotNetWorker.Core\Context\Features\DefaultModelBindingFeature.cs:line 70
[2021-04-21T07:11:43.729Z]    at Microsoft.Azure.Functions.Worker.Invocation.DefaultFunctionExecutor.ExecuteAsync(FunctionContext context) in D:\a\1\s\src\DotNetWorker.Core\Invocation\DefaultFunctionExecutor.cs:line 37
[2021-04-21T07:11:43.731Z]    at Microsoft.Azure.Functions.Worker.OutputBindings.OutputBindingsMiddleware.Invoke(FunctionContext context, FunctionExecutionDelegate next) in D:\a\1\s\src\DotNetWorker.Core\OutputBindings\OutputBindingsMiddleware.cs:line 13
[2021-04-21T07:11:43.733Z]    at Microsoft.Azure.Functions.Worker.GrpcWorker.InvocationRequestHandlerAsync(InvocationRequest request, IFunctionsApplication application, IInvocationFeaturesFactory invocationFeaturesFactory, ObjectSerializer serializer, IOutputBindingsInfoProvider outputBindingsInfoProvider) in D:\a\1\s\src\DotNetWorker.Grpc\GrpcWorker.cs:line 169
Stack:    at Microsoft.Azure.Functions.Worker.Context.Features.DefaultModelBindingFeature.BindFunctionInput(FunctionContext context) in
D:\a\1\s\src\DotNetWorker.Core\Context\Features\DefaultModelBindingFeature.cs:line 70
[2021-04-21T07:11:43.736Z]    at Microsoft.Azure.Functions.Worker.Invocation.DefaultFunctionExecutor.ExecuteAsync(FunctionContext context) in D:\a\1\s\src\DotNetWorker.Core\Invocation\DefaultFunctionExecutor.cs:line 37
[2021-04-21T07:11:43.737Z]    at Microsoft.Azure.Functions.Worker.OutputBindings.OutputBindingsMiddleware.Invoke(FunctionContext context, FunctionExecutionDelegate next) in D:\a\1\s\src\DotNetWorker.Core\OutputBindings\OutputBindingsMiddleware.cs:line 13
[2021-04-21T07:11:43.740Z]    at Microsoft.Azure.Functions.Worker.GrpcWorker.InvocationRequestHandlerAsync(InvocationRequest request, IFunctionsApplication application, IInvocationFeaturesFactory invocationFeaturesFactory, ObjectSerializer serializer, IOutputBindingsInfoProvider outputBindingsInfoProvider) in D:\a\1\s\src\DotNetWorker.Grpc\GrpcWorker.cs:line 169

Hi @asparrowhawk, you tried using [TableInput] but [TableOutput] might help you in this scenario. The following example shows how this would work, it has multi output so it is a bit bloated.
Another option if you would need to directly interact with the table, would be to use DI to inject some helper in your functions class.
    public class TableFunction
    {
        [Function(nameof(HttpToTableFunction))]
        public async Task<MyOutput> HttpToTableFunction([HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = "test-table-output")] HttpRequestData req)
        {
            var tableEntity = new MyTableData { PartitionKey = "test-table-output", RowKey = Guid.NewGuid().ToString(), Text = "some text" };

            var httpResponse = req.CreateResponse();
            await httpResponse.WriteStringAsync("Success");

            return new() { HttpResponse = httpResponse, MyTableData = tableEntity };
        }

        public class MyOutput
        {
            public HttpResponseData? HttpResponse { get; set; }

            [TableOutput("TestTable", Connection = "AzureWebJobsStorage")]
            public MyTableData? MyTableData { get; set; }
        }

        public class MyTableData
        {
            public string? PartitionKey { get; set; }
            public string? RowKey { get; set; }
            public string? Text { get; set; }
        }
    }

//from: https://github.com/oscarvantol/azure-functions-dotnet5-examples/blob/main/ExampleFunction/TableFunction.cs

@oscarvantol Many thanks for the response. That was very useful with my scenario. Lets hope the documentation for these bindings improves.

What is the way if I would like to manipulate the binded output (or input) CloudTable's data?
Currently I got "row exist" exception if I want to "owerwrite" a row's value.
I couldn't find any example how can I remove the old one or update an existing one.

@brettsam Can we control what kind of operation is performed when sending the data to the output?
Such as InsertOrMerge, InsertOrReplace etc

@brettsam Same problem here.
I'm using the ".NET 5 isolated functions", "Azure Functions Core Tools v. 3", and Microsoft.Azure.Functions.Worker.Extensions.Storage v. 4.0.4. I have added the ETag column and I have set it to *, but I can't update the Table Storage row, I get the error:
System.Private.CoreLib: Exception while executing function: Functions.AddCustomer.
Microsoft.Azure.WebJobs.Host: Error while handling parameter _binder after function returned:.
Microsoft.Azure.WebJobs.Extensions.Storage: 0: The specified entity already exists.
Time:2021-09-29T17:40:46.9093149Z (HTTP status code 409: EntityAlreadyExists. 0:The specified entity already exists.

Can we update or delete row using function bindings?

Following up on this the Table attributes are missing from Microsoft.Azure.Functions.Worker.Extensions.Storage version 5.0.0-beta.4. Are there plans to add them back? And will they make use of managed identity auth?
Meanwhile I believe I can use TableServiceClient from  Azure.Data.Tables.

@asparrowhawk @MarGraz @cleferman I wrote up a bug to get this fixed.  https://github.com/Azure/azure-sdk-for-net/issues/25694
This is nuts with an unannounced, unplanned massive breaking change.  Really frustrating with non-sense like this.
I just tried the Microsoft.Azure.Functions.Worker.Extensions.Storage stable version 5.0.0 and it's gone.

Is there any possibility TableOutput and TableInput are going to be supported again in the future? If not, we have to change all our functions like @longtimedeveloper proposed.
Why was it even dropped? The whole idea of Functions is to only focus on business logic and outsource the data retrieval/storage stuff as much as possible, right?

@brettsam Is there anywhere that the removal of these attributes was announced with reasons why?
I can't find anything anywhere warning of the removal or release notes for the new version. It's not a great experience for users, any idea what's going on?

Tables was initially missing in the v5 extension. It's being added here: https://github.com/Azure/azure-sdk-for-net/issues/25342
Once the preview is on NuGet, we can ship a corresponding worker package.

Hi @anthonychu,
It seems the previes is on NuGet:

Microsoft.Azure.WebJobs.Extensions.Tables 1.0.0-beta.1 is out on NuGet
https://www.nuget.org/packages/Microsoft.Azure.WebJobs.Extensions.Tables/1.0.0-beta.1

https://github.com/Azure/azure-sdk-for-net/issues/25342#issuecomment-1013301396

Hmm, still missing the TableOutput attribute in Microsoft.Azure.Functions.Worker.Extensions.Storage?
What is the preferred  workaround at the moment?
Would be great to have it back.
Would it be possible to use CosmosDb workerextension and target table storage some way maybe?

@Hipster74 see my above comments.  It's very easy to change the code to use the APIs without the input binding.  Hope this helps.

Ok, thanks @longtimedeveloper , yes, I have already injected a TableService client in DI for other purposes so should be an easy fix to use it for function output as well.
Just feels a bit backwards since it is available in the samples :-)

https://www.nuget.org/packages/Microsoft.Azure.WebJobs.Extensions.Tables/1.0.0 is now a GA version, and we should have a similar package here, too. It should not be housed under Microsoft.Azure.Functions.Worker.Extensions.Storage, but rather something like Microsoft.Azure.Functions.Worker.Extensions.Tables. The readme for the Storage package (5.x version) on nuget.org should be updated to help redirect.

@mattchenderson - may I ask when the expected GA is for Microsoft.Azure.Functions.Worker.Extensions.Tables?


detect-crop doesn't work on Lord of rings 3

I ripped a DVD of Lord of the rings 3 and the mkv file is fine for playback but detect-crop only shows 3 dot-lines. It works in general when I supply another film, but only with this one (I also tried renaming it, but it wasn't the name), it doesn't work.

@JMoVS I'm sorry but I don't really understand what the problem is here. From your brief description it sounds as if you're using your transcoded file as input to the detect-crop tool. Which makes no sense.
Is detect-crop hanging? Does it just not produce any output? I can't diagnosis this without more information.
So, please be very explicit about the steps you're using and provide me sample output from detect-crop tool that you think looks wrong.
Thanks.

@JMoVS BTW, you can get more diagnostic information from detect-crop by using the --verbose option. And if you use that option twice, i.e. --verbose --verbose, you get debugging output. The shortcut to applying that option twice is -vv so, try this:
detect-crop -vv "/path/to/video.mkv"

@JMoVS Based on our out-of-band conversation, it seems like the mplayer tool, which is called by detect-crop, is somehow stuck scanning your input .mkv file. I suspect the file may be corrupt since I've never seen behavior like this before. And, unfortunately, there's not much I can do about that.
Sorry. :(
I'll close this now but feel free to continue adding comments.


Unable to locate library dec.

Hi,
When opening the file lattice.v with coqide (8.9.1) I obtain the error message
Unable to locate library dec.
Regards
Jens

Hi @jensgerlach,
Thanks for the heads up! I've pushed the missing file to the repo.


Use :rack_test as default capybara driver

The capybara documentation was suggesting that if the app was working without JavaScript we could use the :rack_test driver which is smaller and faster, and simply configure a Capybara.javascript_driver and use the js: true annotation on specs that need to use the javasdript_driver.
It's not a lot faster, but on my computer it's 4-5 sec faster, which is not bad, tho.

I like this! Great job.


WS2812 SPI LED strip driver produces bad SPI data

Describe the bug
The WS2812 SPI LED strip driver
seems to produce data with large gaps between SPI words and produces SPI words that are too long.
To Reproduce
Run the WS2812 sample code on an nRF52840 based board (and possibly others). You should see gaps between SPI words and also see SPI words that are too long.
Expected behavior
An 8 bit long SPI word with no gaps between words.
Impact
We're using this driver on the ZMK keyboard firmware. This issue results in flickering of the LEDs, which is a big annoyance.
Logs and console output
We have a few oscilloscope readings which show both some gaps between SPI words as well as SPI words that look to be 11bits in length rather than 8bits. https://imgur.com/a/iAFWzPA
The first image shows the length of an SPI word is about 11 bits (4MHz is about 250ns per bit, length of word from oscilloscope shows 2,760ns). The 4th image shows a big gap between the 2nd and 3rd bit. It also shows a few smaller but still possibly problematic gaps between SPI words. These are from ZMK. The last 4 signify the sample from Zephyr. Here it seems that the length of an SPI word is about 12 bits.
Additional context
This is occurring on a nice!nano, which is an nRF52840 based board as well as a couple other nRF52840 based boards. Interestingly, not all users have found this results in the LEDs flickering. We've speculated that some WS2812 variants are able to deal with these SPI inconsistences better than others. We've also noted that another
different keyboard firmware using GPIO bitbanging does not have this same issue using the same hardware.

Thanks for filing the report and for all the screenshots!
Have you tried hooking up the scope to the SPI clock pin as well? That may help diagnose. It's also a bit hard to understand what some of the other pictures in the imgur link are trying to show without context in the issue. Are any of the T0L or T1L pulses out of spec or otherwise getting close to Trset?

Hi Nicell,
For the nRF52840 you have multiple SPI driver choices, which one are you using?  You ideally should be using the SPIM driver so each word transfer is off loaded to DMA.

@nixward, thanks for the suggestion! Switching to SPIM has indeed fixed our issues for those who were affected. I'm assuming the CPU was busy for a few µs, which caused the strip update to get reset in the middle of transfer.
I'd like to note the fact that the LED strip sample uses SPI rather than SPIM for the nRF52dk. Should this be changed or noted as a possible fix? This issue doesn't seem to be present in the sample program, which again I'm assuming is because there's nothing else going on to make the CPU busy.

I'd like to note the fact that the LED strip sample uses SPI rather than SPIM for the nRF52dk. Should this be changed or noted as a possible fix?

Ah, yep, I was unaware this was defaulting to SPI instead of SPIM on this SoC. Thanks, I've sent #29984 to address this.
Please make sure you understand PAN 58 before using this in ZMK. It should be fine as long as the SPI peripheral is only being used for LEDs and not some other chip on the same bus.


Minimum Kafka cluster version

Search for existing answers
Many questions have been asked before. Please have a quick search to see if there is already an answer to your question. There are two main resources for finding answers:

Existing Jikkou Github issues, including closed issues

If you can't see an answer to your question, then please raise a new Github or Stack overflow issue.
Provide details of the setup you're running
Let us know what version of Jikkou you're running, what OS, Docker version etc,
Outline your question
I couldn't find a compatibility matrix for supported Kafka versions.
I am running Kafka 2.8 and only need acl support from Jikkou atm.
Thanks!

Hi @haunted-pancakes, Jikkou relies on the Admin Client Kafka API which is compatible with Kafka 2.8.
https://streamthoughts.github.io/jikkou/docs/user-guide/kafka/compatibility/

Thank you very much!


python: deadlock when importing modules that register tasks

We are observing a deadlock with python processors when running on multiple nodes that goes like this:

thread 1 on the python processor is running a task (the top level task?) that imports a python module.
as part of that import, legion tasks are registered, which causes a task registration to be queued up on the local python processor, and then that task waits on an event related to inter-node communication in the legion code
the thread scheduler sees that thread 1 wants to wait on an event, so it suspends that thread and spins up a new thread (thread 2) to try to use useful python processor work
thread 2 sees that there are pending task registrations, so starts to work on the first one of those.
part of task registration is importing the python module (I don't think we test that it's already present), and that import blocks on the python interpreter's module import mutex (https://docs.python.org/2.7/library/threading.html#importing-in-threaded-code).
this block is not done with realm's knowledge, so now we have thread 2 waiting for thread 1 to finish its import, and thread 1 waiting for thread 2 to finish or sleep so that it can resume execution and finish its import

There are a number of possible things we could consider doing to address this, but it's likely they all have drawbacks, so let's discuss them here:

Eliminate the blocking wait in the Legion task registration path so that thread 1 never gets suspended [@lightsighter to explain the drawbacks here]
Change how task registration works in python bindings to defer the actual registration until any imports are complete. [@elliottslaughter to comment here]
Somehow poke into the python interpreter and understand if the import lock is held by a task that is about to suspend, and if it is, spinwait instead.
Change the execution model of a Realm LocalPythonProcessor to allow tasks to run concurrently.  Any time a python task suspends and leaves zero active workers, another worker is started as before.  The difference is that blocked workers would be resumed immediately rather than waiting for the active worker to complete or suspend.  This would complicate the performance model for python tasks (i.e. you wouldn't be able to count on exclusive access to an execution resource).
Create a "scheduler lock" for a Realm LocalPythonProcessor and expose it as part of the python bindings.  Python code that takes any python-interpreter-implemented mutex could be wrapped in a with realm_python_scheduler_lock:, causing that task to spinwait rather than yield the execution resource if/when it waits on events.

As to solution 1, that's not the only place this happens. @elliottslaughter can also cause the hang doing any number of other things such as setting semantic values or registering any number of other kinds of resources with the runtime. I also think it's even worse than that, pretty much anywhere that the runtime takes a Realm fast reservation could potentially result in a preemption of the thread and therefore a hang. It's going to be very difficult to make all the potential paths that the python interpreter could call at import time lock-free.
I've always disliked that Python could call into the Legion runtime after starting the runtime but not inside of any legion tasks. I feel like the best answer in this case is solution 2, that modules just need to track what they've registered and only register it lazily on demand inside of actual tasks and not inside of their import statements.

I think some clarifications are in order.
The scenario in which we're freezing is one where we are using DCR on the top-level task legion.legion_main. This is a built-in task whose job it is to call exec on the actual, user-defined script. The whole goal of this process is to make it appear like a normal Python execution (i.e. how python user.py would behave), where the script-style execution of user.py is DCR'd.
This has several implications:
a. This is not happening in the "started-but-not" uncanny valley that plagued us before. The top-level task has started.
b. The module for user.py is not a real module and therefore cannot be imported normally. The module is called __legion_main__ (following the convention on Python's __main__) and you can only look it up in sys.modules; you cannot just do a raw import on it. I assume Python's import logic has some smarts to deal with this, because otherwise things would be even more wildly broken than they are now, just something to keep in mind.
c. The more important implication is that we must block in task registration. To do otherwise would mean that some shards in the DCR'd top-level task could run ahead of others and get a task launch through, and in theory the mapping and execution of those tasks could race with registration on other nodes.
Point (c) reminded me that I was supposed to add a barrier to block execution of the user script around task registration. However, an initial patch to do that makes the situation worse so I think at the moment I'm just going to ignore this aspect. I will at least put the patch in a branch so we have it.
I think this means (1) can't work for us. (2) is similarly not a solution; we're in the top-level task. I don't know about (3) through (5).

I still think (2) is a solution. You can still defer the runtime calls from inside the import statement until you get another call into the library at which point you check for any deferred runtime calls and then perform them.

Here's the code we're running. We don't call import, and we don't hold any locks:
https://github.com/StanfordLegion/legion/blob/master/bindings/python/legion.py#L1527-L1537
Therefore I don't see how (2) is applicable.

Ok, so how did we get into import code? We're definitely inside an import statement when we hang.

I guess what must be happening is that user.py is importing another module, and that module is transitively registering a task.
I'm still confused about (2) though. We're not running in a mode where there is a clear boundary between "all imports" and "all execution"; these are interleaved. Unless I can override Python's import mechanism or add a callback that occurs when an import completes, I don't see how to do this.
Also, I'm not sure this is likely to be an issue, but it's worth mentioning because it will violate Python's normal semantics: it will be impossible to invoke any tasks at the top-level of a module.

Ok, here's a proposal:
We could add a new callback in the Realm Python module. When loading legion.py this would be auto-populated with a Python function that would handle the deferred registration of tasks. Here are the steps that would be followed during task registration:

From user code: define a task with @task

legion.task puts the user's task into a list to be processed later

From the Realm Python processor main loop: calls legion.py registration callback

The registration callback completes any registrations (via register_task_variant)
The Realm Python processor main loop then receives the task registrations and processes them as normal
Note that for safety, the Realm Python Processor has to call the legion.py registration callback again, because if any imports occurred there could be more tasks waiting to be registered
Repeat until convergence
Finally call the next task in the queue

I guess the two questions are how does Realm get the call-back registered and how often does Realm have to invoke this call-back?

@lightsighter , you said above that there are other things a legion task can do that will cause blocking waits.  If so, then deferring only the task registrations (i.e. #2 or the soon-to-be-copied-up #6) wouldn't be sufficient, would it?

Actually, wait, this is going to run into exactly the same freeze we're already hitting. Because Realm spins up a new thread whenever the Python processor is blocked, we're just going to call the callback and cause the freeze. The real root of the condition is holding the import lock while blocking in Realm, and my approach doesn't fix that.
Is there a reason why we couldn't do something like: processes deferred task registrations only if there are > 0 tasks in the queue? That way we'll still spin up the thread, but it won't actually attempt to grab the import lock unless there is work in the queue, which we can avoid as long as you don't call any tasks at the top level of a module import. That sure sounds like a footgun, but it was exactly the same footgun I was proposing before, so we aren't actually losing any robustness in that respect.

Yes, all runtime calls need to be deferred, not just task variant registration. Anything that can call into the runtime can potentially attempt to take a Realm fast reservation and block the thread if there is contention.

Ok, the current consensus is to do proposal #5 (the scheduler lock), along with some monkeypatching of the Python bindings for the Legion API to wrap the various registration tasks (@lightsighter to provide a list) with automatic locking of the scheduler for the duration of that call.

Here is a first pass on a list of API calls that I think would be called inside an import statement and could potentially cause bad behavior:
MPI Interop: https://gitlab.com/StanfordLegion/legion/blob/master/runtime/legion.h#L6089-6118
Semantic Values: https://gitlab.com/StanfordLegion/legion/blob/master/runtime/legion.h#L6120-6423
Printing: https://gitlab.com/StanfordLegion/legion/blob/master/runtime/legion.h#L6426-6446
Mapper Registration: https://gitlab.com/StanfordLegion/legion/blob/master/runtime/legion.h#L6448-6514
Projection and Sharding: https://gitlab.com/StanfordLegion/legion/blob/master/runtime/legion.h#L6517-6619
Layout Constraint Registration: https://gitlab.com/StanfordLegion/legion/blob/master/runtime/legion.h#L6988-7045
Task Variant Registration: https://gitlab.com/StanfordLegion/legion/blob/master/runtime/legion.h#L6988-7045

I think the burst of commits ending in c2f4c9d takes care of this.  @elliottslaughter , can you think of anything that still needs doing here?

I need to check whether the workaround at https://upc-bugs.lbl.gov/bugzilla/show_bug.cgi?id=3908 is still necessary, but otherwise I think we're good at the moment.

I don't think we should remove that workaround.  The problem with forks occurring while active messages are being sent is real, and it may be that changes to the python startup cause it to stop reproducing the fork issue, giving us a false sense of security.