Discrete Control tests based on realistic model

At the time of writing controlling node types other than Pump has been merged (https://github.com/Deltares/Ribasim/pull/424), and making nodes active and inactive is on its way (https://github.com/Deltares/Ribasim/pull/425). These features do not have proper tests yet, but they came about form a discussion about a model of the Netherlands. So, this issue is for

[ ] Constructing said model;
[ ] Testing discrete control on non-pump nodes and node activity.

For the details of this model see below.

@gijsber thanks for the information. I have some questions:

Are the areas equal to the totalwidth tomes the length?
What are the parameters of LinearResistance (3, 4, 11), TabulatedRatingCurve (8, 13), PidControl (20), LevelBoundary (16), Pump (14)?

Other points:

Time dependent flow boundaries are not yet supported: https://github.com/Deltares/Ribasim/issues/439
See here for the current draft script of the model: https://github.com/Deltares/Ribasim/blob/8c1d126ee9be52bd27b8dd7a7be74c1e0b6fd1a1/python/ribasim_testmodels/ribasim_testmodels/dutch_waterways.py

Are the areas equal to the totalwidth times the length?
yes
Here are some best guesses (actually this is a task for Hydrologic):
LinearResistance: 5000 (assumption)
LevelBoundary: 0 (assumption)
Pump: min 0, max 50, default setpoint 25 (assumption)
PID: Kp: -0.005, Ki:0, Kd: -0.002 (from SBK3 model for gate)
QH: 7.45m Q=418m3/s
elke centimeter stijgt Q met 2.15m3/s

@gijsber I have distilled the discrete control of nodes 17 and 18 as follows.
Notation

q: Flow at edge (1,2), flow of flow boundary with id 1
w: Flow at edge (6,8), used to indicate whether node 8 is active

Conditions

q > 250
q > 750
q > 800
w > 0

Control states

"pump_low":

pump (node 9) active
pump (node 9) flow rate 15
TabulatedRatingCurve (node 8) inactive

"pump_high":

pump (node 9) active
pump (node 9) flow rate 25
TabulatedRatingCurve (node 8) inactive

"rating_curve":

pump (node 9) inactive
TabulatedRatingCurve (node 8) active

Logic
To massively reduce the number of logic table entries needed, I want to introduce the wildcard A  for any as an addition to T  and F for true and false respectively.

FAAA -> "pump_low"
TFAA -> "pump_high"
ATFF -> "pump_high"
ATFT -> "rating_curve"
AATA -> "rating_curve"

Final notes

I am aware that using the flow condition on edge (6,8) is an ugly thing to do, but the problem is that this 'memory for control' that we discussed for 'being on your way to a setpoint' is a different type of condition than the ones currently supported and fitting in the framework of VectorContinuousCallback.
This requires an introduction of the wildcard A and possibly some validation on the logic data. Note though that not all truth states have to be covered because many are impossible, for instance FTAA.

Hi @SouthEndMusic, That looks pretty good. We miss one condition (q>275). If we add that it would become:
Conditions

'q > 250'

'q > 275'

'q > 750'

'q > 800'

'w > 0'

Logic

'FAAAA' -> "pump_low"

'TFAAT' -> "pump_low"

'TFAAF' -> "pump_high"

'AATFF' -> "pump_high"

'AATFT' -> "rating_curve"

'AAATA' -> "rating_curve"

I realized a potential problem with listening to a flow influenced by a time series as a condition, and that is that such flows are not continuous. This might cause problems in the root finder of VectorContinuousCallback. Thoughts @visr?

What about the option of linear interpolation of the time series? Though that makes integration a bit more complex.

Yeah I've gone back and forth on that, see https://github.com/Deltares/Ribasim/pull/188 for the reasoning for the current approach. Though the discontinuities in the flow are from the callback, and I think that is allowed to do that there, so perhaps the ContinuousCallback can handle it fine, if it only assumes continuous functions between tstops?
I've looked a bit at similar arguments online but couldn't find much more than https://github.com/SciML/ModelingToolkitStandardLibrary.jl/issues/123#issuecomment-1296212344

From a hydrological perspective, I would think that linear interpolation is acceptable.

I think it depends, e.g. if you have a non-equidistant precipitation timeseries like
2020-01-01 00:00:00 0.0
2020-02-01 03:02:40 10.0
2020-02-01 03:06:00 0.0
which represents a short intense shower for a few minutes, do you want to gradually go from 0 to 10 over the month of January?
If it is equidistant it might be good enough in general.

@gijsber in the image you sent of the model edge (5,7) to the terminal is invalid, there must be a flow prescribing node in between.

For precipitation timeseries, this assumption is indeed not wise, but I would think that flow series have more continuous behaviour (and indeed you provide them typically as equidistant series)

@SouthEndMusic I would suggest to add a linear resistance as needed

@SouthEndMusic I would suggest to add a linear resistance as needed

A LinearResistance is not allowed between a Basin and a Terminal because the Terminal has no level. Shall I replace the Terminal with a LevelBoundary with an assumed level?

using a LevelBoundary is fine to me

Using https://github.com/Deltares/Ribasim/issues/447 and https://github.com/Deltares/Ribasim/issues/440,
With conditions

q > 250
q > 275
q > 750
q > 800,

I think this logic would work:

FAAA -> pump low
UAAA -> pump low
TAFA -> pump high
ATAF -> pump high
AAAD -> rating curve
AAAT -> rating curve.

The 'pump high' cases have some overlap which should be cleaned up.

Suggested initial levels:
basin 2: 8.31m+NAP
basin 6, basin 5: 7.5m+NAP
basin 10: 7.0m+NAP
basin 12: 6m+NAP
basin 15: 5.5m+NAP

@gijsber with the above initial levels the simulation now converges but only without the analytical Jacobian. The result doesn't seem very sensible though:

With much lower resistances (1e-2 instead of 5000) the result already looks better:

This at least explains why the PID controlled basin doesn't reach its target level: the PID controlled pump can only take out water while the level is generally too low:


CRAYSAT-1852: Resolve dependabot alerts for pydantic, idna, urllib3

Summary and Scope
Combines https://github.com/Cray-HPE/cfs-config-util/pull/24, https://github.com/Cray-HPE/cfs-config-util/pull/26, and https://github.com/Cray-HPE/cfs-config-util/pull/27 from dependabot into a single PR with a changelog update.
Issues and Related PRs
Resolves 3 dependabot alerts in CRAYSAT-1852.
Testing
List the environments in which these changes were tested.
Tested on:
YTT
Test description:
YTT
Risks and Mitigations
Low
Pull Request Checklist

[x] Version number(s) incremented, if applicable
[x] Copyrights updated
[x] License file intact
[x] Target branch correct
[x] CHANGELOG.md updated
[ ] Testing is appropriate and complete, if applicable
[ ] HPC Product Announcement prepared, if applicable

ncn-m001:/mnt/developer/sann/SAT/cfs-config-util # cfs-config-util update-configs --base-query role=Management --save-suffix .annapoorna --product sat --playbook sat-ncn.yml
INFO: Querying CFS configurations for the following NCNs: x8000c4s22b0n0, x8000c4s28b0n0, x8000c4s25b0n0, x8000c4s26b0n0, x8000c4s24b0n0, x8000c4s23b0n0, x8000c4s31b0n0, x8000c4s30b0n0, x8000c4s27b0n0, x8000c4s29b0n0
INFO: Found configuration "management-24.3.1-4.17.24" for component x8000c4s22b0n0
INFO: Found configuration "management-24.3.1-4.17.24" for component x8000c4s28b0n0
INFO: Found configuration "management-24.3.1-4.17.24" for component x8000c4s25b0n0
INFO: Found configuration "management-24.3.1-4.17.24" for component x8000c4s26b0n0
INFO: Found configuration "management-24.3.1-4.17.24" for component x8000c4s24b0n0
INFO: Found configuration "management-24.3.1-4.17.24" for component x8000c4s23b0n0
INFO: Found configuration "management-24.3.1-4.17.24" for component x8000c4s31b0n0
INFO: Found configuration "management-24.3.1-4.17.24" for component x8000c4s30b0n0
INFO: Found configuration "management-24.3.1-4.17.24" for component x8000c4s27b0n0
INFO: Found configuration "management-24.3.1-4.17.24" for component x8000c4s29b0n0
INFO: Updating existing layer with repo path /vcs/cray/sat-config-management.git and playbook sat-ncn.yml
INFO: Property "commit" of layer with repo path /vcs/cray/sat-config-management.git and playbook sat-ncn.yml updated from 120b6538a22f829af55664b1978aba841f8873f6 to e5c5f8e2e0310fd5fe4b3405db9c5718b70278a2
INFO: Property "name" of layer with repo path /vcs/cray/sat-config-management.git and playbook sat-ncn.yml updated from sat-ncn-2.6.14 to sat-sat-ncn-e5c5f8e-20240513T135215
INFO: Successfully saved CFS configuration "management-24.3.1-4.17.24.annapoorna"
INFO: Successfully saved 1 changed CFS configuration(s) to CFS.


A problem with the APP on Linux

For a few minutes, I have been trying to solve a problem with the application view. However, I had no return. What could be done?

Hi @felipesuri 🙏 Thank you for taking the time to report your issue and sorry for the inconvenience. This looks like a duplicate of https://github.com/getferdi/ferdi/issues/543 so I recommend you take a look at the discussion there. Thank you!


doc: add day-2 operations documentation

This commit is the first of a serie in order to describe all day-2 operations
that are possible via ceph-ansible using a set of playbook provided in
infrastructure-playbooks directory.
Fixes: #5061
Signed-off-by: Guillaume Abrioux<EMAIL_ADDRESS>

jenkins test docs


Support GH emojis

the indexer is in Ruby right? https://github.com/github/gemoji could convert them to unicode

You are right @Haroenv this has to be done at indexing time.


Edits to Change Data Capture

Description & motivation

Closes issue #1729 and removes old version of blog post.
@joellabes Open to feedback if this doesn't fix the issues you brought up!

looks great! Can we also add a redirect in the _redirect file? Think it would look like blog/change-data-capture-metrics	blog/change-data-capture	301 - that way any existing references to the other url get redirected over to the new one.

I like it @joellabes - just added that change, let me know what you think!


Twitter account doesn't exist

Documentation bug, there are links everywhere to @apexserverless on twitter, but it doesn't exist.

ah yeah that needs to be removed


code cleanup and adding randBytes()

This is to address:

https://github.com/rust-lang/libc/issues/1469
https://github.com/rust-random/getrandom/pull/86#issuecomment-521440946

Let's add randABytes and randUBytes too. I need to check which of these is the best to use in the rand crate.

Let's add randABytes and randUBytes too. I need to check which of these is the best to use in the rand crate.

Sure.


Added sqlite package for working with sqlite3 databases

This package allows us to open connections to the SQLite database, run queries and retrieve results. This module will lay the foundation for work in supporting other database engines.

I think this should be archived and added as an external module when a package manager is done.

Your absolutely right. As soon as I added it, I realized that and started work on enabling external modules for Blade. That is, allowing Blade to load modules from shared libraries built in C. And once that is ready (it's almost ready), package managers will become more useful and libraries like the sqlite library can move out of Blade into a separate project.


CI: replace flake8 by ruff check - WIP

Replace flake8 CI job with a ruff CI job.

In pyproject.toml, I'm wondering how to choose the error codes for the section [tool.ruff.lint]. Should we use the same as fmriprep or mriqc, or create specific rules for eddymotion? What are your thoughts on this @effigies @oesteban ?

Except for src/eddymotion/estimator.py:43:9: C901 fit is too complex (17 > 10), I don't think we want to have exceptions there. Copying from fmriprep/mriqc does not seem useful at all -- you want to be very explicit about what you add as an exception.
And maybe we should look into that C901 and try to pacify ruff by improving the code, as opposed to adding one exception.

Except for src/eddymotion/estimator.py:43:9: C901 fit is too complex (17 > 10), I don't think we want to have exceptions there. Copying from fmriprep/mriqc does not seem useful at all -- you want to be very explicit about what you add as an exception.
And maybe we should look into that C901 and try to pacify ruff by improving the code, as opposed to adding one exception.

More explicitly, are you suggesting adding B028 and C408 to the list of errors to ignore and modifying estimator.py to address C901?

Except for src/eddymotion/estimator.py:43:9: C901 fit is too complex (17 > 10), I don't think we want to have exceptions there. Copying from fmriprep/mriqc does not seem useful at all -- you want to be very explicit about what you add as an exception.
And maybe we should look into that C901 and try to pacify ruff by improving the code, as opposed to adding one exception.

More explicitly, are you suggesting adding B028 and C408 to the list of errors to ignore and modifying estimator.py to address C901?

I think we should not have exceptions. We may add C901 temporarily so that this PR passes but open an issue to remind us of looking into it at a later time.

Except for src/eddymotion/estimator.py:43:9: C901 fit is too complex (17 > 10), I don't think we want to have exceptions there. Copying from fmriprep/mriqc does not seem useful at all -- you want to be very explicit about what you add as an exception.
And maybe we should look into that C901 and try to pacify ruff by improving the code, as opposed to adding one exception.

More explicitly, are you suggesting adding B028 and C408 to the list of errors to ignore and modifying estimator.py to address C901?

I think we should not have exceptions. We may add C901 temporarily so that this PR passes but open an issue to remind us of looking into it at a later time.

Can you provide more details? If I leave out B028 and C408 from the exceptions, the PR won't pass. Right now, I've used error codes from the old Flake8 configuration for the extend and ignore lists in [tool.ruff.lint] in the pyproject.toml, but I'm not sure if this is the most effective approach.

Okay, we have this at the moment:
src/eddymotion/data/dmri.py:254:13: B028 No explicit `stacklevel` keyword argument found
src/eddymotion/model/base.py:93:18: C408 Unnecessary `tuple` call (rewrite as a literal)
src/eddymotion/viz.py:346:24: C408 Unnecessary `dict` call (rewrite as a literal)
src/eddymotion/viz.py:398:22: C408 Unnecessary `dict` call (rewrite as a literal)

These are trivial changes we can (and should) address.

Okay, we have this at the moment:
src/eddymotion/data/dmri.py:254:13: B028 No explicit `stacklevel` keyword argument found
src/eddymotion/model/base.py:93:18: C408 Unnecessary `tuple` call (rewrite as a literal)
src/eddymotion/viz.py:346:24: C408 Unnecessary `dict` call (rewrite as a literal)
src/eddymotion/viz.py:398:22: C408 Unnecessary `dict` call (rewrite as a literal)

These are trivial changes we can (and should) address.

Should this be addressed in this PR, or would it be more appropriate for a separate PR? (Considering the PR subject is transitioning from flake8 to ruff)

It's only four little changes -- I would stuff it in this one and reduce the noise overall


Update GitHub Actions CI

The following updates are performed:

update actions/cache to v3
update actions/checkout to v3
replace unmaintained actions-rs/toolchain by dtolnay/rust-toolchain

Still using the outdated / unmaintained actions will generate several warnings in CI runs, for example in https://github.com/enquo/cretrit/actions/runs/4527585660:

Node.js 12 actions are deprecated. Please update the following actions to use Node.js 16: actions/checkout@v2, actions-rs/toolchain@v1, actions/cache@v2. For more information see: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/..

The PR will get rid of those warnings for actions/cache, actions/checkout and actions-rs/toolchain.

Hi!  Thanks for taking the time to put this together.
Pity that actions-rs/toolchain has dropped off the radar.  What do you think about this comment, which suggests that even dtolnay's action is somewhat unnecessary?

What do you think about this comment, which suggests that even dtolnay's action is somewhat unnecessary?

Well, as long as it is guarantueed that some version of Rust and rustup are already installed on the GitHub-hosted runners, you also could just call rustup directly and add all the required parameters including --profile minimal and --no-self-update to reduce the amount of downloaded data, just as dtolnay/rust-toolchain does. However, I am not sure whether there is any kind of guarantee for that, so the action is probably the safer route to go.
As far as actions are concerned, dtolnay/rust-toolchain is already quite minimalistic, compared to actions-rs/toolchain (no extensive JavaScript code, just a few shell commands). As far as I know it just checks whether rustup is already installed, downloads it if not, and than runs the required rustup commands to install the desired Rust version.

Fair call on sticking with dtolnay's action.  Thanks!

Thanks.
Just a heads up: The change of dtolnay/rust-toolchain version from master to v1 in https://github.com/enquo/cretrit/commit/163ee9efcb04f43b28cb5caec3ed86767c9f1e94 may not be what you want, because according to the creator of that action that is not a supported way to use it:

The v1 tag only exists so that the action gets a page on https://github.com/marketplace/actions. (And it has to be a tag for that to happen.) [...] master and all the other branches (stable, 1.65 etc) are the only supported and documented way to use the action.

/me sobs quietly
Why do people have this unerring need to contravene an ecosystem's norms Just Because?  (He says as he indents all his code with tabs...)
Thanks for the heads up, I'll rework things.


4166 promote refactored vis endpoint

Closes #4166
Pending build results to stabilize specs

Frontend tests were OK :+1: (details)

Frontend tests were OK :+1: (details)

Frontend tests were OK :+1: (details)

@juanignaciosl CR please

Wow, a lot of :fire: here! LGTM


First wave of build/test updates

PR Summary
Haven't touched the pipeline stuff yet, just getting the build script in order.
Also updated the tests to be Pester 5 compatible.

PR Context

I can't seem to get Pester running (yes even after updating and importing Pester 5) or the package task to work. Will ping you Monday.

Thank you so much for getting this started Patrick! Honestly you figured out the things I was going to have issues with (what with the dynamic removal, nuspec, and directory props artifacts thing), worked out great.


Assertion failed: file ../src/fsevents.c, line 46 for Chokidar 3.0

Describe the bug
When upgrading to Chokidar 3.0. It fails during jest run
Versions (please complete the following information):

Chokidar version 3.0.0
Node version 10.8.0
OS version: OSX Mojave

To Reproduce
This PR shows the error
https://github.com/unional/test-progress-tracker/pull/35
All tests are passing before the change, using chokidar 2.1.5
The calling code is essentially:
const w = chokidar.watch(filepath, { awaitWriteFinish: { pollInterval: 10, stabilityThreshold: 50 })
w.on('add', path => invokeCallback(c, path, callback))
w.on('change', path => invokeCallback(c, path, callback))
return { close: w.close.bind(w) }

https://github.com/unional/test-progress-tracker/blob/chokidar/src/monitor.ts#L44-L47
The error is:
Assertion failed: (napi_call_function(env, recv, callback, 3, args, &recv) == napi_ok), function fse_dispatch_events, file ../src/fsevents.c, line 46.

Thanks. 🌷

https://github.com/unional/test-progress-tracker/blob/chokidar/src/monitor.spec.ts#L102
https://github.com/unional/test-progress-tracker/blob/chokidar/src/monitor.spec.ts#L122
These are the two tests that failed.

Update to 3.0.2

I see crashes with the same error using 3.0.2 on High Sierra (10.13.5) and<EMAIL_ADDRESS>

I need a reproducible repo to debug this.

@paulmillr, I made the effort to reproduce this outside of our main repo, but wasn't able to.  The problem only occurs on the second of repeated test runs, and only when I clean up each watcher's watched content immediately after closing the watcher.  So now I have two workarounds, disabling FS events, or simply cleaning up and then closing the watcher.
My gut feel is that this is probably a subtle timing issue that may very well be my fault.  If I can narrow it down to the point where I can demonstrate the issue in isolation, I'll file a new detailed ticket.

@the-t-in-rtf i've spent two days and found the root cause of this. Try 3.2.4 in a few days, or github version right now.

Thanks, @paulmillr.  I assume 3.3.0 includes your fix?  I temporarily undid my previous workaround (changing the timing of my calls to watcher.close()), and  I don't see errors any more.

Yes.

Perfect, seems like you fixed it then.  Thanks again!


Size menu is not sizing correctly? Content is cutoff.

My size menu looks different from yours. Its as if the content is cut off.
Here is a link where you can see it in action: bum.io

It looks fine to me:

Are you sure you don't have cache issues?


fix(table): unset word-break, add modifiers

remove word-break: break-word; from <td>
add .pf-c-table-container - set overflow-x: auto: for when table run out of available space
add .pf-m-break-word - set word-break: break-word;
add .pf-m-text-overflow- set text overflow to ellipsis
removed --pf-c-table--BorderWidth: 0; as this causes overflow-y to scroll

PatternFly-Next preview: https://patternfly-next-pr-2266.surge.sh

@mcoker updated

@mceledonia signed off on headers wrapping
added modifiers to control wrapping and text overflow

holding this until breaking change release


Feature add TaskTest and update TagTest

This PR adds unit tests in the TaskTest class and updates the unit tests TagTest to cover more types of valid and invalid inputs.

Codecov Report
Base: 68.72% // Head: 68.72% // No change to project coverage :thumbsup:

Coverage data is based on head (af06e2b) compared to base (73912b3).
Patch has no changes to coverable lines.

Additional details and impacted files
@@            Coverage Diff            @@
##             master     #168   +/-   ##
=========================================
  Coverage     68.72%   68.72%
  Complexity      478      478
=========================================
  Files            91       91
  Lines          1605     1605
  Branches        178      178
=========================================
  Hits           1103     1103
  Misses          451      451
  Partials         51       51

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Freezing when starting to install?

Hello, I have an O3DS(2DS) on 11.3 A9LH and recently formatted my SD Card to have a 64 kb cluster size instead of 32 kb. I sucessfully got all my stuff back from a backup, and tried installing a CIA in FBI, however when I tried installing it, FBI froze my 2DS, causing me to turn it off. I tried with another CIA and same results occurred. Any solution?

Never experienced this before. Did it work fine with a 32kb cluster size, and do other apps like the eShop seem to work fine?

It worked fine with a 32kb cluster size, however I have yet to tried other stuff with this, I'm gonna see if freeShop works in the mornin'

On freEshop, I tried installing a game and it took a while for it to start on Installing Seed and Installing TMD.

Have you tried leaving FBI installing for a while to make sure it's actually frozen and not just taking a while?

I did for almost 10 minutes. I saw that the time never started moving and stayed at the same time it froze. Not sure if this was a solution or not, but I reformatted my SD Card to a 32 kb cluster size. I only did this because I saw there was a glitch with my GBA games in 64 kb.

If 32kb works fine, I'll have to chalk it up to some 3DS system issue. FBI goes through the usual services to access the filesystem, so changing the filesystem like that should not impact FBI directly.

Umm, now I got a new problem,
FBI tells me that it can't install CIA files, as it gives me the result code of 0xC960454B. Any solution? I tried formatting my SD card and backing up my stuff

Please make a thread on a support forum. This space is for bugs/suggestions in the FBI code, not for support.

Can you post the full error output?

Failed to install CIA file.
Result code: 0xC960454B
Level: Status (25)
Summary: Internal (11)
Module: FS (17)
Desc: Bus error / TWL partition invalid (331)
The weird thing is that I successfully installed two 8mb cias

It definitely looks like there's something wrong with your SD card. Do you have any others you could try to make sure?

I tried with my 4gb SD Card, and it worked on installing a GBA cia

Maybe try running a disk check on it and see if it fixes any errors. If you just formatted it though I doubt that would help.

Yeah, I already tried that, it said there's nothing wrong with the SD Card.


output stream views and visualization

For easier distribution and visualization, create per-species scenario views of stream modelling:

salmon (CH/CM/CO/PK/SK)
steelhead
westslope cutthroat trout
etc

todo:

[x] add/tweak the upstr/dnstr columns in bcfishpass.streams necessary to create the views
[x] create the views
[x] update QGIS layer to display the views
[ ] update web map to display the views

Perhaps the views are not necessary for the visualization (QGIS/web map) but I think a view is likely required for publication of the salmon access model

done except for web map, that is covered in #211


Right Click Context Menu Item not working

The Linkwarden extension should appear whenever a user right clicks on a link in both Chrome and Firefox. The code for the same is also present with a comment signifying its for the context menus. However on multiple machines it does not work whatsoever.

Attaching screenshots of context menu on firefox, running latest mozilla addons marketplace version of the extension.
As previously discussed in #49

To restate, it does not work on Chrome either

Hey, im the guy that initially opened that request, I kind of forgot about it to be honest, but just realised it again and i can also confirm that this has never worked for me, I would have seen the option and remembered.
Im on Chrome and i actually reinstalled the extension, so it definetly does not work.

Which request do you mean?

I see the problem.
is anyone able to use the git version instead of installing from chrome store?
That would help me to actually know, because the git version 'dev branch' does work properly, but the other does not, so i dont know if this is something from the Chrome version or something


Consider shading the flatbuffers dependency (java)

Hi,
Just a heads up here :-
After some discussion with the Flatbuffers maintainer, it appears that FB generated code is not guaranteed to be compatible with any other version of the runtime library other than the exact same version of the flatc used to compile it.
This makes depending on flatbuffers in a library (like arrow) quite risky, as if an app depends on any other version of FB, either directly or transitively, it's likely the versions will clash at some point and you'll see undefined behaviour at runtime.
Shading the dependency looks to me the best way to avoid this.

@alexnixon can you open a JIRA about that?

Can you someone from Java-side open a JIRA issue? cc @pravindra @emkornfield @siddharthteotia

https://issues.apache.org/jira/browse/ARROW-5579


do not encourage deployments after 4pm

Usually it is not particularly good idea to deploy on normal deployment days: monday to thursday after 4pm.
reasons:
Call your wife?

Closed with #73


Clipping with dynamic volume normalisation

Originally posted by acid-sun January 10, 2022
I have some sound problem with new type (dynamic) volume normalisation. Sound clipping.

Track: Fast Flux - Zealous.
[2022-01-10T14:06:56Z INFO  librespot_playback::player]  (141606 ms) loaded
[2022-01-10T14:06:56Z WARN  librespot_playback::player] This track will at its peak be subject to 2.93 dB of dynamic limiting.
[2022-01-10T14:06:56Z WARN  librespot_playback::player] Please lower pregain to avoid.
[2022-01-10T14:06:56Z DEBUG librespot_playback::player] Normalisation Data: NormalisationData { track_gain_db: 2.933378219604492, track_peak: 0.7943402528762817, album_gain_db: 2.933378219604492, album_peak: 0.7943402528762817 }
[2022-01-10T14:06:56Z DEBUG librespot_playback::player] Calculated Normalisation Factor for Track: 140.17%

Track: Lucien - Revasser
[2022-01-10T14:10:14Z INFO  librespot_playback::player]  (209250 ms) loaded
[2022-01-10T14:10:14Z WARN  librespot_playback::player] This track will at its peak be subject to 6.20 dB of dynamic limiting.
[2022-01-10T14:10:14Z WARN  librespot_playback::player] Please lower pregain toavoid.
[2022-01-10T14:10:14Z DEBUG librespot_playback::player] Normalisation Data: NormalisationData { track_gain_db: 6.802104949951172, track_peak: 0.7412295341491699, album_gain_db: 6.802104949951172, album_peak: 0.7412295341491699 }
[2022-01-10T14:10:14Z DEBUG librespot_playback::player] Calculated Normalisation Factor for Track: 218.83%

I tried to upper threshold to 0 dBFS, but it did not ideal result. Earlier it was possible to specify threshold > 0.
If I switched to basic method, music played normal.

These are great reference tracks! I can confirm and will see what I can come up with.
For now I recommend that you set pregain to something like -5 or even -9 when listening to tracks with a wide dynamic range.


Fix some documentation:

Overview
The script had an incorrect name.
Also, add a link to the installation instruction and make some
cosmetic changes.

Improve createPublicApiHandlers Javadoc:

fix endpoint name
clarify the base path for user handlers
specify protocol
add link to the documentation

Q: That affects 0.6.0, how do we handle that? Cherry-pick and move the tag? Make 0.6.1?

Definition of Done

[ ] There are no TODOs left in the code
[ ] Change is covered by automated tests
[ ] The coding guidelines are followed
[ ] Public API has Javadoc
[ ] Method preconditions are checked and documented in the Javadoc of the method
[ ] Changelog is updated if needed (in case of notable or breaking changes)
[ ] The continuous integration build passes

I think we should move the tag for 0.6.0.

Yeah, it would be easier (and Javadoc fixes are non-critical).


WIP: Add support for AWS RHEL 9.0 EUS images to Image Builder

WIP PR for easier reviewing

Added EC2 EUS image config
Added EC2 EUS image type
Added package set with dummy package name for not yet released rhui client package.

Open questions:

What am I missing to enable image builder to build RHEL 9.0 EUS images for AWS?

Quickly read through and it all looks good.  I don't think there would be anything left for the image definition itself.  We might need to figure out which repository sources to use for building the image, especially for EUS versions that aren't the current GA (e.g., when building 9.0 EUS after 9.1 is released).

Closed as the implementation was deemed unnecessary at this point. I will keep the changes in my branch for now just in case the issue comes up again in the near future.


Fix broken build for Windows

Windows does not provide a mkdir function with two parameters.
Signed-off-by: Stefan Weil<EMAIL_ADDRESS>

Currently the file training/lang_model_helpers.cpp is only compiled when building with autotools, but not when building with cmake (that's strange and needs more investigation). Continuous integration uses cmake, so it does not get that build error.

Ray never touches the CMake build files.


func reuse tip

Why the L658-L663 in IGListAdapter.m can't coding the following:
[self _updateBackgroundViewShouldHide:![self _itemCountIsZero]];

Feel free to submit a PR! I'd be happy to take a look.


Feat/testing validators maj

Validators should work now. They haven't been tested on normal trx since I only have coinbase trx.
TODO @bajla:

[ ] approve this pull request
[ ] delete console.writelines from miner
[ ] add code into miner for saving and sending valid blocks
[ ] check above comments
[ ] we need to save utxo that are ours into separate array of strings
[ ] frontend


[Team C] [Feature] Dashboard - Create teacher dashboard

As a teacher, I want a dashboard where I can view recent activities, manage tests, and see assessment results.

[Team C] [Task] [Create teacher dashboard] Implement the header component.
[Team C] [Task] [Create teacher dashboard] Add images to the dashboard.
[Team C] [Task] [Create teacher dashboard] Implement the "Create Assessment" button.
[Team C] [Task] [Create teacher dashboard] Implement the table for displaying assessments.
[Team C] [Task] [Create teacher dashboard] Add search and filter functionalities.


Is there a keyboard shortcut to send the email to marvin instead of clicking with mouse?

Instead of moving the mouse I wanted to ask if the functionality of the button can get activated with a keyboard shortcut?

Hi @wadid! The extension doesn't have a keyboard shortcut for sending emails to Marvin but I could try to add one! I've been meaning to add a few improvements to it so I'll leave this open for now and let you know when I make some progress on that.
Due to the way this Gmail integration is made, I think the shortcut could only work when you're looking at a single email, like here for example:


What happens on a custom image?

This article seems to be written for an azure base image.
What happens on a custom container? Is any of the logging possible? If yes, what should I do to enable them on a custom container?
E.g. do I need to stream it to stdout, should I write it myself to blob storage?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: bfc316af-0d30-b449-a04f-3019b7a16bc1
Version Independent ID: 01c4c435-fda9-374a-70b3-f1d7f13c7651
Content: Enable diagnostics logging for apps - Azure App Service
Content Source: articles/app-service/troubleshoot-diagnostic-logs.md
Service: app-service
GitHub Login: @cephalin
Microsoft Alias: cephalin

@jorgecarleitao Thanks for the feedback! We are currently investigating and will update you shortly.

@jorgecarleitao The diagnostic logging is for Azure PaaS offering. Meaning if you deploy your code, this diagnostics logging will pick up that logging you have in your app code once you've configured Diagnostics Log. Using a custom docker image deployed to a web app (e.g. https://docs.microsoft.com/en-us/azure/app-service/app-service-web-get-started-windows-container), things are different.
You'll have to instrumented your logging using stdout, so you can use methods such as az container logs CLI command to stream those logs out. See this doc for more information. You should be able to use getwebsitecontainerlogs REST API method as well to retrieve the last lines of container logs. Another option is retrieving the logs from kudu
Having said that, if you want to have more retention of your logging, then I agree with you in writing your logs to blob storage yourself. Before writing the logs to storage yourself, I would advise exploring the methods I mentioned above to see how best to leverage it. Perhaps startup activities could be logged via stdout and more granular transaction correlation logging be written to blob storage.
I hope this helps.

@jorgecarleitao please let us know if the above suggestions work.

We will now proceed to close this thread @jorgecarleitao. If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.


Benchmark

I need to compare what is the performance of RCache vs RMap or redisson vs other libraries. Could you provide benchmarks?
This is more of a suggestion than an issue per se.

Hi!
Currently, I'm working on it. I'll publish performance comparison of two Redisson editions: Open-source vs PRO.

Since there is no RCache I assume you are asking us to compare the performance RMap against other types of Cache implementations that Redisson provides such as RMapCache, SpringCache and JCache. I think it is quite obvious that RMap is going to out perform the others. It is simply because the Caching system itself is a lot more complex than a simple Map structure.
As to compare Redisson with other libraries, I am not sure if it is comparable to be honest. It is like compare apples with oranges.
Redisson is a "higher level" library compared to other Redis clients, it offers much more than just a way of executing a Redis command like any others. If offers an abstraction to hide Redis, as a separation of concern, if you like, that when you architecting your application, you don't have to stop and think which Redis command is equivalent to putIfAbsent in Java. People choose Redisson won't because it can execute the hset command faster than other clients, although I am not saying it can't.
As to compare Redisson with other in-memory data grid solutions, it is another story on its own. Redisson is different to other solutions simply because others tend to provide the client and storage together, such as HazelCast, GridGain and Coherence, whereas Redisson is a client side library that's working with your exising Redis NoSQL server. So strictly speaking it is not comparable.

Here is the Benchmark whitepaper. It provides performance charts for both RMap and RMapCache objects.

@mrniko benchmarks mentioned in the above comment is with 6bytes of size. Do you have benchmarks with higher size like 50KB 1MB.

@ALSowjanya
I'm afraid no.


Clean logging during Service Instance creation failure

When si creation is failed (hostd is down). The log dump is a mess.
This is valid case and we retry to create si (connect to hostd). However, the log during failure needs to be systematically dumped.
Right now its something like
09/22/17 16:53:06 928247 [VMChangeListener] [ERROR  ] Failed to create the local Service Instance as 'dcui', continuing... :
Traceback (most recent call last):
  File "/usr/lib/vmware/vmdkops/bin/vmdk_ops.py", line 1163, in get_si
    _service_instance.CurrentTime()
AttributeError: 'NoneType' object has no attribute 'CurrentTime'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/build/mts/release/bora-4564106/bora/build/esx/release/vmvisor/sys/lib64/python3.5/site-packages/pyVim/connect.py", line 323, in __Login
  File "/build/mts/release/bora-4564106/bora/build/esx/release/vmvisor/sys/lib64/python3.5/site-packages/pyVmomi/VmomiSupport.py", line 557, in <lambda>
  File "/build/mts/release/bora-4564106/bora/build/esx/release/vmvisor/sys/lib64/python3.5/site-packages/pyVmomi/VmomiSupport.py", line 363, in _InvokeMethod
  File "/build/mts/release/bora-4564106/bora/build/esx/release/vmvisor/sys/lib64/python3.5/site-packages/pyVmomi/SoapAdapter.py", line 1338, in InvokeMethod
http.client.HTTPException: 503 Service Unavailable (Failed to connect to endpoint: [N7Vmacore4Http16LocalServiceSpecE:0x05b2e8b8] _serverNamespace = /sdk action = Allow _port = 8307)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/vmware/vmdkops/bin/vmdk_ops.py", line 1146, in connectLocalSi
    version=newestVersions.Get('vim'))
  File "/build/mts/release/bora-4564106/bora/build/esx/release/vmvisor/sys/lib64/python3.5/site-packages/pyVim/connect.py", line 242, in Connect
  File "/build/mts/release/bora-4564106/bora/build/esx/release/vmvisor/sys/lib64/python3.5/site-packages/pyVim/connect.py", line 327, in __Login
pyVmomi.VmomiSupport.vim.fault.HostConnectFault: (vim.fault.HostConnectFault) {
   dynamicType = <unset>,
   dynamicProperty = (vmodl.DynamicProperty) [],
   msg = '503 Service Unavailable (Failed to connect to endpoint: [N7Vmacore4Http16LocalServiceSpecE:0x05b2e8b8] _serverNamespace = /sdk action = Allow _port = 8307)',
   faultCause = <unset>,
   faultMessage = (vmodl.LocalizableMessage) []
}

I guess this is expected behavior.

However, the log during failure needs to be systematically dumped.

@pshahzeb can you add some more detail about "systematically dumped."?


vulkano panicked when using with rust-gpu

Version of vulkano: 0.34.1
Version of rust-gpu: 0.9
OS: Windows10
GPU (the selected PhysicalDevice): Intel Iris Xe Graphics
GPU Driver: <IP_ADDRESS>22
Upload of a reasonably minimal complete main.rs file that demonstrates the issue:

rust-gpu shader:
#![no_std]

use spirv_std::{glam::{vec4, Vec2, Vec4}, spirv};

#[spirv(vertex)]
pub fn main_vs(
    #[spirv(position, invariant)] out_pos: &mut Vec4,
    position: Vec2,
) {
    *out_pos = position.extend(0.0).extend(1.0);
}

#[spirv(fragment)]
pub fn main_fs(output: &mut Vec4) {
    *output = vec4(1.0, 0.0, 0.0, 1.0);
}

vulkano code:
        const SHADER: &[u8] = include_bytes!(env!("minimal_shader.spv"));
        let shader_code = SHADER
            .chunks(4)
            .map(|c| u32::from_le_bytes([c[0], c[1], c[2], c[3]]))
            .collect::<Vec<_>>();

        let shader_module = unsafe {
            ShaderModule::new(
                context.device().clone(),
                ShaderModuleCreateInfo::new(&shader_code),
            )
        }
        .unwrap();

        let vs = shader_module.entry_point("main_vs").unwrap();
        let fs = shader_module.entry_point("main_fs").unwrap();

        let vertex_input_state = MyVertex::per_vertex()
            .definition(&vs.info().input_interface) // panicked here
            .unwrap();

MyVertex:
#[derive(BufferContents, Vertex)]
#[repr(C)]
struct MyVertex {
    #[format(R32G32_SFLOAT)]
    position: [f32; 2],
}

Issue
thread 'main' panicked at 'called Option::unwrap() on a None value', C:\Users\SxC.cargo\registry\src\index.crates.io-6f17d22bba15001f\vulkano-0.34.1\src\pipeline\graphics\vertex_input\definition.rs:45:46

I add SpirvMetadata::NameVariables in build.rs and this issue is fixed. Thank you so much!
use spirv_builder::{MetadataPrintout, SpirvBuilder, SpirvMetadata};
use std::error::Error;

fn main() -> Result<(), Box<dyn Error>> {
    SpirvBuilder::new("minimal-shader", "spirv-unknown-vulkan1.2")
        .spirv_metadata(SpirvMetadata::NameVariables) // this fix this issue
        .print_metadata(MetadataPrintout::Full)
        .build()?;

    Ok(())
}

I just started to learn to use rust-gpu, MetadataPrintout::Full is copied from somewhere else. I just delete '.print_metadata(MetadataPrintout::Full)' and the code also works fine, thanks again!

I just notified that print_metadata is set to MetadataPrintout::Full in SpirvBuilder::new. So deleting '.print_metadata(MetadataPrintout::Full)' changes nothing.
I tried to change MetadataPrintout::Full to MetadataPrintout::DependencyOnly, but this resulted in a compilation error:
error: environment variable `minimal_shader.spv` not defined at compile time
   --> minimal\src\triangle_renderer.rs:132:46
    |
132 |         const SHADER: &[u8] = include_bytes!(env!("minimal_shader.spv"));
    |                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = help: use `std::env::var("minimal_shader.spv")` to read the variable at run time
    = note: this error originates in the macro `env` (in Nightly builds, run with -Z macro-backtrace for more info)

I've made a small sample project on how to integrate vulkano and rust-gpu, feel free to reference: https://github.com/Firestar99/rust-gpu-vulkano-example
The build script I'm using configures spirv-builder like this:
https://github.com/Firestar99/rust-gpu-vulkano-example/blob/eeb8b3ddf4cd475bd24ba58470f9c8b363334228/example/build.rs#L18-L21

multimodule so that every shader has a separate spv file, it will work without, but almost all tooling expects one shader per spv file. And shader! codegen may generate the same struct multiple times resulting in compile errors, if you didn't disable it (currently only master).
print_metadata: if you use a build script and you don't emit any cargo::rerun-if-changed=./path, cargo will only rerun your build script if the script or the contents of src changed. Which means if you only change your shaders, or dependencies that only your shaders depend upon, your shaders won't be recompiled and you'll be confused on why yout changes don't do anything. MetadataPrintout::DependencyOnly makes rust-gpu emit this rerun metadata for cargo on it's own, without you having to care for it at all, and you'll always have up-to-date shaders.

Also feel free to steal the codegen of my main project, which generates a single file containing each shader entry point as a separate shader! macro invocation with the correct path, all of them organized in the same mod structure as your shader crate. But I haven't gotten around to clean it up enough to release it publicly, though that may happen in a few months as I want to publish a bindless API for rust-gpu and vulkano (potentially wgpu as well?).
https://gitlab.com/spacegamedev/space-rust/-/tree/master/vulkano-bindless/vulkano-bindless-shader-builder?ref_type=heads


Accessibility Issues in Add Article Page

The below-mentioned issues are classified as critical, serious, and moderate.
Critical:

"aria-hidden='true' must not be present on the document body" 'helpUrl': https://dequeuniversity.com/rules/axe/3.1/aria-hidden-body?application=axeAPI

"Add Article" button must have discernible text (value="Add Article" not allowed. Name is allowed) 'helpUrl': https://dequeuniversity.com/rules/axe/3.1/button-name?application=axeAPI

Ensure "Category" has an accessible name

Serious:

The contrast between foreground and background colors should meet WCAG 2 AA contrast ratio thresholds. Elements must have sufficient color contrast.'bgColor': None, 'contrastRatio': 0, 'expectedContrastRatio': '4.5:1', 'fgColor': '#212529' 'helpUrl': 'https://dequeuniversity.com/rules/axe/3.1/color-contrast?application=axeAPI '

Document has a non-empty  element 'helpUrl': ' https://dequeuniversity.com/rules/axe/3.1/document-title?application=axeAPI'
Moderate:

Ensure the headings are semantically correct 'helpUrl': ' https://dequeuniversity.com/rules/axe/3.1/heading-order?application=axeAPI'

Page doesn't have banner landmark, contentinfo landmark 'helpUrl': 'https://dequeuniversity.com/rules/axe/3.1/landmark-no-duplicate-banner?application=axeAPI '

The accessibility tests run fine now in CI.


added rel noopener noreferrer nofollow to external link

With the current version of cookieconsent included in my website, Google Lighthouse  raises a security warning related to external (target="_blank") links, lowering the score.
To fix it and follow the best practices, it's a good idea to include a rel="noopener" attribute in the external links, this PR adds it, along with nofollow and noreferrer.
Further info here: https://developers.google.com/web/tools/lighthouse/audits/noopener

issue #185 is asking for this...


Limit to individual project directories

It would be nice if the dependency reporting could be limited to a specific project directory. Usually, I'm not interested in all dependencies of all packages, but only in the (transitive) dependencies of a specific project, ideally even before having installed any of the dependencies.

To implement this, pipdeptree could internally probably do something like the following when being passed a project directory:

Create a virtualenv
Install pipdeptree in the virtualenv
Remember the installed packages in the virtualenv at this point
Install all packages required by the project in the virtualenv
Run pipdeptree -l in the virtualenv
Remove all packages from the list that were in 3. but are no project dependencies
Report the project dependencies

Not sure if I have understood what you mean by a project directory. But I am guessing you want to get all direct dependencies of a particular package, let's say foo. It can be achieved using the -p flag and grep as follows,
$ pipdeptree -p foo | grep -E '^(foo|\s\s-)'

Here, -p will restrict the output to dependencies of foo. Then grep is used to match for two things:

Two spaces followed by a hyphen to match only those lines which are 1st level children (direct dependencies)
line containing the package name itself

This however is not possible without installing the package. The tmp virtualenv approach you've mentioned above should work though.

But I am guessing you want to get all direct dependencies of a particular package

Not quite. By "project directory" I mean the root directory containing the source code of any Python project. Let's take https://github.com/nexB/scancode-toolkit as an example. After git clone, my Git working tree would be the project directory. Now I'd like to programmatically get the transitive dependency tree for this project. If you're familiar with Maven, that would be pretty much what mvn dependency:tree does.

If your repo has a setup.py file, you can try this:

Create a new virtualenv
Run pip install . from the project root.
Run the above command (pipdeptree + grep)

Right. I was just thinking that it would make sense to add this use case straight to pipdeptree so only a single command needs to be run. What do you think?

This is a very specific use case and I don't think the code for it belongs inside pipdeptree. But even if we decide to add it, it would be extremely tricky because pipdeptree works in a virtualenv only after it's manually installed inside it first. This makes the whole thing cyclic - pipdeptree creating virtualenv, then installing pipdeptree inside it and then replacing the current process with that executable.
Also, if you have the setup.py file, that itself would be the source of truth for first level dependencies of the project, right?

[...] replacing the current process with that executable.

I don't think any such complex logic would be necessary. It would be just the global pipdeptree calling the one from the virtualenv, that's it.

Also, if you have the setup.py file, that itself would be the source of truth for first level dependencies of the project, right?

Correct. But like I said, I'm interested in transitive dependencies as well. Also, I'd like to investigate arbitrary projects, so some might have a requirements.txt file instead of a setup.py file.
So feel free to close this if you believe this is out of scope.

I don't think any such complex logic would be necessary. It would be just the global pipdeptree calling the one from the virtualenv, that's it.

This sounds like a better approach than replacing the current process.

So feel free to close this if you believe this is out of scope.

Not sure if it's out of scope entirely, need to think about it. Will check mvn dependency:tree. Keeping the issue open.

Thanks for considering. FYI, see how the docs for mvn dependency:tree say "Displays the dependency tree for this project".

I don't think we plan to support this for now.


SX80 does not respond, but shows successful connection

I added a Cisco SX80 codec using this plugin and then I configured a button action to trigger camera presets. Although Companion shows that it is successfully connected via SSH, the button does nothing.
The SX80 is running the latest version of the TC software, TC7

As TC7 was end of support from Cisco in August 2022, having been replaced by CE8 and subsequently CE9 that the SX80 also supports, this issue should be closed.


Add Gpg4win

Hi @aaronparker,
i added Gpg4win.
Website: https://www.gpg4win.org/
Server: https://files.gpg4win.org/

Greetings
BornToBeRoot

Is there a way that Gpg4Win does a look up for new versions itself rather than scraping files.gpg4win.org?

I searched the source code for gpg4win and didn't find an update check / download...
This is from the website: https://gpg4win.org/download.html

The Kleopatra application to manage the gpg keys has an update routine, but this is a different project and it's included in the gpg4win bundle
https://apps.kde.org/kleopatra/


Update change log for version 1.1.0

Coverage remained the same at 97.515% when pulling 15c61b11455105e8408e680eabb851cfc97fd3f8 on roman/changelog into 757a10012ea13a9bf23daff43c61db9f88c3e0a9 on devel.

Coverage remained the same at 97.515% when pulling 2b3872dfb1aed165bcfc1a6f987de71704fb51a2 on roman/changelog into 757a10012ea13a9bf23daff43c61db9f88c3e0a9 on devel.


Pattern: Content group - with pictograms Design QA testing

User Story

As a [user role below]:
DDS Designer and Developer

I need to:
test the Content group - with pictograms

so that I can:
be sure the developer's code correctly works as designed

Additional Information

Dev link (#1165 )
design folder is on Box
Functional specs

Tasks
Ensure the Content group - with pictograms pattern has been tested and reviewed by design + dev.
Acceptance criteria

[ ] Pattern is reviewed by designer and design defects are identified
[ ] New issues are opened to log individual defects
[ ] Screenshots of as-is and to-be design should be added to the issue when needed
[ ] Developer associated with building the pattern is assigned to the new issues
[ ] Appropriate labels are added to the issue  eg.  bug, dev

@wonilsuhibm , @RobertaJHahn , @oliviaflory
This was merged in, and it's ready for QA,
Thanks!


Refine API and refactor

WIP (do not merge). Tests do not pass.

does not rely on Polymer
API

startup:

constructor => initialize => createRoot
connectedCallback => enableUpdate => invalidate

update/render:

property change => invalidate =>
(async) shouldUpdate => update
resolves any updateComplete promise

Hey @sorvell, @rictic mentioned this PR in a chat about other lit-element things. Looking through this, I’m noticing that it seems  like there’s no equivalent for _propertiesChanged(). Is that intentional? Right now my team is using that fairly extensively to make GETs after a necessary ID attribute is set at runtime. We could change to a different pattern if that’s what you recommend.

@justinfagnani PTAL Lots of changes per offline discussion:

The property options have been updated.

The update lifecycle has been updated.

The logic in invalidate is especially tricky and could probably use better factoring.

I'm still calling invalidate in connectedCallback rather than when constructed due to some trickiness with ShadyCSS that we can discuss.

TODO

We need to decide if UpdatingElement should be separate and in this package or broken out.

Need to move the render-helpers into lit-html.

The current convention is _ prefix for protected members and __ prefix for private members. However, looks like this convention is not strictly followed. Examples:
  private __isChanging: Boolean = false;
  private _root?: Element|DocumentFragment;
  _firstRendered() {}
  protected _createRoot(): Element|DocumentFragment

Reflecting attributes in the property setter does not work since Safari does not allow an element to produce attributes in the constructor. We will need to move this to update or specially handle this on first invalidate.

@sorvell I'm trying out this branch on npm dev channel and I'm wondering how getters and setters can co-exist with properties. Namely, .prop=${value} does not work with prop getter and setter. But declaring an explicit prop property causes weird issues with the setter being called twice on invalidate with wrong arguments, etc. What is the right way to do this?

Also, the "Declarative rendering" part of the README needs an update

@ernsheong We introduced invalidateProperty(name, oldValue) to help address this. There's a test here.

Tests failing but only on Firefox due to known timeout issue.


Be hang when starting

Steps to reproduce the behavior (Required)
In our scenoria, we create some big primary key table and use flink continuously write data into it. when I restarting the be process, it hanged.
It has about 950mb meta file on the disk

Expected behavior (Required)
Starting quickly
Real behavior (Required)

the be process maybe hang in Rockdb::Seek the meta file and this function actually hang about 20min.
StarRocks version (Required)
2.0.1

Maybe you have too many tablets, please provide the result for this cmd
 bin/meta_tool.sh --root-path=pwd/storage --operation=ls

After some investigation, it seems the cause is rocksdb's DBIter::Seek(key) is very slow if the key does not exist and there are lots of deleted entries after key, so rocksdb will iterate those deleted keys until finding a valid next key.
Unfortunately, if there are no pending rowsets, DBIter::seek('tprs_') will search next valid key in "trs_" range, if you have lots of deleted tablets (maybe caused by drop table), it will be very slow.
Will fix this in future PRs.

Currently, you can use this cmd to compact meta data to remove deleted rows to prevent this issue.
bin/meta_tool.sh --root-path=`pwd`/storage --operation=compact_meta

OK, I will try it later with compact_meta  command


VS Code valid contributes.jsonValidation# fileMatch Pattern

VSCode Version: 1.52.0
Commit: 940b5f4bb5fa47866a54529ed759d95d09ee80be
Date: 2020-12-10T22:46:53.673Z
Electron: 9.3.5
Chrome: 83.0.4103.122
Node.js: 12.14.1
V8: <IP_ADDRESS>-electron.0
OS: Darwin x64 19.6.0
OS Version: MacOS 10.15.7

Steps to Reproduce:

In the package.json
contribution.jsonValidation

When I use the following fileMatch pattern for a custom JSON schema, it does not work files haveing extension ".xyz". I tried multiple combinations but none worked for me.
"jsonValidation": [
            {
                "fileMatch": "/*.xyz",
                "url": "http://<IP_ADDRESS>:8080/xyz.schema.json"
            },
            {
                "fileMatch": "/.xyz",
                "url": "http://<IP_ADDRESS>:8080/xyz.schema.json"
            },
            {
                "fileMatch": "*.xyz",
                "url": "http://<IP_ADDRESS>:8080/xyz.schema.json"
            },
            {
                "fileMatch": ".xyz",
                "url": "http://<IP_ADDRESS>:8080/xyz.schema.json"
            },
            {
                "fileMatch": "/xyz",
                "url": "http://<IP_ADDRESS>:8080/xyz.schema.json"
            }
        ]

Does this issue occur when all extensions are disabled?: Yes

Nevermind, I found the answer with the help of one of my colleagues. The following worked for me:
"jsonValidation": [
            {
                "fileMatch": "/*.xyz",
                "url": "http://<IP_ADDRESS>:8080/xyz.schema.json"
            },
            {
                "fileMatch": "*.xyz",
                "url": "http://<IP_ADDRESS>:8080/xyz.schema.json"
            },
        ]


Kubelet certification rotation support

This was initially mentioned in #749 and hopefully addressed in kube-aws once the feature landed to K8S 1.8.
Upstream issue: https://github.com/kubernetes/features/issues/267
cc @danielfm

Seems like the certificate rotation is supported since k8s v1.8.0 and it is a matter of just adding -rotate-certificates=true to kubelet to enable it.

This is done in #1086 and #1108.
In nutshell, kubelet.rotateCerts: true enables the feature.


Merge SQS branch

Add SQS classes enabling creating abstract SQS objects from ATAT-style lattice.in files or concrete ones from abstract or directly via the pymatgen.Structure constructors.
Enable enumeration of an abstract SQS to all of the possible concrete SQS for a given sublattice model

@mxf469 FYI: I am merging the sqs branch into master and deleting the branch. New work you do should all be branched from the master branch.


Added optional boolean to GetProductInfo

See https://github.com/DoctorMcKay/node-steam-user/issues/112 and commit messages

The code looks good besides the feedback I left and the changing of unrelated code style. I appreciate you doing this.

I tried to avoid it, but my editor didn't like it 👍
I replaced spaces with tabs, as u wanted.

I had set all inclTokens booleans in existing self.getProductInfo and this.getProductInfo to false as before, but maybe you'd like them to be true now. Please check.

Well, it's been several months. Any progress on this?

Bump


Fix context manager for older node versions

Description
Async hooks context manager needs to be activated by calling .enable()
before it can be used as a context manager. This is important for older
versions of node that don't use support local storage context manager.
We need to release this as a hot fix. Will follow up with a test case to catch any regressions in future.

Codecov Report

Merging #53 (ee9bb13) into main (9988fae) will increase coverage by 0.17%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main      #53      +/-   ##
==========================================
+ Coverage   85.62%   85.79%   +0.17%
==========================================
  Files          10       10
  Lines         167      169       +2
  Branches       29       29
==========================================
+ Hits          143      145       +2
  Misses         24       24

Impacted Files
Coverage Δ

src/tracing.ts
96.29% <100.00%> (+0.29%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9988fae...ee9bb13. Read the comment docs.


dron commit

Description 📜
In this commit i have written the code of UI and javascript which is link to the google spreadsheet and directly submit the input enter by the user in the UI section
Fixes #341

Type of change 📝

[ ] Documentation (Content Creation in the form of codes or tutorials)

Domain of Contribution 📊

[x] Frontend Dev #HTML/CSS/JS

Checklist ✅

[x] I follow Contributing Guidelines & Code of conduct of this project.
[x] I have performed a self-review of my own code or work.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] My changes generates no new warnings.
[x] I'm GWOC'21 contributor

Screenshots / Gif (Optional) 📸

The concept is nice 😄 But you have three things to take care of

There are some typos, please resolve those.
There are some formatting issues too, please look in those as well.
You don't need to have separate code files, embed them in your Readme.md itself.

The concept is nice 😄 But you have three things to take care of

There are some typos, please resolve those.
There are some formatting issues too, please look in those as well.
You don't need to have separate code files, embed them in your Readme.md itself.

now check i have maid the changes to it all you said

now sir pls check the changes you have said
this is commit i maid
https://github.com/girlscript/winter-of-contributing/pull/4225/commits/44cb77f5272e5d0c396109042bb01156b0d0a108

Sir i wan to add hacktorberfest label to it how can i do that

Sir i wan to add hacktorberfest label to it how can i do that

You don't need that, as the repository has the hacktoberfest topic 😄

Sir i wan to add hacktorberfest label to it how can i do that

You don't need that, as the repository has the hacktoberfest topic 😄

okey now if all okey then merge it sir long awaiting for it


Modal or Dialogbox component?

Would be great to have a modal component. Waiting from long time

Duplicate #781


Ensure that basebackup progress is set to 100 upon success.

Because of how the myhoard.basebackup.estimated_progress metric is calculated, the final value was not always 100 when a backup is successfully generated.
About this change: What it does, why it matters
This does not alter how backups work, just adds a final transmission of the progress metric after a basebackup has completed.
This is important as it allows prometheus metrics to accurately reflect whether a backup was successful.

This depends on #95 and will be merged in lockstep unless someone objects.

I've created this PR to have #95 as a target. If it's merged to master first, this PR will automatically be modified to merge to master.
I've also just pushed this commit, rebased off #95.


add pgh_operation field and delete snapshot event

First, thanks a lot for this package, I really wanted to build something similar for quite sometime.
I have a requirement for knowing what was the operation (insert, update or delete) and also track deletes snapshot.
I ended up splitting Snapshot into 3 other classes SnapshotInsert, SnapshotUpdate and SnapshotDelete so we could also use just the ones you need, like: @pghistory.track(pghistory.SnapshotDelete()).
What you guys think about it? Let me know if this is an interesting contribution and if I can improve the code in anyway.

@nossila thanks so much for your suggestion and code.
I've been unavailable for some time to contribute and am finally coming back around to merging in some of these PRs.
At a high level, I'm very cautious of accepting a change that modifies the default tracking tables. Some customers have hundreds of millions of snapshot rows, and they'd have to migrate all of them.
Let me come back to you with some thoughts on how we can try to handle this use case though. There may be something I didn't fully document that may support your use case well.
One question that comes to mind though - Is it possible for you to determine this information dynamically? Correct me if I'm wrong, but the the first snapshot of a row is always going to be an INSERT,  everything in between will be UPDATE, and the final one will be DELETE if the original record no longer exists. I believe the only time this case doesn't apply is for legacy data where you weren't tracking events.
Let me know though. I will come back with some more thoughts

Thought about it some more. I think this would fit better as a custom base model you set for every snapshot model in your project using this setting
I could see this being added to the library though because this base model would be special. I.e. it would either need to install different tracking triggers or the current triggers would need to be aware of it. The tracking triggers could also be updated to look for a special pgh_operation field like you have.
In other words, the change wouldn't affect any current users, and we could put a section in the docs on this other base model for users that need this additional metadata.
Let me know if that makes sense and perhaps I can propose or make some changes to the PR.

Hey wanted to follow up on this. pghistory.track is more configurable now, which I believe addresses the original intentions of this PR.
By default, pghistory.track will register  pghistory.InsertEvent() and pghistory.UpdateEvent() trackers. They both set pgh_label to insert or update, in effect accomplishing the functionality of tracking the type of operation without introducing a new field.
You can also override the trackers per-model (see the docs here) or use the PGHISTORY_DEFAULT_TRACKERS setting to override the default trackers everywhere.
I cleaned up some of the config/docs a bit more in https://github.com/Opus10/django-pghistory/pull/142, which will be deployed shortly
Hope this helps!


Support Pinecone

Try out Pinecone as an optional alternative to FAISS. Expected pros: smaller container, lower memory use. Expected cons: slower indexing and querying because of network latency, cost for large document stores.

From https://langchain.readthedocs.io/en/latest/ecosystem/pinecone.html:

Add client as dependency: pip install pinecone-client (add to README and Dockerfile)
Add import to microllama.py: from langchain.vectorstores import Pinecone

Initial impressions are the Pinecone is noticeably slower than FAISS. I'd need to refactor get_index to support multiple index types. The check for an existing index (pinecone.describe_index("index-name")) is ~0.7s.


update README.md

update README.md(smm→ssm)
Thank you for your useful code.

@mint3u Thanks for taking the time to submit this! I'm glad you find it useful!


fix (#patch); bridge sdk; fixed argument type for token

In tokens.ts, the argument for getOrCreateToken and getTokenParams is specified as type Address, but Token.id is specified as Bytes in the schema. This PR changes the argument type to Bytes.

Doesn’t it work as it is? Since Address inherits from Bytes.
In case it fails, instead of changing the type there I’d suggest changing it in tokens.ts where the Token.load() and new Token() are called, converting the Address into Bytes right where it is needed.
This way we avoid having the client worrying about converting from one into another, and also prevent them from mistakenly using non-address bytes.

Doesn’t it work as it is? Since Address inherits from Bytes. In case it fails, instead of changing the type there I’d suggest changing it in tokens.ts where the Token.load() and new Token() are called, converting the Address into Bytes right where it is needed.

It reported an error at building:

What you suggest makes sense. I reverted my change and converting Bytes to Address before passing to getOrCreateToken

I realized you're already doing the conversion, for example, in pool.ts. There are just two occurances in account.ts that you missed.

I realized you're already doing the conversion, for example, in pool.ts. There are just a few cases in account.ts that you missed.

Oh sounds good, my bad 👍


How can I inspect the sqlite database directly?

I'm interested in visualizing the transactions on the sqlite database, similar to what has been done with etcd in the past.
How can I inspect the sqlite db?
Thanks.

Try this: https://sqlitebrowser.org/

Thanks for the fast response, I would like to know if there is a service/
address/ port in k3s that I could point the query.
P.S. - I really like k3s.
On Thu, Apr 18, 2019, 10:32 William Zhang<EMAIL_ADDRESS>wrote:

Try this: https://sqlitebrowser.org/
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/rancher/k3s/issues/370#issuecomment-484321546, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABSRMG6J46V7NUNVGJPZN4LPQ7FSFANCNFSM4HGYMRNQ
.

 root@william  /home/fedora  ll /var/lib/rancher/k3s/server/db
total 4.9M
-rw-r--r-- 1 root root 852K Apr 18 05:40 state.db
-rw-r--r-- 1 root root  32K Apr 18 05:44 state.db-shm
-rw-r--r-- 1 root root 4.0M Apr 18 05:45 state.db-wal
 root@william  /home/fedora 

The default path for db files is: /var/lib/rancher/k3s/server/db, and it is defined in the source code:
https://github.com/rancher/k3s/blob/f6e0bad95300054ede990f7adc910f4de32ea923/vendor/github.com/ibuildthecloud/kvsql/clientv3/driver/sqlite/sqlite.go#L68
In contrast to many other database management systems, SQLite is not a client–server database engine.
So there are no service addresses:ports that provide external access like mysql, oracle, PostgreSQL.


Support MachO local relocations

In seemingly really, really old linker configurations, the MachO produced will use the local relocation table for relocations which is stored in the __LINKEDIT segment (which is referenced in the DYNSYM MachO load command).
Additionally it appears that when .dylibs contain the local relocation section they will also report themselves as non-PIE which is as far as I can tell INCORRECT because they can observably be relocated by the macOS linker using the local relocations section.
As well, Binary Ninja should type these relocations and display them in the linear view.

Could you attach or link to a small MachO with the properties you describe?

If you just set your SDK to be really old, you can build for yourself:
// sample.c
#include <stdint.h>
#include <stdlib.h>
#include <time.h>

// You _probably_ don't have these old crt1.o binaries.
uint64_t X asm("dyld_stub_binding_helper") = 0;

uint64_t SERIAL = 0xdeadbeef;

void transmogrify(uint64_t v) {
    srand(time(NULL));
    SERIAL ^= v & rand();
}

$ clang -shared -undefined dynamic_lookup -arch i386 -Wl,-macos_version_min,10.4 sample.c

(should also build an x86_64 variant with -arch x86_64 as they have a different set of relocations)
You can confirm that you've built a binary with the old relocation model using otool (modern binaries should have no output):
$ otool -v -r a.out
a.out:
Local relocation information 9 entries
address  pcrel length extern type    scattered symbolnum/value
00000f4c False long   False  VANILLA False     6 (__DATA,__la_symbol_ptr)
00000f52 False long   False  VANILLA False     6 (__DATA,__la_symbol_ptr)
00000f58 False long   False  VANILLA False     6 (__DATA,__la_symbol_ptr)
00000f5d False long   False  VANILLA False     6 (__DATA,__la_symbol_ptr)
00000f69 False long   False  VANILLA False     6 (__DATA,__la_symbol_ptr)
00000f75 False long   False  VANILLA False     6 (__DATA,__la_symbol_ptr)
00001000 False long   False  VANILLA False     3 (__TEXT,__stub_helper)
00001004 False long   False  VANILLA False     3 (__TEXT,__stub_helper)
00001008 False long   False  VANILLA False     3 (__TEXT,__stub_helper)

And to see that the "bug" is present:
>>> bv.perform_is_relocatable()
False

Binary Ninja will report that the executable is not relocatable (even though it is) and you will not be able to rebase it.

To resolve this, Binary Ninja should support local relocations and external relocations in the LC_DYSYMTAB command and parse both scattered and non-scattered relocations. If you have any questions about this antiquated relocation format, I've had to hack together my own simple linker for some analysis and the dyld linker source is a good ground truth (but I'm also happy to answer questions if I can).

Also see: https://github.com/Vector35/binaryninja-api/issues/1609 (as these new changes should be reflected in the MachO types).

sample_x64.zip
@meme do you know what the address field in these relocations refer to exactly?
$ otool -v -r ./sample_x64
./sample_x64:
Local relocation information 3 entries
address  pcrel length extern type    scattered symbolnum/value
00000000 False quad   False  UNSIGND False     3 (__TEXT,__stub_helper)
00000008 False quad   False  UNSIGND False     3 (__TEXT,__stub_helper)
00000010 False quad   False  UNSIGND False     3 (__TEXT,__stub_helper)

As offsets into the first segment's .vmaddr=0 field, 0, 8, and 10 are still in the header. As offsets into the first segment's first loaded data (section __text) at 0xEF0 the 0 is at the 0EF0: 55 push rbp and 8 is at the 0EF8: xor eax, eax so that can't be right. Where do these things point?
It's interesting llvm-readobj -r --expand-relocs ./sample_x64 doesn't display any of these.
Here's some wide output of a tool to display the struct relocation_info:
                                                                           struct relocation_info(dysymtab)
00002000 00 00 00 00                                      ....               r_address: 0x0
00002000             03 00 00 06                              ....           r_symbolnum: 0x3 section "__TEXT/__stub_helper"
                                                                             r_pcrel: 0
                                                                             r_length: 4 bytes (2)
                                                                             r_extern: 0
                                                                             r_type: X86_64_RELOC_UNSIGNED (0)
                                                                           struct relocation_info(dysymtab)
00002000                         08 00 00 00                      ....       r_address: 0x8
00002000                                     03 00 00 06              ....   r_symbolnum: 0x3 section "__TEXT/__stub_helper"
                                                                             r_pcrel: 0
                                                                             r_length: 4 bytes (2)
                                                                             r_extern: 0
                                                                             r_type: X86_64_RELOC_UNSIGNED (0)
                                                                           struct relocation_info(dysymtab)
00002010 10 00 00 00                                      ....               r_address: 0x10
00002010             03 00 00 06                              ....           r_symbolnum: 0x3 section "__TEXT/__stub_helper"
                                                                             r_pcrel: 0
                                                                             r_length: 4 bytes (2)
                                                                             r_extern: 0
                                                                             r_type: X86_64_RELOC_UNSIGNED (0)

The r_address points into the first writeable section. In the example it is __DATA. So the calculation is base_address + data_address + r_address. The r_address can also have tags, so make sure to check if the scattered bit is set (for scattered relocations: & R_SCATTERED != 0)

@meme Ah the first writeable section, thanks. Would you mind double checking, for the 64-bit attachment, that Binary Ninja is not applying the relocations?
I'm going to blab out loud here, so if I go wrong, the exact place can be identified:
Those three r_address locations point to PLT entries:
__DATA/__la_symbol_ptr:
1000: 0F4C
1008: 0F58
1010: 0F64

and those values are initially the addresses of instructions that jump through the PLT:
__TEXT/__stub_helper:
0F4C: lea	r11, qword ptr [rip + 0xad]; jmp dyld_stub_binding_helper
0F58: lea	r11, qword ptr [rip + 0xa9]; jmp dyld_stub_binding_helper
0F64: lea	r11, qword ptr [rip + 0xa5]; jmp dyld_stub_binding_helper

but after relocations (and in Binary Ninja) they're changed to:
__DATA/__la_symbol_ptr:
1000: 3000 ; extern _rand
1008: 3008 ; extern _srand
1010: 3010 ; extern _time

This all seems correct, and disassembly of transmogrify() correctly shows the calls to rand, srand, and time. Is it just that the return from perform_is_relocatable() is wrong?

@meme could you update to latest dev and try to rebase now? Note that bv.perform_is_relocatable() isn't meant to be called directly (and will show false), try the property bv.relocatable instead, which should return true now. Rebasing works on this end now, please confirm.
sample_x86.zip

Thanks for your help, it works as expected on latest now :smile:

Great, thanks for creating the issue and following up! See you on the next issue :P


Program received signal SIGSEGV in submodule h264bitstream/h264_stream.c

I like the idea of this software. Very nice. Since the repository seems to be active I'd like to report a segmentation fault in the h264bitstream/h264_stream.c sub-module.
I don't see how function read_slice_layer_rbsp( ) works as intended. I suspect this function makes an assumption about direction of malloc( ) buffers that is different from x86_64 architecture or I really have no clue what the software is doing.
This fragment on my system would evaluate to b->p or b->p +1 depending on value of bits_left

uint8_t *sptr = b->p + (!!b->bits_left)

So the following will always be equal to 0 or -1.

slice_data->rbsp_size = b->end - sptr

Of course, this causes the subsequent memcpy to segfault when rbsp_size is -1.

memcpy( slice_data->rbsp_buf, sptr, slice_data->rbsp_size );

SEGV Here:
(gdb) print b->bits_left
$5 = 3
(gdb) print b->p
$2 = (uint8_t *) 0x555555f441d6 ""
(gdb) print sptr
$3 = (uint8_t *) 0x555555f441d7 ""
(gdb) print slice_data->rbsp_size
$4 = -1
file named h264bitstream/h264_stream.c

//<IP_ADDRESS> Slice layer without partitioning RBSP syntax
void read_slice_layer_rbsp(h264_stream_t* h,  bs_t* b)
   ...
    if ( slice_data != NULL )
    {
        if ( slice_data->rbsp_buf != NULL ) free( slice_data->rbsp_buf );
        uint8_t *sptr = b->p + (!!b->bits_left); // CABAC-specific: skip alignment bits, if there are any
        slice_data->rbsp_size = b->end - sptr;

        slice_data->rbsp_buf = (uint8_t*)malloc(slice_data->rbsp_size);
        memcpy( slice_data->rbsp_buf, sptr, slice_data->rbsp_size );
        // ugly hack: since next NALU starts at byte border, we are going to be padded by trailing_bits;
        return;
    }

Details:
DISTRIB_ID=LinuxMint
DISTRIB_RELEASE=19.1
DISTRIB_CODENAME=tessa
Linux Hostname 4.15.0-43-generic #46-Ubuntu SMP Thu Dec 6 14:45:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
IP Camera: Reolink RLC-420-5MP
VLC Media Info
Stream: 0
Codec: H264 - MPEG-4 AVC (part 10) (h264)
Video Resolution: 640x480
Buffer Dimensions: 640x480
Decoded format:
Orientation: Top left
Chroma location: Left
Stream 1:
Codec: MPEG AAC Audio (mp4a)
Type: Audio
Channels: Stereo
Sample rate: 32000Hz
Bits per sample: 32
install live.2018.12.14.tar.gz

git clone https://github.com/mpromonet/v4l2rtspclient.git
cd v4l2rtspclient
make update
make

username@hostname:~/v4l2rtspclient$ ddd v4l2rtspclient &

set args rtsp://username:password@<IP_ADDRESS>:554//h264Preview_01_sub
run

Program received signal SIGSEGV, Segmentation fault.
__memcpy_ssse3 () at ../sysdeps/x86_64/multiarch/memcpy-ssse3.S:2812
(gdb)
Specified first and last lines are in different files.
(gdb) up
#1  0x000055555555a299 in read_slice_layer_rbsp (h=0x5555557952d0, b=0x555555f44120) at h264bitstream/h264_stream.c:900
(gdb) where
#0  __memcpy_ssse3 () at ../sysdeps/x86_64/multiarch/memcpy-ssse3.S:2812
#1  0x000055555555a299 in read_slice_layer_rbsp (h=0x5555557952d0, b=0x555555f44120) at h264bitstream/h264_stream.c:900
#2  0x00005555555586cc in read_nal_unit (h=0x5555557952d0, buf=0x7ffff645e014 "\001H \006\377", <incomplete sequence \360>, size=6) at h264bitstream/h264_stream.c:207
#3  0x0000555555572d3d in V4l2Writer::onData (this=0x7fffffffda10, id=0x555555f428d0 "liveMedia7", buffer=0x7ffff645e010 "", size=516, tv=...) at src/v4l2writer.cpp:60
#4  0x0000555555576fb6 in RTSPConnection::SessionSink::afterGettingFrame (this=0x555555f428c0, frameSize=512, numTruncatedBytes=0, presentationTime=..., durationInMicroseconds=0) at src/rtspconnectionclient.cpp:51
#5  0x0000555555578580 in RTSPConnection::SessionSink::afterGettingFrame (clientData=0x555555f428c0, frameSize=512, numTruncatedBytes=0, presentationTime=..., durationInMicroseconds=0) at inc/rtspconnectionclient.h:84
#6  0x00007ffff7b5ec78 in MultiFramedRTPSource::doGetNextFrame1() () from /usr/lib/x86_64-linux-gnu/libliveMedia.so.62
#7  0x00007ffff7b5ed8a in MultiFramedRTPSource::networkReadHandler1() () from /usr/lib/x86_64-linux-gnu/libliveMedia.so.62
#8  0x00007ffff74b3c63 in BasicTaskScheduler::SingleStep(unsigned int) () from /usr/lib/x86_64-linux-gnu/libBasicUsageEnvironment.so.1
#9  0x00007ffff74b2e3c in BasicTaskScheduler0::doEventLoop(char volatile*) () from /usr/lib/x86_64-linux-gnu/libBasicUsageEnvironment.so.1
#10 0x0000555555578797 in Environment::mainloop (this=0x7fffffffdab0) at src/environment.cpp:30
#11 0x000055555557290e in main (argc=2, argv=0x7fffffffdfd8) at src/v4l2rtspclient.cpp:74

Hi Nikole,
The live555 process only video/H264 stream, the audio stream is not given to h264bitstream.
see V4l2Writer::onNewSession .
Best Regards,
Michel.

Understood. Will raise a problem report with https://github.com/aizvorski/h264bitstream to get feedback.
For now I will continue prototyping with ffmpeg and then circle back to v4l2rtspclient when I settle on deployment hardware. There are far too many blocking problems to solve before ffmpeg or v4l2rtspclient component selection becomes a priority.

Hi Nikole,
The buffer seems strange "\001H \006\377\360".
The first byte means :

forbidden_zero_bit = 0
nal_ref_idc = 0
nal_unit_type = 1
From https://tools.ietf.org/html/rfc6184
+---------------+
|0|1|2|3|4|5|6|7|
+-+-+-+-+-+-+-+-+
|F|NRI|  Type   |
+---------------+

The buffer is surely too small for a SLICE, probably something is going wrong before H264 parsing.
I look to the line number and it seems you update h264bitstream instead of using the submodule link.
So I tried with the latest one, that have similar number and it doesnot seems to work properly, the SPS is not decoded in my tests.
Best Regards,
Michel.

Hi Nikole,
I fix the overload problem of onNewSession  in commit https://github.com/mpromonet/v4l2rtspclient/commit/38352336ec24e82811aafbcaaee44703f2392bb3
It will avoid to not parse audio as an H264 stream.
Best Regards,
Michel.

Thanks for the RFC reference.
Attempting to test your changes. Unfortunately I've added extra SSD storage to my development platform using LVM so I can't just use timeshift to easily go back in time to the original environment. My apologies. I know this makes my development system a bit of a moving target. Not ideal.
The mint packager has updated the kernel and kernel header files

Linux Elvis 4.15.0-45-generic #48-Ubuntu SMP Tue Jan 29 16:28:13 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Live555 has updated their tarball:

wget http://www.live555.com/liveMedia/public/live.2019.02.03.tar.gz
tar -zxvf live.2019.02.03.tar.gz
cd live
./genMakefiles linux-64bit
make
sudo make --environment-overrides PREFIX=/usr install

New kernel means new v4l2loopback device.

modinfo v4l2loopback
filename:       /lib/modules/4.15.0-45-generic/extra/v4l2loopback.ko
license:        GPL
author:         Vasily Levin, IOhannes m zmoelnig zmoelnig@iem.at,Stefan Diewald,Anton Novikovet al.
description:    V4L2 loopback video device
srcversion:     C60D15B48372C3244BCC995
depends:        videodev
retpoline:      Y
name:           v4l2loopback
vermagic:       4.15.0-45-generic SMP mod_unload
parm:           debug:debugging level (higher values == more verbose) (int)
parm:           max_buffers:how many buffers should be allocated (int)
parm:           max_openers:how many users can open loopback device (int)
parm:           devices:how many devices should be created (int)
parm:           video_nr:video device numbers (-1=auto, 0=/dev/video0, etc.) (array of int)
parm:           card_label:card labels for every device (array of charp)
parm:           exclusive_caps:whether to announce OUTPUT/CAPTURE capabilities exclusively or not (array of bool)
parm:           max_width:maximum frame width (int)
parm:           max_height:maximum frame height (int)

Rebuild v4l2rtspclient from git repository:

git clone https://github.com/mpromonet/v4l2rtspclient.git
cd v4l2rtspclient
make

Verifed new interface signature is
bool V4l2Writer::onNewSession(const char* id, const char* media, const char* codec, const char* sdp)

./v4l2rtspclient -vv rtsp://user:password@<IP_ADDRESS>:554/h264Preview_01_sub
log level:700
Created new TCP socket 3 for connection
Connecting to <IP_ADDRESS>, port 554 on socket 3...
...remote connection opened
Sending request: DESCRIBE rtsp://user:password@<IP_ADDRESS>:554/h264Preview_01_sub RTSP/1.0
CSeq: 2
User-Agent: LIVE555 Streaming Media v2019.02.03
Accept: application/sdp
Received 175 new bytes of response data.
Received a complete DESCRIBE response:
RTSP/1.0 401 Unauthorized
CSeq: 2
Date: Wed, Feb 06 2019 16:46:19 GMT
WWW-Authenticate: Digest realm="LIVE555 Streaming Media", nonce="6a63b6ac2a0b3fd9369fe3e98a07f449"
Resending...
Sending request: DESCRIBE rtsp://user:password@<IP_ADDRESS>:554/h264Preview_01_sub RTSP/1.0
CSeq: 3
Authorization: Digest username="v4l2", realm="LIVE555 Streaming Media", nonce="6a63b6ac2a0b3fd9369fe3e98a07f449", uri="rtsp://user:password@<IP_ADDRESS>:554/h264Preview_01_sub", response="b11a3a2799b441441fc6f18e272b6329"
User-Agent: LIVE555 Streaming Media v2019.02.03
Accept: application/sdp
Received 883 new bytes of response data.
Received a complete DESCRIBE response:
RTSP/1.0 200 OK
CSeq: 3
Date: Wed, Feb 06 2019 16:46:19 GMT
Content-Base: rtsp://<IP_ADDRESS>/h264Preview_01_sub/
Content-Type: application/sdp
Content-Length: 711
v=0
o=-<PHONE_NUMBER>633263 1 IN IP4 <IP_ADDRESS>
s=Session streamed by "preview"
i=h264Preview_01_sub
t=0 0
a=tool:LIVE555 Streaming Media v2013.04.08
a=type:broadcast
a=control:*
a=range:npt=0-
a=x-qt-text-nam:Session streamed by "preview"
a=x-qt-text-inf:h264Preview_01_sub
m=video 0 RTP/AVP 96
c=IN IP4 <IP_ADDRESS>
b=AS:500
a=rtpmap:96 H264/90000
a=fmtp:96 packetization-mode=1;profile-level-id=640033;sprop-parameter-sets=Z2QAM6zoCgPZ,aO48sA==
a=control:trackID=1
m=audio 0 RTP/AVP 97
c=IN IP4 <IP_ADDRESS>
b=AS:256
a=rtpmap:97 MPEG4-GENERIC/16000
a=fmtp:97 streamtype=5;profile-level-id=15;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3;config=1408; profile=1;
a=control:trackID=2
Sending request: SETUP rtsp://<IP_ADDRESS>/h264Preview_01_sub/trackID=1 RTSP/1.0
CSeq: 4
Authorization: Digest username="user", realm="LIVE555 Streaming Media", nonce="6a63b6ac2a0b3fd9369fe3e98a07f449", uri="rtsp://<IP_ADDRESS>/h264Preview_01_sub/", response="4a4f8d725000fe3b9d4b5de2ee7b66ff"
User-Agent: LIVE555 Streaming Media v2019.02.03
Transport: RTP/AVP;unicast;client_port=57340-57341
Received 202 new bytes of response data.
Received a complete SETUP response:
RTSP/1.0 200 OK
CSeq: 4
Date: Wed, Feb 06 2019 16:46:19 GMT
Transport: RTP/AVP;unicast;destination=<IP_ADDRESS>;source=<IP_ADDRESS>;client_port=57340-57341;server_port=6974-6975
Session: C40026F2
Start playing sink for "video/H264" subsession
markerSize:4
Sending request: SETUP rtsp://<IP_ADDRESS>/h264Preview_01_sub/trackID=2 RTSP/1.0
CSeq: 5
Authorization: Digest username="user", realm="LIVE555 Streaming Media", nonce="6a63b6ac2a0b3fd9369fe3e98a07f449", uri="rtsp://<IP_ADDRESS>/h264Preview_01_sub/", response="4a4f8d725000fe3b9d4b5de2ee7b66ff"
User-Agent: LIVE555 Streaming Media v2019.02.03
Transport: RTP/AVP;unicast;client_port=55066-55067
Session: C40026F2
Received 202 new bytes of response data.
Received a complete SETUP response:
RTSP/1.0 200 OK
CSeq: 5
Date: Wed, Feb 06 2019 16:46:19 GMT
Transport: RTP/AVP;unicast;destination=<IP_ADDRESS>;source=<IP_ADDRESS>;client_port=55066-55067;server_port=6976-6977
Session: C40026F2
Sending request: PLAY rtsp://<IP_ADDRESS>/h264Preview_01_sub/ RTSP/1.0
CSeq: 6
Authorization: Digest username="user", realm="LIVE555 Streaming Media", nonce="6a63b6ac2a0b3fd9369fe3e98a07f449", uri="rtsp://<IP_ADDRESS>/h264Preview_01_sub/", response="1fa0409130ebd709f01108ee42cbc25c"
User-Agent: LIVE555 Streaming Media v2019.02.03
Session: C40026F2
Range: npt=0.000-
Received 252 new bytes of response data.
Received a complete PLAY response:
RTSP/1.0 200 OK
Server: Rtsp Server/2.0
CSeq: 6
Date: Wed, Feb 06 2019 16:46:19 GMT
Range: npt=0.000-
Session: C40026F2
RTP-Info: url=trackID=1;seq=22134;rtptime=4046275296;ssrc=968c217f,url=trackID=2;seq=61687;rtptime=633753551;ssrc=c055e821
[DEBUG] src/v4l2writer.cpp:61
NAL:7
[NOTICE] src/v4l2writer.cpp:67
VIDIOC_STREAMOFF: Bad file descriptor
VIDIOC_REQBUFS: Bad file descriptor
NOTIFY failed
geometry:640x480
[DEBUG] src/v4l2writer.cpp:61
NOTIFY failed
NAL:8
[DEBUG] src/v4l2writer.cpp:61
NOTIFY failed
NAL:5
[DEBUG] src/v4l2writer.cpp:61
NOTIFY failed
NAL:1
[DEBUG] src/v4l2writer.cpp:61
NOTIFY failed
NAL:1
[DEBUG] src/v4l2writer.cpp:61
NOTIFY failed
NAL:1
[DEBUG] src/v4l2writer.cpp:61
NOTIFY failed
NAL:1
[DEBUG] src/v4l2writer.cpp:61
NOTIFY failed
NAL:1
[DEBUG] src/v4l2writer.cpp:61
NOTIFY failed
NAL:1
^C

VLC is unable to open the MRL 'v4l2:///dev/video0'. Check the log for details.
Your input can't be opened:
Verify that the environment is functional by comparing to ffmpeg

./ffmpeg -i rtsp://user:password@<IP_ADDRESS>:554//h264Preview_01_sub -vcodec copy -an -f v4l2 /dev/video0
ffmpeg version N-93044-g2e2b44baba Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 7 (Ubuntu 7.3.0-27ubuntu1~18.04)
configuration:
libavutil      56. 26.100 / 56. 26.100
libavcodec     58. 46.100 / 58. 46.100
libavformat    58. 26.100 / 58. 26.100
libavdevice    58.  6.101 / 58.  6.101
libavfilter     7. 48.100 /  7. 48.100
libswscale      5.  4.100 /  5.  4.100
libswresample   3.  4.100 /  3.  4.100
Input #0, rtsp, from 'rtsp://user:password@<IP_ADDRESS>:554//h264Preview_01_sub':
Metadata:
title           : Session streamed by "preview"
comment         : h264Preview_01_sub
Duration: N/A, start: 0.000000, bitrate: N/A
Stream #0:0: Video: h264 (High), yuv420p(progressive), 640x480, 24.75 tbr, 90k tbn, 180k tbc
Stream #0:1: Audio: aac (LC), 16000 Hz, mono, fltp
Output #0, video4linux2,v4l2, to '/dev/video0':
Metadata:
title           : Session streamed by "preview"
comment         : h264Preview_01_sub
encoder         : Lavf58.26.100
Stream #0:0: Video: h264 (High), yuv420p(progressive), 640x480, q=2-31, 24.75 tbr, 90k tbn, 90k tbc
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
[video4linux2,v4l2 @ 0x559a493fd480] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
frame=   69 fps= 20 q=-1.0 Lsize=N/A time=00:00:05.73 bitrate=N/A speed= 1.7x
video:391kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

VLC is able to open the MRL 'v4l2:///dev/video0'

Hi Nikole ,
As I understood it doesnot crash since the modification, isn"t it ?
Did you try to specify output device using something like :
./v4l2rtspclient -vv rtsp://user:password@<IP_ADDRESS>:554/h264Preview_01_sub /dev/video0

By default it use /dev/video1.
Best Regards,
Michel.

You are correct, it does not SEGV since the modification. Very good! Thank you!
I mistakenly used v4l2rtspclient default /dev/video1 even though modprobe default loopback device is /dev/video0. My ignorance of the video loopback device workflow and architecture is the problem. :-)
Would suspect installation of v4l2loopback however ffmpeg uses the same device setup without issue. ffmpeg is verified to be working immediately after testing v4l2rtspclient as shown below.
./v4l2rtspclient -vv -w<EMAIL_ADDRESS>/dev/video0

[snip]
Received 253 new bytes of response data.

Best Regards,
Michel.

Yes. Results include with and without -w. Log is slightly different for each option. Inspecting V4l2MmapDevice.cpp error codes with gdb to see what can be learned.

OK. Spent some time learning some things about v4l2 and shared memory buffers. The nuance of how and when to use 'bytesused' vs 'length' in 'struct v4l2_buffer' currently escapes me.
https://static.lwn.net/kerneldoc/media/uapi/v4l/buffer.html#struct-v4l2-buffer

Breakpoint 3, V4l2MmapDevice::start (this=0x5555558b3140) at src/V4l2MmapDevice.cpp:91
(gdb) p req
$15 = {count = 8, type = 2, memory = 1, reserved = {0, 0}}

Breakpoint 3, V4l2MmapDevice::start (this=0x5555558b3140) at src/V4l2MmapDevice.cpp:91
(gdb) p buf
$7 = {index = 0, type = 2, bytesused = 1228800, flags = 2, field = 1, timestamp = {tv_sec =<PHONE_NUMBER>, tv_usec = 753075}, timecode = {type = 0, flags = 0, frames = 0 '\000', seconds = 0 '\000', minutes = 0 '\000', hours = 0 '\000', userbits = "\000\000\000"}, sequence = 0, memory = 1, m = {offset = 0, userptr = 0, planes = 0x0, fd = 0}, length = 0, reserved2 = 0, reserved = 0}
(gdb) c
Continuing.
mmap: Invalid argument
Device /dev/video0 buffer idx:0 size:0
[INFO] src/V4l2MmapDevice.cpp:90

Breakpoint 3, V4l2MmapDevice::start (this=0x5555558b3140) at src/V4l2MmapDevice.cpp:91
(gdb) p buf
$8 = {index = 1, type = 2, bytesused = 1228800, flags = 2, field = 1, timestamp = {tv_sec =<PHONE_NUMBER>, tv_usec = 753076}, timecode = {type = 0, flags = 0, frames = 0 '\000', seconds = 0 '\000', minutes = 0 '\000', hours = 0 '\000', userbits = "\000\000\000"}, sequence = 0, memory = 1, m = {offset = 0, userptr = 0, planes = 0x0, fd = 0}, length = 0, reserved2 = 0, reserved = 0}

gdb) c
Continuing.
mmap: Invalid argument
Device /dev/video0 buffer idx:1 size:0
[INFO] src/V4l2MmapDevice.cpp:90

Workaround
#!/bin/bash
sudo modprobe -r v4l2loopback
sudo modprobe v4l2loopback
v4l2rtspclient "$@"


Log prints functions as undefined

The following code return the log:
undefined
undefined
Test

but should be something like:
function()
function()
Test

Code:
function test() {log("Test")}

class myClass {
    constructor(){}

    func(a){
        log(a)
        a()
    }
}

log(test)

const mc = new myClass()
mc.func(test)

decentraland-ecs 6.6.3

It is not a bug, log can only be used to print JSON-serializable values

It is not a bug, log can only be used to print JSON-serializable values


Fix bug with duplicated hints

In gmail for example I occasionally have a duplicate hint on the right hand side panel

I managed to fix this by, after each hints update, check if there's duplicate hint nodes and doing a full hints refresh if that's the case. It's not ideal and I would like to find the root cause but the issue is hard to reproduce

That's another alternative, but then I'd had to loop through all the hints everytime I assign one. I figured is cheaper just to check once at the end

Yes, probably a set would be quick enough but I'd need to ask the background script for a new hint and it would add unnecessary complexity. This way if we find a duplicate hint we just reset everything.
But I think checking before assigning a hint would be a good way to find the cause of the issue. I can add a breakpoint and see the state that led to that. Going to Gmail and scrolling up and down usually produces the error, it might take a couple of minutes of scrolling and clicking around but at least I can reproduce it.

Sounds like a plan!


fix concatlatentmodel fail to construct vector of non-prefix models

This fixes the bug identified #353 . The origin of the bug was a vector of models with defined eltype trying to write back into itself with a different eltype.
I've made the bug a unit test.
Closes #353

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 93.15%. Comparing base (e1dc5f1) to head (f436122).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #355      +/-   ##
==========================================
- Coverage   93.18%   93.15%   -0.03%
==========================================
  Files          50       50
  Lines         528      526       -2
==========================================
- Hits          492      490       -2
  Misses         36       36

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Only thing here is using map vs array comprehension but I really don't think that makes much difference!


custom Font Family for labels?

Not able to apply custom fontFamily to the Label Text

I had the same problem

use custom label
renderLabel = params => {
return (

{this.state.labels[params.position]}

);
};