ADBDEV-5484: Fix deadlock by shared SegmentGeneral CTE & planning failure by shari…

Depending on the usage of the shared CTE with General or SegmentGeneral
subplan, the number of segments the CTE is executed on may be single for joins
with Singleton nodes, and multiple for nodes with other types of locus. In the
current implementation this may lead to deadlock if the CTE is used for both
join targets: the Join with the Singleton node results in the Share Input Scan
producer being executed with 1-gang, while the Join with the N-gang node
creates Share Input Scan reader on multiple segments, so the plan execution
hangs for the SegmentGeneral, and throw error inside
shareinput_mutator_xslice_2() functions for the General. If we force execution
of CTE on multiple segments, it will cause redundant motions in case of
joining the CTE with another General or SegmentGeneral. At the moment, we do not
have the way to execute a section of the plan on the N segments and take the
result from only one. Because replicated tables are considered small, the most
universal and optimal way to fix deadlock would be to inline CTE scans with
General or SegmentGeneral locus.
Inline is implemented by adding subpath to CtePath and removing subroot from
cteplaninfo. This should be done after processing the volatile functions in the
target list and having qual so as not to do inline in case this part of the plan
will still be executed on singleQE.
This is forward port of 6cac355c7cfc4684018b210697cec37b0d7dd021
Differences from 6X:
The inline method is adapted for 7X.
The test for replicated tables has been changed so that it reproduces the
problem on 7X.
Do not turn off the optimizer in the test because in 7X we have the expected
output for the test with the optimizer.
Co-authored-by: Vyacheslav Kompan<EMAIL_ADDRESS>

If it's a cherry pick with modifications, then I think the patch should consist of just one commit.

Only the approach and tests are taken from the original patch. The original author is listed as a co-author. Is that not enough?

Only the approach and tests are taken from the original patch. The original author is listed as a co-author. Is that not enough?

Why not make the patch as a cherry-pick with changes?

Add title to PR description and also something like "Note: do not squash to preserve authorship"

shareinput_mutator_xslice_2() functions

maybe "shareinput_mutator_xslice_2 function"?

Maybe I don't understand something or do something wrong, but new tests pass without a patch with the same plans and results.

Maybe I don't understand something or do something wrong, but new tests pass without a patch with the same plans and results.

Are you sure you ran the tests with the optimizer turned off? Because this patch does not affect the behavior of ORCA.

Maybe I don't understand something or do something wrong, but new tests pass without a patch with the same plans and results.

Are you sure you ran the tests with the optimizer turned off? Because this patch does not affect the behavior of ORCA.

yes I run sql
set optimizer=off;

CREATE TABLE d (a int, b int) DISTRIBUTED BY (a);
INSERT INTO d VALUES ( 1, 2 ),( 2, 3 );
CREATE TABLE r (a int, b int) DISTRIBUTED REPLICATED;
INSERT INTO r VALUES ( 1, 2 ),( 3, 4 );

-- Test if sharing is disabled for a SegmentGeneral CTE to avoid deadlock if CTE is
-- executed with 1-gang and joined with n-gang
EXPLAIN (COSTS off)
WITH cte AS (
    SELECT a FROM r
) SELECT * FROM cte JOIN (SELECT * FROM d JOIN cte USING (a) LIMIT 1) d_join_cte USING (a);

WITH cte AS (
    SELECT a FROM r
) SELECT * FROM cte JOIN (SELECT * FROM d JOIN cte USING (a) LIMIT 1) d_join_cte USING (a);

-- Test if sharing is disabled for a General CTE to avoid deadlock if CTE is
-- executed with coordinator gang and joined with n-gang
EXPLAIN (COSTS OFF)
WITH cte AS (
    SELECT count(*) a FROM (VALUES ( 1, 2 ),( 3, 4 )) v
)
SELECT * FROM cte JOIN (SELECT * FROM d JOIN cte USING (a) LIMIT 1) d_join_cte USING (a);

WITH cte AS (
    SELECT count(*) a FROM (VALUES ( 1, 2 ),( 3, 4 )) v
)
SELECT * FROM cte JOIN (SELECT * FROM d JOIN cte USING (a) LIMIT 1) d_join_cte USING (a);

set gp_cte_sharing = on;?


check symfony integration and handlers management

o2o for command and query buses
o2m for domain event bus
it must be checked on clear cache

done, it works at runtime only.


[Bug]: Distrod replaced my ZSH shell with its own Bash

Describe the bug
Distrod replaced its own Bash
The following shell is there now
kruse:x:1000:1000:create with care by Ansible:/home/kruse:/opt/distrod/alias/bin/bash

It used to be:
kruse:x:1000:1000:create with care by Ansible:/home/kruse:/usr/bin/zsh

Why was this change needed? Can I change it back without losing systemd? Is there a zsh alternative?
Can this behavour at least be documented?
I would rather not do an exec zsh in my bashrc.
Steps to reproduce

Install Distrod in an existing Debian destribution.
Enable it.

Expected behavior
I have:

a zsh alias as alternative
just my own defined shell but with running systemd

Windows version
11 21h2
Linux kernel version
Linux xios <IP_ADDRESS>-microsoft-standard-WSL2 #1 SMP Wed Aug 25 23:20:18 UTC 2021 x86_64 GNU/Linux
Distro
Debian 11
How did you install that distro?
Enabled distrod in an existing WSL2 distro
Logs
I think this is default programmed behaviour by distrod. No reason to include any non related logs.
additional comment
My distro was created from the default store version with  [WSL debian boxes Powershell scripts]( (https://github.com/nicenemo/wsl-debian-boxes). Except changing the Debian mirror and updating the image it does nothing special. It allows me to create multiple Debian instances side by side.

Hi Hans, thanks for trying Distrod.
Firstly, indeed Distrod will replace the default shell to its alias by design, but it should not replace zsh with bash, or it should not change any shell to an alias to different shell. Was the default shell really zsh when you installed Distrod?

Can I change it back without losing systemd?

No, that alias is necessary for Distrod to hook each WSL session. It's loosely documented in "How Distrod works" section in the README. I will improve the docs to make it more detailed. By the way, there is another possible approach to hook sessions, such as bind-mounting the default shell. The current shell hook approach is adopted because it is simpler and it I think is easier for users to understand how Distrod works.

Is there a zsh alternative?

Yes, of course. You can just run chsh and change your default shell to zsh or any your favorite shell. chsh will ask you to run sudo /opt/distrod/bin/distrod enable after that. See #7 for the detail.
Actually, these replaced shells work as just aliases to the actual shells. They're not some custom implementations.
However,


Adding notes and links to Office Add-ins documentation

Hi there - I work on the docs for Office Add-ins (https://github.com/OfficeDev/office-js-docs-pr). Data from our customer surveys and interviews show that most people who have used previous versions of related Office technology do not discover the modern Office Add-in docs, so we are trying to drive discoverability by cross-linking in some other docsets. We've done a couple notes before in this docset for VSTO; this PR continues that work, using the same note as before but on additional pages.
We've selected these pages through analysis of Adobe Analytics data (they may seem kind of random by just looking at them). I added an "includes" file for the note and changed the previous notes to use the includes file, so that the note content is centralized and can be changed easily across all files if needed.
Let me know if you have any questions. Unfortunately I will be going on medical leave soon but you can also email @davidchesnut or @Reezaali with questions. Thanks!

Hi @mscharlock
Microsoft authors and contributors need to make content contributions through the private repository so the content can be staged and validated by the current validation rules.
After you review staging and fix all validation issues, you can add the sign-off comment to let PR reviewers know that the updates are ready to be merged.
For more information, see Should I work in a private or a public repo?
If you need help getting set up to work in the private repo, see Set up Git repository locally for documentation.
Would you create a pull request in the private repo to make these updates? We'll close this PR. Thank you.

cc @gewarren @ghogen

So you need to approve me as a contributor or something to this repo in order for me to push to the private repo. Otherwise, I have to fork like I've done here. Also in the forked situation, I can't see the validation problems, because I can't see the build report. Can this get done ASAP? I apologize for the urgency but I am going on medical leave as of tomorrow. Happy to chat more via email.

@mscharlock @gewarren @ghogen We're in the process of moving this commit to the private repo. I'll post information about the new PR here when the PR has been created. Then, we'll close this PR.

So you need to approve me as a contributor or something to this repo in order for me to push to the private repo. Otherwise, I have to fork like I've done here. Also in the forked situation, I can't see the validation problems, because I can't see the build report. Can this get done ASAP? I apologize for the urgency but I am going on medical leave as of tomorrow. Happy to chat more via email.

Yes you'll need to work in a fork of the private repo; we all do. Can you elaborate on why you can't see the validation problems in a fork? The PR will have a link to the build report.

Yeah as a forked contributor all you see is "Merging is blocked, the base branch restricts merging to authorized users" and you don't get a build report.

Yeah as a forked contributor all you see is "Merging is blocked, the base branch restricts merging to authorized users" and you don't get a build report. What you do see is that CLA requirements are met or not, but it doesn't tell you about the actual issues in the PR if there are any.

There is no build validation in the public repo. You'll get it in the private repo, which is why we ask MS folks to use the private repo.

Private repo PR #https://github.com/MicrosoftDocs/visualstudio-docs-pr/pull/5211 replaces this PR. After we get a successful build in the new PR, we can close this PR.

this was recreated by @ghogen in the private repo https://github.com/MicrosoftDocs/visualstudio-docs-pr/pull/5215


Engagement  auto save errors

The auto save is woefully inadequate.
It makes API mutation calls in response user actions AND data updates.
It calls immediately causing necessary mutations while the user is typing.
It looks to send a mutation of empty while the component is unmounting, clearing a saved value.
https://app.logrocket.com/x6d22y/cord-prod/s/5-5cd7ac0b-ec88-4098-9892-7db97ee8acfc/0?fromTab=sessions&network_id=0-xhr-19&network_root_tab_id=36d0ec86-eb85-4d55-9974-504a946796c8&network_tab_id=36d0ec86-eb85-4d55-9974-504a946796c8&t=1651703593325.101&timeRange=86400000

This is caused by an error in DateField's initial value handling. in lieu of time & priorities I'm going to call this done with #1120 and we can circle back to a different auto save implementation later.


AMGX and serac::buildBallMesh on a second order H1 space do not place nice.

There seems to be a strange behavior where AMGX does not play nice with a mesh generated from serac::buildBallMesh(10000) when an order (2+) H1 space is used.
I've dumped of the mesh generated by serac::buildBallMesh(10000) and also one where I ran pmesh->ReorientTetMesh() and they are in <root>/data/meshes/ in the branch [feature/jonwong12/amgx_playground](SHA: b4e165ab0682e0246dc1eb9a23968076b1250f2a).
I added a modified mfem_ex1p example that can be run to see the observed behavior. Some of the more interesting defaults are: -no-amgx, order=2, and , -m data/meshes/star.mesh.
./tests/mfem_ex1p -o <order> [-amgx | -no-amgx] [-m ../data/meshes/ballmesh.mesh]

Observations:

order=1 should work on star.mesh, ballmesh.mesh and ballmesh.reorder.mesh with both -amgx and without (-no-amgx).
order=2 should work on star.mesh, ballmesh.mesh and ballmesh.reorder.mesh without (-no-amgx), but only star.mesh with -amgx.

It seems the default configuration isn't guaranteed to work. So it may take some experimentation to use a different amgx smoother (mfem conversation and resolution)
{
    "config_version": 2,
    "solver": {
        "solver": "AMG",
        "smoother": "JACOBI_L1",
        "presweeps": 1,
        "interpolator": "D2",
        "max_iters": 2,
        "scope": "main",
        "max_row_sum": 0.9,
        "strength_threshold": 0.25,
        "max_levels": 1000,
        "postsweeps": 1,
        "tolerance": 0.0,
        "norm": "L1",
        "cycle": "V"
    }
}


make the terms and conditions page responsive

Fixes #1058
Before :

After :

@Its-Aman-Yadav please review my pull request and merge it


Missing deployment slot support for az functionapp update

Describe the bug
Hi there!
When attempting to update one of the properties on my function app, I found that az functionapp update does not have slot support. To attempt to work around this I tried using az webapp (which does have slot support) but it gave me an error telling me to use az functionapp instead.
Other commands that may/may not need slot support:
az functionapp config hostname get-external-ip
az functionapp deployment user set
az functionapp deployment user show
az functionapp devops-pipeline create
az functionapp function *

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.
az functionapp update --resource-group <rg> --name <app> --slot staging --set siteConfig.healthCheckPath=/api/HealthCheck

Errors:
az: error: unrecognized arguments: --slot staging

To reiterate, az webapp update does have slot support.
A separate issue that I don't need addressed is that passing in the resourceId of the slot gets ignored, e.g.
az functionapp update --ids /subscriptions/{SubscriptionId}/resourceGroups/{ResourceGroup}/providers/Microsoft.Web/sites/{FunctionApp}/slots/staging --set siteConfig.healthCheckPath=/api/HealthCheck

Ends up silently updating the production slot instead of the staging slot. I imagine this is probably the behavior across all commands though. This is undesired behavior because it will cause an app restart on the production slot when an out-of-rotation slot should have been updated+restarted instead.
Expected Behavior
Update the config on the slot as expected in the same way that it does on the production slot.
Environment Summary
Windows-10-10.0.19041-SP0
Python 3.6.8
Installer: MSI

azure-cli 2.11.0 *

Extensions:
front-door 1.0.9

Additional Context
Thanks for your help!

functionapp

@gzuber could you please take a look at this?

Hi team,
I'm having the same issue that az functionapp update does not support updating httpsonly for slot. Any update on this?
Version              Name                                Repository           Description

2.0.0                Az.Functions                        PSGallery            Microsoft Azure PowerShell - Azure Functions service cmdlets for Azure Resource Manager in Windows PowerShell and PowerShell Core....

do we have any updates / ways to make this work?
updating staging slot is something we would love to use in our environment.

@kevin-secrist Apologies for the late reply. The similar issue has been discussed here and I see the PR has been merged now https://github.com/Azure/azure-cli/pull/21698. Please let me know if you are still facing the same issue using most recent version of the AzCLI. Awaiting your reply.

@kevin-secrist Thanks for getting back. Just checked the details about the PR https://github.com/Azure/azure-cli/pull/21698 and got a confirmation that the fix is yet to be released. Sending you the next release milestone details: https://github.com/Azure/azure-cli/milestone/116 which is planned for April 6th. Hope this helps. In the meantime, if you have any questions on this, please let me know.

May be fixed by https://github.com/Azure/azure-cli/pull/22745, will retest upon the next release

Tried to reproduce again, created a new test app using instructions from above:
$ az --version
azure-cli                         2.38.0

core                              2.38.0
telemetry                          1.0.6

Dependencies:
msal                            1.18.0b1
azure-mgmt-resource             21.1.0b1

Python location '/opt/az/bin/python3'
Extensions directory '/home/kevin/.azure/cliextensions'

Python (Linux) 3.10.5 (main, Jul  1 2022, 04:14:52) [GCC 11.2.0]

Legal docs and information: aka.ms/AzureCliLegal

Your CLI is up-to-date.

Before Updating Config
$ az functionapp show --resource-group azclitest070622 --name azclitest070622 | jq .siteConfig.healthCheckPath
null
$ az functionapp show --resource-group azclitest070622 --name azclitest070622 --slot staging | jq .siteConfig.healthCheckPath
null

Update Config
az functionapp update --resource-group azclitest070622 --name azclitest070622 --slot staging --set siteConfig.healthCheckPath=/api/HealthCheck

After Update
$ az functionapp show --resource-group azclitest070622 --name azclitest070622 | jq .siteConfig.healthCheckPath
null
$ az functionapp show --resource-group azclitest070622 --name azclitest070622 --slot staging | jq .siteConfig.healthCheckPath
"/api/HealthCheck"

✔ Looks like the staging slot took the setting instead of the production slot
Newest version (2.38.0) fixes the issue, fixed by https://github.com/Azure/azure-cli/pull/22745, thanks @haroonf


everything that has 1 layer of pixels are actually pretty pixxely

forgot to add that this is also caused by Betatweaks

Issues relating to add-ons should be fixed on the add-on's end in most cases.


Added legal notice to homepage

Added a simple legal disclaimer to the home-page.
Closes issue #66

You meant this closes issue #63 ... closing it now.


Youtube 60 Fps support

Hi, not sure if an issue or a feature request, but livestreamer doesn't recognize 60fps quality avaible for some youtube videos.
Since firefox support is still experimental and I can't watch 60fps gameplays on it and I do not want install chrome, this support would be very useful for me.
p.s. if you don't know what I am talking about enable "media.mediasource.enabled" from about:config and you'll see the 60fps quality in video options (for videos that support it, just search a 60fps gameplay to see it)

See #223

Not possible until we support MPEG-DASH, we'll use #223 to track the issue.


Plugins download error

Coverage increased (+0.01%) to 47.152% when pulling fab491b59201116ad85ecb575849bc3ef4ba048f on compae:bug/plugins-download into c18c69fc6916e642bcfc8549f10645937f7c1044 on Stratio:branch-1.2.


Add Track Compression Option

This adds an option to set whether or not a track will be compressed when muxing. The default behavior is to not pass the --compression flag and let mkvmerge decide whether or not to use compression. This change would allow the user to manually enable or disable compression, which would set the --compression flag to either zlib or none.

Great. If a new parameter is introduced, it’s best to add it at the end because many people rely on positional parameters, and adding it elsewhere could break compatibility.

Great. If a new parameter is introduced, it’s best to add it at the end because many people rely on positional parameters, and adding it elsewhere could break compatibility.

Sounds good, I have moved it to the end of the parameters.


How to handle exception raised in rescue_from block?

The purpose of a rescue_from block is of course to handle exceptions in a custom way. However, what should happen if the rescue_from block itself raises an exception? Currently, Grape does not handle such exceptions, likely resulting in an Internal Server Error returned by the web server.
In some cases, it would be beneficial if an exception raised inside a rescue_from block was handled by another rescue_from block. However, it’s unclear whether that’s a good idea in general, as it could result in an infinite rescue_from block loop.
I have a use case where Grape::Exceptions::ValidationErrors should be remapped to a different exception that is a subclass of Grape::Exceptions::Base. What I currently do is the following:
rescue_from Grape::Exceptions::ValidationErrors do |e|
  begin
    raise Api::Exceptions::InvalidValueError, e.full_messages
  rescue Api::Exceptions::InvalidValueError => invalid_value_error
    run_rescue_handler(:error_response, invalid_value_error)
  end
end

However, it no longer works with newer Grape versions (probably since #2377).
For this use case, it would be best if it was possible to raise an exception in the rescue_from block that is then handled by the default rescue handler for Grape exceptions. If that’s an option, I can try to come up with sensible semantics that allow that while avoiding infinite loops.
Otherwise, I’d welcome suggestions for handling this use case cleanly and such that it works with current Grape versions. The best I’ve come up with is to call #error!, however it can be a bit fiddly to convert the exception to the correct arguments for #error!. Note that this requires a recent fix (#2471).

If you throw :error the code will handle the case but not a raise. It's quite easy to handle it.

Does #2483 is solving it ?

If you throw :error the code will handle the case but not a raise. It's quite easy to handle it.

Putting throw :error, invalid_value_error in the rescue block in the above example doesn’t work with or without #2483. It causes the middleware to return an Invalid Response error. The line error_response(response) is not executed in this case because #error? returns false for non-Hashes.

Does #2483 is solving it ?

In case of an exception in the rescue_from block, with your code, it passes the exception to #default_rescue_handler which always returns HTTP error code 500. If it called #error_response for subclasses of Grape::Exceptions::Base, it would work.

Looking at #2483 I think we need more tests, @manueljacob care to add one to that PR that demonstrates what you describe?


[Feature] Exercise dashboard button in exercise details pages

Checklist

[ ] I tested all changes and all related features with different users (student, tutor, editor, instructor, admin) on the test server https://artemistest.ase.in.tum.de.
[x] Client: I followed the coding and design guidelines.
[x] Client: I added authorities to all new routes and check the course groups for displaying navigation elements (links, buttons)
[x] Client: I documented the TypeScript code using JSDoc style.
[x] Client: I added multiple screenshots/screencasts of my UI changes
[x] Client: I translated all newly inserted strings into English and German.
[x] Language: I followed the guidelines for inclusive, diversity-sensitive, and appreciative language.

Motivation and Context

In the exercise details pages I added a button to directly access the exercise-assessment dashboard.
Furthermore I cleaned up the button layout a bit. There is still some room for further improvements.
Description

Steps for Testing

Navigate to the course oveview -> exercises
Click on an exercise of every type (directly on the name), and check out the exercise detail page (Text, Modeling, Programming, File-Upload)
Test every button on those pages
Now, in an exam with a programming exercise, check the exercise details page too.

Screenshots

File-Upload-exercise details page:

Programming exercise details page:

Modeling & text exercise details page: (they are identical)

Tested on TS2, with artemis_test_user_11.
Everything works fine except the following:
Programming exercise:
Manage Hints doesn't seem to do anything

Are you sure? For me it works. Which exercise did you use, specifically?

Newly created programming exercises doesn't have a dashboard button

This is intended, as the button is only shown when the manual assessment for the exercise is enabled. This is option comes available as soon as you set a due date.

Some old programming exercises also doesn't have a dashboard button, even though they have multiple graded submissions

Again, probably manual assessment is not enabled.

Recreate Build Plans does not seem to trigger a rebuild of Template Result (reset the timer)

I have to look into that, I'm not sure in this case tbh.

there is a programming exercise with no Update Structure Test Oracle button (KotlinExercise)

This is intended, the button is only shown for Java exercises.

Tested on TS2, with artemis_test_user_11.
Everything works fine except the following:
Programming exercise:
Manage Hints doesn't seem to do anything

Are you sure? For me it works. Which exercise did you use, specifically?
All of the exercises I tested was like that, but it could be because artemis_test_user_11 was not granted Instructor permission, which I did not realize until later (artemis_test_user_11 is noted as Instructor in the test credentials page)

Recreate Build Plans does not seem to trigger a rebuild of Template Result (reset the timer)

I have to look into that, I'm not sure in this case tbh.
Perhaps the problem lies with the triggering account's permission too.

tested on ts2 works as expected. code seem good in general, there are only minor client-style issues need to be fixed.... 👍


cmd/go: test coverpkg=./... not ignoring folders with test only files

Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (go version)?
go1.9.2 linux/amd64
Does this issue reproduce with the latest release?
yes
What operating system and processor architecture are you using (go env)?
not revelent
What did you do?
If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.
go test -coverpkg ./... mypkg/anotherpkg
Where mypkg contains folders containing test only files
What did you expect to see?
PASS
coverage: 100% of statements...
What did you see instead?
go build mypkg/examples: no non-test Go files in ~/go/src/mypkg/examples
FAIL	mypkg/anotherpkg [build failed]

Fixed in Go 1.10 because -coverpkg now takes a pattern filtering the list of packages already going into the test binary, not a pattern generating a list of packages.


Improve test utils for running request

Motivation
To land the support for async/await, we need test utilities. We already have a MockRequestExecutor. Let's improve this to better handle on and off request processing.
Changes

Move MockRequestExecutor into its own file
Add blocking APIs to MockRequestExecutor when called from another thread

Nightly fails because of Sendable requirements.


ER: documentation about UDFs

The AQL documentation on UDFs states (rather ominously):

User functions should be kept purely functional and thus free of side effects and state, and state modification.

Does this mean: "purely functional with respect to the input arguments" or "purely functional with respect to the input arguments and the state of the database when the transaction begins"?
If the former, what happens if, for example, the UDF accesses and returns a value that does depend on the database?  Consider for example, a UDF which returns a value as follows:
return db._query('RETURN LENGTH(' + collection + ')').toArray()[0];

where collection is the string-name of an existing collection.

Hi,
you should rather use https://docs.arangodb.com/3.2/AQL/Functions/Miscellaneous.html#collectioncount to fill that count into a parameter to your UDF.
You should see your UDF like being stateless for each document it sees.

Some example what not to do in a UDF:

Create or modify global variables
Modify arguments passed to the function (if they are passed "by reference")
Define anything that survives the scope of the UDF (like creating properties on the function object itself)

In a nutshell: in your UDF code, define everything you need inside the function and only modify what you defined yourself. Do not touch anything beyond the scope of the function.

Thanks for the response. Please understand the example was just that - an example of a UDF that does examine the state of the database, and that under certain circumstances at least, dies produce rhe expected/“correct” result. The request therefore is for enhanced documentation regarding the precise semantics of such functions.

Fetching the document count of a collection could be seen like accessing a global variable I guess. UDFs are not transactions and should not access the database at all in my understanding.

@Simran-B - The documentation is quite clear about when collections are locked, so it should follow that if a UDF accesses “locked” values, the values should be as expected (and as observed when contention is not an issue).
Simply saying the behavior is “undefined” is highly unsatisfactory - it largely contradicts the existing documentation, and more importantly greatly diminishes UDFs and therefore also AQL (in the absence of a much larger library of pre-defined AQL functions).

meanwhile the documentation has been updated.
ArangoDB 3.7 is available for Download; closing this as solved.
please see https://www.arangodb.com/2020/08/arangodb-3-7-a-big-step-forward-for-multi-model/ for details.


Scheduler Odd behavior

I have been testing SwiftJobs for the last few weeks on GCP and noticed a funny behavior with the onMinutes scheduler. I think this is true for all the on functions because the implement the same API.
If an instance gets behind, the job will never get executed until the permitted schedule is removed from the database and the service is restarted. Most folks will not see this issue if there is at least one Job with the every interval scheduled (i.e .everyMinute(), .everyHour e.t.c)
2024-10-21T22:01:08-0400 debug SwiftJobService : JobName=StatsJob JobTime=2024-10-18 21:25:00 +0000 [Jobs] Next Scheduled Job

value in the Postgres Database
_jobScheduleLastDate:<PHONE_NUMBER>39323235

After deleting the persisted schedule and restarted the container, the following is printed
2024-10-21T22:16:03-0400 debug SwiftJobService : JobName= StatsJob JobTime=2024-10-22 02:17:00 +0000 [Jobs] Next Scheduled Job

The schedule is as follows:
.init(
    job: StatsJob(),
   schedule: .onMinutes([0, 5, 10, 15, 20, 25, 30, 35, 40, 45, 50, 55])
)

I was expecting the next scheduled to be 2024-10-22 02:17:05 +0000 since the job is supposed to run every 5 minutes.
I think popping the first value in the queue is what's causing last issue from this list. I am trying to debug what's causing the job to never catch up.

If an instance gets behind, the job will never get executed until the permitted schedule is removed from the database and the service is restarted.

Im not totally sure what you mean here. Is it if the jobs are not being cleared fast enough eventually it stops working? And what do you mean by the "schedule is removed"?

After deleting the persisted schedule and restarted the container

Did you delete the database? The last scheduled date is stored in the database.

If an instance gets behind, the job will never get executed until the permitted schedule is removed from the database and the service is restarted.

Im not totally sure what you mean here. Is it if the jobs are not being cleared fast enough eventually it stops working? And what do you mean by the "schedule is removed"?
I described "schedule is removed" below. I deleted the last scheduled entry in the Postgres table. The "StatsJob" itself takes about 100ms to complete from start to finish.

After deleting the persisted schedule and restarted the container

Did you delete the database? The last scheduled date is stored in the database.

Yes, I had to delete the last scheduled entry in the database for the job to start running again. The last scheduled date never got updated even though it was in the past.

I was expecting the next scheduled to be 2024-10-22 02:17:05 +0000 since the job is supposed to run every 5 minutes.

Do you not mean `2024-10-22 02:20:00 +0000 given it is every 5 minutes and not 5 seconds?
Can you try and replicate this with #36 and see if the output gives us a better idea what is going on.

I was expecting the next scheduled to be 2024-10-22 02:17:05 +0000 since the job is supposed to run every 5 minutes.

Do you not mean `2024-10-22 02:20:00 +0000 given it is every 5 minutes and not 5 seconds?

Yes, 2024-10-22 02:20:00 +0000 is the correct time. Thanks for pointing that out.

Can you try and replicate this with #36 and see if the output gives us a better idea what is going on.
I will try later and will report back. Thank You!

Worked out what this is. Should be fixed by #38

Worked out what this is. Should be fixed by #38

I apologize I have been active as of late. I have been trying to figure out the best way to reproduce this bug. It looks like you just opened a PR to fix it.

Fixed in beta5


Ensure that opengraph images are available for each docs page

Current Behavior
See this example of a missing image - https://twitter.com/layer5/status/1782612959094624636

Desired Behavior

A general and generic branded Layer5 image - same as that used on layer5.io.
A specific image where appropriate (if at all) for specific docs pages.

Implementation
Reuse opengraph image from layer5.io site.

Contributor Guide

🎨 Wireframes and designs for Layer5 site in Figma (open invite)

Fixed


kexi: 3.1.0 -> 3.2.0

Motivation for this change
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

Does it build for you?
I get:
CMake Error at CMakeLists.txt:159 (find_package):
  Could not find a configuration file for package "KDb" that is compatible
  with requested version "3.1.91".

  The following configuration files were considered but not accepted:

    /nix/store/c1k1ajs9r720wn0r7gd2v0ynlp7zcfbw-kdb-3.1.0/lib/cmake/KDb3/KDbConfig.cmake, version: 3.1.0

Still failing, even after the other upgrade :P
  Could not find a configuration file for package "KReport" that is
  compatible with requested version "3.1.91".

  The following configuration files were considered but not accepted:

    /nix/store/jaz8ph7hl24v7gsl8gvdqnavkzram7h1-kreport-3.1.0/lib/cmake/KReport3/KReportConfig.cmake, version: 3.1.0

CMake Error at CMakeLists.txt:179 (find_package):
  Could not find a configuration file for package "KPropertyWidgets" that is
  compatible with requested version "3.1.91".

  The following configuration files were considered but not accepted:

    /nix/store/j0l85n15kpj3dgdq9l6jpw5bm6zc60qi-kproperty-3.1.0/lib/cmake/KPropertyWidgets3/KPropertyWidgetsConfig.cmake, version: 3.1.0

D'oh, sorry i tested mostly with the entire upgrade stack and split for PR's.
From the posted log, looks like also needs #60047 and #60048 ?

Yeah, looks like it.
Hmm, if they're all that interdependent would upgrading these packages break other packages?

Hmm, don't think so re:breakage-- quick grep suggests kreport and kproperty are only used in calligra (long since broken) and this package, kexi.  So maybe they should be all upgraded together, yes! :).

@GrahamcOfBorg kexi

nix-review says it's fine now :)
Builds fine but can't find the breeze-icons.
I think #54525 would fix that.

Agreed, runs fine here.  Thanks for testing!


Add details on deployments with authentik

Add the solution reported by user Terminalfoo on Discord to the docs.

thanks for pointing this out.


Possibility to use with angular2 DI Framework?

Hi,
I just wanted to know, if there is a way to use TypenJSON with Angular2 DI Framework?
I need to pass a few Services in the constructor of my Objects.
Regards
Benny

You can take a look and try also my library jackson-js, which has a lot of decorators and features to easily manipulate JSON, including class transformation as this one. Class transformation is only a little part of what my library can do! As the name implies, it is heavily inspired by the famous Java FasterXML/jackson library and implements almost all of the Java Jackson annotation!
An example with Angular 9 can be found in this example repository. The example repository gives a simple example using the jackson-js library with Angular 9 for the client side and two examples for the server side: one using Node.js + Express + SQLite3 (with Sequelize 5) and another one using Node.js + LoopBack 4.


error TS7016 with "noImplicitAny": true Enabled in tsconfig.json

error TS7016: Could not find a declaration file for module 'fancy-ansi'. 'node_modules/fancy-ansi/dist/main.cjs' implicitly has an 'any' type.
Try npm i --save-dev @types/fancy-ansi if it exists or add a new declaration (.d.ts) file containing declare module 'fancy-ansi';
import { FancyAnsi } from 'fancy-ansi';

Error persists even with a declaration file like it is done here: https://stackoverflow.com/questions/41292559/could-not-find-a-declaration-file-for-module-module-name-path-to-module-nam

What does your tsconfig.json file look like? It might be an issue with the "moduleResolution" setting. Here are two working examples with vite and nextjs that import declaration files correctly:
https://github.com/kubetail-org/fancy-ansi/tree/main/examples/vite-react
https://github.com/kubetail-org/fancy-ansi/tree/main/examples/nextjs


⚠ ClamAV has degraded performance

In 96b32c7, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 66 ms

Resolved: ClamAV performance has improved in bf4a2c8.


Item properties panel height is too small in new modern UI

Target SharePoint environment
SharePoint Online
What SharePoint development model, framework, SDK or API is this about?
💥 SharePoint Framework
Developer environment
Windows
What browser(s) / client(s) have you tested

[ ] 💥 Internet Explorer
[X] 💥 Microsoft Edge
[X] 💥 Google Chrome
[ ] 💥 FireFox
[ ] 💥 Safari
[ ] mobile (iOS/iPadOS)
[ ] mobile (Android)
[ ] not applicable
[ ] other (enter in the "Additional environment details" area below)

Additional environment details

Edge 128.0.2739.79 (Official build) (64-bit), Chrome 129.0.6668.59 (Official Build) (64-bit)
SPFx 1.16.1
Node.js 14.20.1

Describe the bug / error
On sites with deployed SPFx app (containing ListView Command Set extension), the out of the box properties panel used for properties editation became practically unusable, because it`s height is too small for normal usage. Screenshot:

We have noticed this behavior today for the first time. On the sites / lists where no SPFx extensions are used, the panel behaves correctly for the lists with new modern UI.
Steps to reproduce

Go to the list, on which some SPFx ListView Command Set is executed
Click on New button (or Title of some existing item)
The properties panel for the new (or existing) item is opened on the right side.
The height of the panel has only about 90px, what makes it unusable.

Expected behavior
The properties panel for the new (or existing) item that is opened on the right side should have normal height (as it used to have), to allow normal work with the metadata.

The same behaviour was observed on the client tenant after the See All click, in the right side list filter. The specific filter field panel, that is open, is too small:

For now, actual workaround is an injection of the following css:
/* Prevent SharePoint Online issue with too small OOTB panels */
.od-panel-md-listform,
.od-Panel--autoScroll {
  height: 100%;
}


Can't use okhttp combine with otto?

My Gradle
compile 'com.squareup.okhttp:okhttp:2.6.0'
compile 'com.squareup:otto:1.3.8'
public final class BusProvider {
  private static final Bus BUS = new Bus(); <--------Error on this line

  public static Bus getInstance() {
    return BUS;
  }

  private BusProvider() {
    // No instances.
  }
}

java.lang.NoClassDefFoundError: com.squareup.otto.ThreadEnforcer

These libraries do not interact. Try a clean build or disable/update your ProGuard config as it might be erroneously stripping classes.
As this is a usage question and not a bug or feature request, StackOverflow is a better place for these types of general questions.


[UI Configuration] Allow for feature icon to be configured to another octicon

This PR adds a configuration option in Flipper::UI::Configuration called feature_icon to allow for use of any of the other ocitcons, and defaults it to the good ol' classic squirrel.

@the-robear thanks for taking the time. Could you explain why you are making this change? Do you use a different octicon? If so, what is it?

Yep! I will be using the lightning bolt, called zap in the octicon set. Also considering using the lightbulb.

Ok. Why change it? I know it seems small, but I will need to support it going forward and I'm struggling a bit with why it is even worth changing it or what sparked the desire. Are you migrating from something else where you used a zap or light bulb or just want to tinker and change it?

Closing to keep things clean here, but I'd still love to know the why and may still re-open/merge. Thanks!


Improve ballerina Math API

Purpose

Support Java Math functions in ballerina Math API

Goals

Include ballerina native functions to do numerical operations

Approach

Used Java Math API for implementation.

Automation tests

Unit tests
Line converage - 100%

The @since version number should be 0.94

Added the math API sample and fixed the doc issue.


Issue Templates

Proposal
It would be great to have fancy issue templates for reporting...

Feature requests
Bug reports
Documentation enhancements

These templates could be implemented on a global level in the .github location
Reference any relevant documentation, other materials or issues/pull requests that can be used for inspiration.
No response

Published


Scaffolding: remove component

When we deprecate don;t forget to update the Hall Of Fame page which I think is home.xhtml

HA I already put it there a while ago with a link to the OutputPanel PF demo!  Hall of Fame is all updated.


How to extend monolog levels in monolog 3

Monolog version 3
Hi, We were using Monolog 2 and after upgrading to monolog 3, our app is not working anymore.
In monolog2 we have this piece of code in a custom Logger child class to extend monolog levels to add an 'AUDIT' level , for tracability, we need to keep these logs for a long time.
class Logger extends MonologLogger implements LoggerInterface
{
    public static function addLegalLevel()
    {
        self::$levels[LoggerInterface::AUDIT] = 'AUDIT';
    }
}

Now in monolog 3, levels are defined with an ENUM, which cannot be extended.
And the \Monolog\Logger class is referencing this ENUM.
What's the new way to extend monolog levels ?
(we Use Symfony too.)

Oh, that's a bad news, thanks for the quick response.
Monolog a quite a standard and used is all php libraries.
What's the motivation to remove the possiblity to extend it (I just saw a commit to mark Logger as final)
In Symfony, the DI can help us to use the new Logger service, but duplicate the Logger logic is not very interesting

IMO custom levels was always a bad idea, because many handlers are built to handle the specific levels monolog ships with. And many log storage backends are also only using the standard 8 levels.
So yes as @stof mentioned using a custom channel so you can store these logs in a special place probably makes most sense. Or tag the logs somehow using a special context/extra data that lets you identify them.


[plugin.api.websocket][error] [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)

The same problem as https://github.com/niess/python-appimage/issues/24 ?

This is not an AppImage packaging issue. Streamlink issues do not belong here. Also no debug log at all, which is useless.

What do you expect anyone to do here if you don't even post anything useful at all, like the exact appimage you were using and the exact command you were running which led to the problem?
The error message you've posted is coming from Streamlink's websocket module, and the issue is a TLS/SSL certificate issue. Streamlink's AppImages bundle a Python install which is built from the official Python manylinux2014 docker images, and all the latest dependencies are bundled as well, including certifi for the necessary root certificates.
The issue you've linked is unrelated.

The exception itself is self-explaining. Python (somehow) failed to get local issuer certificate. It's a CA issue, which does not exists in host environment at all.

Appimage prepared all the dependencies, but the local ca list could not be retrived. Why the issue is unrelated?

It seems the issue is actually unrelated, but the behavior is strange. I'll investigate it later to find out what has happened. Sorry for disturbing.

Streamlink doesn't set any custom paths for the cert param of any session initialization or specific request made by requests, which means it defaults to the value which certifi.where() returns. certify.where() returns its locally bundled cacert.pem file. This should work even in cases of AppImages, where the squashfs gets mounted to a temporary location before execution, because certify uses relative file paths.
requests however allows you to override this file via the REQUESTS_CA_BUNDLE or CURL_CA_BUNDLE env vars.

I tried to add $SSL_CERT_FILE manually with:
export SSL_CERT_FILE='path-to-cacert.pem'

And everything works now.

Let me have a look. I was talking about requests, but websocket-client apparently doesn't use the bundled CA file.

So there are two options to avoid having Streamlink use two different CA cert files:

Set OpenSSL's SSL_CERT_FILE env var with a value pointing towards the cacert.pam file bundled by certifi in the AppImage's AppRun and only make changes to the AppImage. Optionally also only set it if it's unset on the user's system, so custom values can still be set.
Update Streamlink's WebsocketClient and set the ca_cert parameter on the SSL options dict, with a default value of the cacert.pam file bundled by certifi, similar to all HTTP requests made by requests.
https://github.com/streamlink/streamlink/commit/4c36d19d0b2173e4db186009177793b04c4de1b2

Since Streamlink shouldn't be using two different implementations regardless whether an AppImage is used or not, the second solution sounds like a better idea.


2587 Added CRUD REST API Smoke Test

This work is done when there is a new test in TestReportStream.kt,
that uses the Settings API to do the following:

Do a 'get' to confirm that no organization exists called 'dummy'.
CREATE: Do a 'set' to submit an Organization called 'dummy'
READ: Do a 'get' to confirm that 'dummy' now exists, and that it has correct fields.
UPDATE: Do a 'set' to submit a modification of the 'dummy' organization, with some minor tweak
Do another 'get' to confirm you get back the new/modified version of 'dummy'
DELETE: Do a 'delete' to remove 'dummy'
Re-run step 1, to confirm that it's gone.
That covers full 'CRUD' for Organizations.

Run ./prime test --list to make sure your new test has a good readable but brief description.
Run ./primt test --run CrudRestApi
Output:
Test passed: CRUD REST API
All tests passed
This PR ...
If you are suggesting a fix for a currently exploitable issue, please disclose the issue to the prime-reportstream team directly outside of GitHub instead of filing a PR, so we may immediately patch the affected systems before a disclosure. See SECURITY.md/Reporting a Vulnerability for more information.
Test Steps:

Include steps to test these changes

Changes

Include a comprehensive list of changes in this PR

Checklist
Testing

[ ] Tested locally?
[ ] Ran ./prime test or ./gradlew testSmoke against local Docker ReportStream container?
[ ] Added tests?

Process

[ ] Are there licensing issues with any new dependencies introduced?
[ ] Includes a summary of what a code reviewer should test/verify?
[ ] Updated the release notes?
[ ] Database changes are submitted as a separate PR?
[ ] DevOps team has been notified if PR requires ops support?

Fixes

#issue - List GitHub issues this PR fixes

To Be Done
Create GitHub issues to track the work remaining, if any

#issue

Specific Security-related subjects a reviewer should pay specific attention to

Does this PR introduce new endpoints?

new endpoint A
new endpoint B

Does this PR include changes in authentication and/or authorization of existing endpoints?
Does this change introduce new dependencies that need vetting?
Does this change require changes to our infrastructure?
Does logging contain sensitive data?
Does this PR include or remove any sensitive information itself?

If you answered 'yes' to any of the questions above, conduct a detailed Review that addresses at least:

What are the potential security threats and mitigations? Please list the STRIDE threats and how they are mitigated

Spoofing (faking authenticity)

Threat T, which could be achieved by A, is mitigated by M

Tampering (influence or sabotage the integrity of information, data, or system)
Repudiation (the ability to dispute the origin or originator of an action)
Information disclosure (data made available to entities who should not have it)
Denial of service (make a resource unavailable)
Elevation of Privilege (reduce restrictions that apply or gain privileges one should not have)

Have you ensured logging does not contain sensitive data?
Have you received any additional approvals needed for this change?

Closing this as its been superceded by #2776


Listening to all the events on a window does not work as expected

When listening to all the events on a window, the callback value is just a <string> instead of an <Events.Window.Any>.
const window = sdl.video.createWindow()

window.on('*', (event) => console.log(event))

This should be fixed in the latest version. I had the type definition wrong: when you listen to all events the first argument is the event type, and the second is the event. Thy this code instead: window.on('*', console.log).

Cool, thx!


Update Lab 1.2 with Network CLI

@willtome - Update Lab 1.2 with Network CLI

fixed with https://github.com/network-automation/linklight/commit/80e0663fbb320b5bba88918e20ce8c0c62324072
please re-open if you continue to see the issue


Callout: Component fixes

Describe the problem
Acceptance criteria:

line height is incorrect in development on heading when stack to second line. should be coming from heading medium token
(--goa-typography-heading-m)
(--got-line-height-4)

Additional context
No response

:tada: This issue has been resolved in version 1.17.0-alpha.66 :tada:
The release is available on:

npm package (@alpha dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This issue has been resolved in version 1.21.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This issue has been resolved in version 4.21.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This issue has been resolved in version 4.17.0-alpha.31 :tada:
The release is available on:

npm package (@alpha dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This issue has been resolved in version 3.0.0-alpha.4 :tada:
The release is available on:

npm package (@alpha dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This issue has been resolved in version 3.0.3 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Too many bugs, can not run any code with digit sensor

More detailed documentation and tutorial is needed

Hi @DavidYaonanZhu,
Could you be a bit more specific about what is not working for you and what bugs are you encountering?
Have you already tried to follow the tutorials here: https://facebookresearch.github.io/PyTouch/docs/tutorials/intro
We would be happy to assist and improve the documentation wherever needed.

Thank you for replying, I have been trying to run surface_3d in example folder
First thing is that where can I get data set? It requires path to data set

Hi @DavidYaonanZhu,
Are you using DIGIT sensor hardware or are you looking to run examples with a dataset you've previously collected?
The surface_3d can use a dataset collected from physical DIGIT hardware, or datasets generated by TACTO
Are you specifically looking for the dataset files? I'll provide an update to the documentation on how to collect your own dataset.

@AlphaBetaPhi Hi, I am trying to test pytouch with DIGIT sensor.
I have already assembled and tested basics functions of DIGIT.
I have not collect any dataset yet, it will be much better to have a tutorial about how to collect data.

I have also tried to run the contact_area code.
I saved some frames from DIGIT sensor and set base_img_path and sample_img_path as:
    base_img_path = "/home/ur/Downloads/digit/digit-interface/data/digit_1.png"     sample_img_path = "/home/ur/Downloads/digi
However, there is an error saying:
File "/home/ur/anaconda3/envs/plot/lib/python3.9/site-packages/pytouch/tasks/contact_area.py", line 107, in _compute_contact_area
return poly, major_axis, major_axis_end, minor_axis, minor_axis_end
UnboundLocalError: local variable 'poly' referenced before assignment

For the touch_detect.py
the error is:
cannot find model touchdetect_resnet18_DigitSensor.pth

I guess I might have missed some process

For the touch_detect.py the error is: cannot find model touchdetect_resnet18_DigitSensor.pth

The example contains various methods of initializing tasks,
Initialize defaults specified by the task,
    pt = pytouch.PyTouch(DigitSensor, tasks=[TouchDetect])
    is_touching, certainty = pt.TouchDetect(source.image)

Initialize with a custom configuration, including model,
 touch_detect = TouchDetect(DigitSensor, zoo_model="touchdetect_resnet18")

This can be specified to any model for touch detection providing the model has been submitted to the model zoo,  touch_detect = TouchDetect(DigitSensor, zoo_model="touchdetect_my_model_name_here
Example:
import pytouch
from pytouch.handlers import ImageHandler
from pytouch.sensors import DigitSensor
from pytouch.tasks import TouchDetect

def touch_detect():
    source = ImageHandler("/path/to/image")

    # initialize with task defaults
    pt = pytouch.PyTouch(DigitSensor, tasks=[TouchDetect])
    is_touching, certainty = pt.TouchDetect(source.image)
    print(f"Is touching? {is_touching}, {certainty}")

if __name__ == "__main__":
    touch_detect()

I have also tried to run the contact_area code. I saved some frames from DIGIT sensor and set base_img_path and sample_img_path as:  base_img_path = "/home/ur/Downloads/digit/digit-interface/data/digit_1.png" sample_img_path = "/home/ur/Downloads/digi However, there is an error saying:
File "/home/ur/anaconda3/envs/plot/lib/python3.9/site-packages/pytouch/tasks/contact_area.py", line 107, in _compute_contact_area return poly, major_axis, major_axis_end, minor_axis, minor_axis_end UnboundLocalError: local variable 'poly' referenced before assignment

Can you provide the code?

This is basically the code, I just modified some part of digit_demo.py from digit-interface
rom digit_interface.digit import Digit
from digit_interface.digit_handler import DigitHandler

digit = Digit("D00046", "Left Gripper")
digit.connect()

for rgb in rgb_list:
    digit.set_intensity_rgb(*rgb)
    time.sleep(1)
digit.set_intensity(15)

for i in range(10):
    digit.save_frame(path="./digit_%d.png"%i)
    time.sleep(1)

digit.disconnect()

I commented out resnet18 part as suggested.
I can now run touch_detect.py, but the result is always 1, even without touching anything.

Now I have solved the error, by referencing here
But, now the detection is always 1 even without touching the sensor, I guess it is the model issue?

In addition, I am trying to run contact_area.py, the error says:
File "/home/ur/anaconda3/envs/plot/lib/python3.9/site-packages/pytouch/tasks/contact_area.py", line 107, in _compute_contact_area
return poly, major_axis, major_axis_end, minor_axis, minor_axis_end
UnboundLocalError: local variable 'poly' referenced before assignment

Now I will try with surface_3d.py

@AlphaBetaPhi Hi, I am trying to test pytouch with DIGIT sensor (hardware). I have already assembled and tested basics functions of DIGIT. I have not collect any dataset yet, it will be much better to have a tutorial about how to collect data.

There is a dataset available through,
wget https://www.dropbox.com/s/19jmxxoobe3ixpl/local.zip -P $PKG_DIR

unzip $PKG_DIR/local.zip -d $PKG_DIR
rm -r $PKG_DIR/local.zip

The surface_3d example expects a dataset with the following structure:
root
-- 0000
---- image1
---- image2
-- 0001
-- abcd

Running with the above dataset the following is produced,

Can you provide a few examples of your DIGIT outputs you are using?

Yes, the output of DIGIT is images

I mistakenly closed

For surface_3d.py:
The error I get is:
File "/home/ur/Downloads/digit/PyTouch/examples/surface_3d.py", line 103, in visualize_surface_3d
output = surface3d.point_cloud_3d(img_color=img)
File "/home/ur/anaconda3/envs/plot/lib/python3.9/site-packages/pytouch/tasks/surface_3d.py", line 115, in point_cloud_3d
normal_pred = self.normals(img_color)
File "/home/ur/anaconda3/envs/plot/lib/python3.9/site-packages/pytouch/tasks/surface_3d.py", line 62, in normals
img_normal_pred = self.model.color_to_normal(img_input).to("cpu")
File "/home/ur/anaconda3/envs/plot/lib/python3.9/site-packages/pytouch/models/pix2pix/pix2pix.py", line 93, in color_to_normal
img = self.preprocess_image(img_input)
File "/home/ur/anaconda3/envs/plot/lib/python3.9/site-packages/pytouch/models/pix2pix/pix2pix.py", line 120, in preprocess_image
img = Image.fromarray(img)
File "/home/ur/anaconda3/envs/plot/lib/python3.9/site-packages/PIL/Image.py", line 2815, in fromarray
raise TypeError("Cannot handle this data type: %s, %s" % typekey) from e
TypeError: Cannot handle this data type: (1, 1, 320), |u1
Seems to be related to pix2pix.py

Now I have solved the error, by referencing here
But, now the detection is always 1 even without touching the sensor, I guess it is the model issue?

I also always get 1, with certainty always greater that 98-98%.
Do we need to train model for new sensors?
I am trying to use DIGITs
My code is almost the same as [DavidYaonanZhu](https://github.com/facebookresearch/PyTouch/issues/22#issuecomment-1064932017)

Yes, the classification model, namely, "touchdetect_resnet_DigitSensor.pth" was trained for that particular sensor by PyTouch developers.
I guess you can fine-tune this pre-trained model with RGB images from your sensor, or train ResNet classification from scratch. It shows always above 90% for our sensors too.
On that note, pix2pix model, which is used for surface3d task, needs to be trained from scratch too.  Both provided models are sensor specific and factors such as RGB lighting, gel properties differ for each sensor.

Yes, the classification model, namely, "touchdetect_resnet_DigitSensor.pth" was trained for that particular sensor by PyTouch developers. I guess you can fine-tune this pre-trained model with RGB images from your sensor, or train ResNet classification from scratch. It shows always above 90% for our sensors too. On that note, pix2pix model, which is used for surface3d task, needs to be trained from scratch too. Both provided models are sensor specific and factors such as RGB lighting, gel properties differ for each sensor.

Do you know how to train or tune the model with the provided code?

I am not sure how to do that with the provided code but you can refer to this repo for training pix2pix, CycleGan, MLP model for RGB to Surface3d mapping.
https://github.com/psodhi/tactile-in-hand/tree/main/inhandpy/scripts/img_translation
Again,  it's not straight forward and you may have to do some tweaking for dataset creation for training.


Shortcut Guide: Add PowerToys shortcuts to the guide

Description of the new feature / enhancement
As stated in the title, it would be very useful if the PowerToys shotcuts were shown in the Shotcut Guide.
The ideal scenario would be to implement a toggle for these shortcuts to be added in the guide.
Scenario when this would be used?
Some PowerToys shotcuts are used less often than others, thus being harder to remember in a pinch. Having them shown in the guide would help a lot in assuring them to be used more often.
Supporting information
No response

/dup https://github.com/microsoft/PowerToys/issues/15405


Cannot run tests of empty project with plain karma -> integration with JetBrains's IntelliJ is broken

Bug Report or Feature Request (mark with an x)
- [ x] bug report -> please search issues before submitting
- [ ] feature request

Area
- [ x] devkit
- [ ] schematics

Versions
~$ node --version
v8.11.1
~$ npm --version
5.6.0
~$
Repro steps

Create an empty project using ng new myproject
Open the project in Intellij (2018.1.2 Ultimate)
Launch the unit tests using Karma plugin (181.4668.7)

The log given by the failure
/usr/bin/node /home/user/.IntelliJIdea2018.1/config/plugins/js-karma/js_reporter/karma-intellij/lib/intellijServer.js --karmaPackageDir=/home/user/sandbox/angular/myproject/node_modules/karma --configFile=/home/user/sandbox/angular/myproject/src/karma.conf.js
/home/user/sandbox/angular/myproject/node_modules/@angular-devkit/build-angular/src/angular-cli-files/plugins/packages/angular_devkit/build_angular/src/angular-cli-files/plugins/karma.ts:53
throw new Error(The '@angular-devkit/build-angular/plugins/karma' karma plugin is meant to +
^
Error: The '@angular-devkit/build-angular/plugins/karma' karma plugin is meant to be used from within Angular CLI and will not work correctly outside of it.
at init (/home/user/sandbox/angular/myproject/node_modules/@angular-devkit/build-angular/src/angular-cli-files/plugins/packages/angular_devkit/build_angular/src/angular-cli-files/plugins/karma.ts:53:11)
Desired functionality
karma start --config=karma.conf.js should just work.
Mention any other details that might be useful

see https://github.com/angular/angular-cli/issues/10703
@AlexGS74 presents a working solution

It works if you create a new run configuration. It will detect Cli and use it as Karma engine. If you run an old configuration still pointing to a global/local karma instance, it fails.

This issue was moved to angular/angular-cli#12108


Add diagram about A/B testing

I found this diagram in the architecture Google Drive:
https://docs.google.com/drawings/d/1rx4brKZBzj-9q3evkiUw2MbqwqTYWkc0Lku6u3cLXqU/edit
I'm not sure who created it ("Deleted user", perhaps @carvil?). It looks accurate so maybe it'll help someone.
I got the URL by doing File > Publish to the web.... I think it auto-updates.

@tijmenb 👋 I don't recall having done that diagram, only a few on paper :) must have been someone else. Great to see one, though :)

Thanks @suzannehamilton. Updated the diagram & made it so everybody in GDS can edit it. I didn't have to update the link, changes are auto-published.


creating speak streaming client and api

Summary by CodeRabbit

New Features

Introduced a default callback handler for live transcription and text-to-speech API.
Added various response types and constants for text-to-speech streaming.
Implemented a new MessageRouter for handling different message types.

Enhancements

Updated SpeakOptions to include Callback and CallbackMethod.
Improved error handling with new error constants for connection and validation issues.

Examples

Added an interactive text-to-speech example using the Deepgram SDK.

These changes enhance the text-to-speech functionality, provide better error handling, and introduce examples for easier implementation.

This is ready to go, but we need to merge this PR https://github.com/deepgram/deepgram-go-sdk/pull/245 and cut a release before merging this one.

Hold this for right now, but I think we are in the clear.


Add editor config

Issue #, if available:
Closes #35
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

are there any warnings in the build , Mark ? If not, feel free to check in once others are fine

are there any warnings in the build , Mark ? If not, feel free to check in once others are fine

There were, but i resolved before committing, so we should be fine.


Eliminate RSpec/MessageSpies offenses in RuboCop

Working on the TODO. This change fixes all offenses of RSpec/MessageSpies in RuboCop itself.

Before submitting the PR make sure the following are checked:

[X] Wrote good commit messages.
[ ] Commit message starts with [Fix #issue-number] (if the related issue exists).
[X] Feature branch is up-to-date with master (if not - rebase it).
[X] Squashed related commits together.
[ ] Added tests.
[ ] Added an entry to the Changelog if the new code introduces user-observable changes. See changelog entry format.
[X] The PR relates to only one subject with a clear title
and description in grammatically correct, complete sentences.
[X] Run bundle exec rake default. It executes all tests and RuboCop for itself, and generates the documentation.

👍


Fix Is null predicate get error query result

What problem does this PR solve?
cherry pick #41668

run buildall

run buildall


styling

add promo code input
add quantity in beer/wine description page after login, not working
styling login or signup page in checkout before login
styling sort
add "gear" logo in navbar to admin

LGTM


On Dynamic content, Resize is causing issues

I have a page, where on a click height increases of the page, but on another link with reduction in data the size should be reduced, its not reducing dynamically.
Kindly assist.
With page content being reduced resizer is not reducing the content.

This will be an issue with you pages CSS, you can try different height
calculation methods, if that doesn't work you will have to investigate your
CSS
On Mon, Sep 21, 2020 at 2:52 PM AnkurMadaan<EMAIL_ADDRESS>wrote:

I have a page, where on a click height increases of the page, but on
another link with reduction in data the size should be reduced, its not
reducing dynamically.
Kindly assist.
With page content being reduced resizer is not reducing the content.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/davidjbradshaw/iframe-resizer/issues/859, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAEQOEUJL6UNPLZBBTBZIG3SG5LBPANCNFSM4RUR5V4A
.


Fix miss this. on PC

Sergio,
This is a tiny little fix, and I'm a poor JS programmer, but this did fix an issue for me.
let me know if I need to tweak this or otherwise.
Thank you for writing this. (and for WHIP!)

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Getting error while trying to start training: "Unrecognized arguments: ros_nerfacto"

Hi!

Cloned the project
Initialized "Nerfstudio" submodule
Installed all of Nerfstudio's dependencies
Executed "pip install -e ." inside the nerfstudio directory
Run: "python ros_train.py ros_nerfacto --data nsros_config_sample.json"
Getting parsing error:

It seems that the ros_nerfacto method is not recognized.
Does anybody know the fix for this?
Much appreciated!

Executed "pip install -e ." inside the nerfstudio directory

Is this a typo? If you didn't pip install -e . in the nerfbridge directory then the method will not register with nerfstudio.
Also, it appears you're using the ROS1 interface. Just FYI I will be archiving that branch and dropping ROS1 support.

@javieryu Thank you for the quick response. Oh I see now... I completely forgot to check out the other ROS2 branch.
Ended up being confused why things didnt run the way I wanted to with Nerfstudio. Seems I was working with the ROS1
interface!
Never mind me, I will close the issue.


Reformat City, Company name, Job Title to have the first letter of every word to be uppercase

@elhmn are this a duplication of #88 ?

Yes it looks like it 🤦♂, closing now


aya: remove Rc<RefCell> usage in BPF program links.

This is a breaking change which transfers all ownership of a link to an
eBPF program to the caller return value of the "attach" functions. This
eliminates the overhead of Rc<RefCell>, as well as prevents the
links section of ProgramData from growing without bound as programs are
attached and detached.

In retrospect, this could actually be a drop-in change if OwnedLink is named LinkRef. My concern is that it's a little misleading of a name. On the other hand, I don't think that users will typically import and directly refer to LinkRef / OwnedLink in the first place, so perhaps this is a moot point.

This is a breaking change which transfers all ownership of a link to an eBPF program to the caller return value of the "attach" functions. This eliminates the overhead of Rc,

I'm wondering is this hypothetical overhead or is it something you were able to measure? Based on this and other PRs I'm assuming you're using aya in some kind of embedded system.
I'm not against this change, but it's not something that I'd do lightly since it does make the API harder to use, and it makes it easier to misuse it (programs can get detached silently because you failed to store or propagate a link up the stack).
Also if we do go this route, I think we should make the returned link type opaque. The way programs get attached - and so the type of link used - can change between kernel versions, and will change more in the future as more programs adopt BPF_LINK_CREATE. At some point in the next 6 months I think we should release 1.0, at which point we won't be able to break API easily.

as well as prevents the links section of ProgramData from growing without bound as programs are attached and detached.

This is something that should be fixed.

I'm wondering is this hypothetical overhead or is it something you were able to measure? Based on this and other PRs I'm assuming you're using aya in some kind of embedded system.

I did not measure; the overhead introduced by the Rc<RefCell> should be very minor in an "attach-once-and-forget" use case. We are indeed using aya in an embedded system, and we regularly have to attach or detach XDP programs from network interfaces, so every memory allocation - and to a lesser extent, every pointer indirection - counts.

I'm not against this change, but it's not something that I'd do lightly since it does make the API harder to use, and it makes it easier to misuse it (programs can get detached silently because you failed to store or propagate a link up the stack).

I can empathize that this is a more contentious change. In my eyes, the fact that dropping the returned LinkRef still leaves a resource occupied (ie: the program is still attached) is not idiomatic Rust behavior, especially given that there is no way to detach it from that point on until the Bpf is dropped.

Also if we do go this route, I think we should make the returned link type opaque. The way programs get attached - and so the type of link used - can change between kernel versions, and will change more in the future as more programs adopt BPF_LINK_CREATE. At some point in the next 6 months I think we should release 1.0, at which point we won't be able to break API easily.

Totally understandable and I agree; it should be trivial to add that on to this PR with a wrapper struct around the OwnedLink enum.

as well as prevents the links section of ProgramData from growing without bound as programs are attached and detached.

I think that this would have to become a HashMap keyed on some unique specifier so that links with the same inputs are always slotted to the same bucket, but perhaps this is an XDP-centric perspective. Or of course get rid of it entirely as this PR does. :)

I'm wondering is this hypothetical overhead or is it something you were able to measure? Based on this and other PRs I'm assuming you're using aya in some kind of embedded system.

I did not measure; the overhead introduced by the Rc should be very minor in an "attach-once-and-forget" use case. We are indeed using aya in an embedded system, and we regularly have to attach or detach XDP programs from network interfaces, so every memory allocation - and to a lesser extent, every pointer indirection - counts.

I'm not against this change, but it's not something that I'd do lightly since it does make the API harder to use, and it makes it easier to misuse it (programs can get detached silently because you failed to store or propagate a link up the stack).

I can empathize that this is a more contentious change. In my eyes, the fact that dropping the returned LinkRef still leaves a resource occupied (ie: the program is still attached) is not idiomatic Rust behavior, especially given that there is no way to detach it from that point on until the Bpf is dropped. It resembles a leak.

I would tend to agree with this as well. But I think we should very carefully document the potential repercussions of dropping a link unexpectedly if we end up going this route.

Also if we do go this route, I think we should make the returned link type opaque. The way programs get attached - and so the type of link used - can change between kernel versions, and will change more in the future as more programs adopt BPF_LINK_CREATE. At some point in the next 6 months I think we should release 1.0, at which point we won't be able to break API easily.

Totally understandable and I agree; it should be trivial to add that on to this PR with a wrapper struct around the OwnedLink enum.

as well as prevents the links section of ProgramData from growing without bound as programs are attached and detached.

I think that this would have to become a HashMap keyed on some unique specifier so that links with the same inputs are always slotted to the same bucket, but perhaps this is an XDP-centric perspective. Or of course get rid of it entirely as this PR does. :)

This is actually how it's done in libbpf-rs IIRC (they are actually keyed by the program name in the libbpf-rs skeleton, but of course that assumes that the mapping of programs to links is one-to-one, which is definitely not necessarily the case. We may need to put some thought into what exactly the keys of this map would contain.). I think this would be a totally valid approach.

I've updated the PR with implementation hiding and a longer documentation note.

I'm not against this change, but it's not something that I'd do lightly since it does make the API harder to use, and it makes it easier to misuse it (programs can get detached silently because you failed to store or propagate a link up the stack).

I can empathize that this is a more contentious change. In my eyes, the fact that dropping the returned LinkRef still leaves a resource occupied (ie: the program is still attached) is not idiomatic Rust behavior, especially given that there is no way to detach it from that point on until the Bpf is dropped. It resembles a leak.

I would tend to agree with this as well. But I think we should very carefully document the potential repercussions of dropping a link unexpectedly if we end up going this route.

I disagree that it's not idiomatic rust. I think it's conceptually not very different form say tokio::spawn (or std::thread::spawn): you can use the returned handle if you want, but if you don't, the task is still managed by the runtime.
I could have used other examples but the reason I chose tokio::spawn is that I think it makes the same trade off that attach() currently makes: if one had to keep alive all the task handles returned by spawn() the API would objectively become a lot less ergonomic (and incidentally, futures have this problem in rust and it's one of the things that make them a bit painful to use).
The way I see it, the whole point of having the Bpf object is to manage resources. In fact a first iteration of aya didn't have the Bpf object at all; I wanted the API to be as flexible as possible, so you'd use the obj::* API to parse ELF, then manually create maps and programs and call relocation functions. I started using that API, realised it was extremely tedious and decided that convenience was more important than flexibility.
For example consider the following code with the current API:
fn init_some_feature(bpf: &mut Bpf) -> Result<(), anyhow::Error> {
    let program: &mut TracePoint = bpf.program_mut("sys_enter_connect")?.try_into()?;
    program.load()?;
    program.attach("syscalls", "sys_enter_connect")?;

    let program: &mut TracePoint = bpf.program_mut("sys_enter_accept")?.try_into()?;
    program.load()?;
    program.attach("syscalls", "sys_enter_accept")?;
    program.attach("syscalls", "sys_enter_accept4")?;

    // [snip - I'm attaching to a bunch more places here]

   Ok(())
}

fn init_some_other_feature(bpf: &mut Bpf) -> Result<(), anyhow::Error> {
    for sym in ["sym1", "sym2", "sym3"] {
        let program: &mut UProbe = bpf.program_mut(&sym)?.try_into()?;
        program.load()?;
        program.attach(Some(sym), 0, "libXXX", None)?;

        if let Ok(program) = bpf.program_mut(&format!("{}_ret", sym)) {
            let program: &mut UProbe = program.try_into()?;
            program.load()?;
            program.attach(Some(sym), 0, "libXXX", None)?;
        }
    }

    Ok(())
}

After having called Bpf::load() the calling code can treat init_some_feature() and init_some_other_feature() pretty much as black boxes. Those two functions can do whatever BPF operations they need to do and be essentially self contained.
// ... call Bpf::load()
init_some_feature(&mut bpf)?;
init_some_other_feature(&mut bpf)?;

The code above is taken from a real program that includes about ~10 such "features" that can be turned on or off depending on configuration settings.
Now if we merge this PR, the code becomes:
fn init_some_feature(bpf: &mut Bpf) -> Result<Vec<OwnedLink>, anyhow::Error> {
    let mut links = Vec::new();

    let program: &mut TracePoint = bpf.program_mut("sys_enter_connect")?.try_into()?;
    program.load()?;
    links.push(program.attach("syscalls", "sys_enter_connect")?);

    let program: &mut TracePoint = bpf.program_mut("sys_enter_accept")?.try_into()?;
    program.load()?;
    links.push(program.attach("syscalls", "sys_enter_accept")?);
    links.push(program.attach("syscalls", "sys_enter_accept4")?);

    // [snip - many more times]

   Ok(links)
}

fn init_some_other_feature(bpf: &mut Bpf) -> Result<Vec<OwnedLink>, anyhow::Error> {
    let mut links = Vec::new();

    for sym in ["sym1", "sym2", "sym3"] {
        let program: &mut UProbe = bpf.program_mut(&sym)?.try_into()?;
        program.load()?;
        links.push(program.attach(Some(sym), 0, "libXXX", None)?);

        if let Ok(program) = bpf.program_mut(&format!("{}_ret", sym)) {
            let program: &mut UProbe = program.try_into()?;
            program.load()?;
            links.push(program.attach(Some(sym), 0, "libXXX", None)?);
        }
    }

    Ok(links)
}

And the caller:
// ... call Bpf::load()
let mut links = init_some_feature(&mut bpf)?;
links.append(init_some_other_feature(&mut bpf)?));

// now the caller needs to keep both the `bpf` instance and  `links` around

Now the init_* functions leak this weird link stuff I'm supposed to always keep in a vec (like - I literally never want them to be dropped). This seems a strictly worse UX to me. We're pushing some tedious bookkeeping  work we're now doing internally back to the user, and I'm not sure what the benefit is?

I disagree that it's not idiomatic rust. I think it's conceptually not very different form say tokio::spawn (or std::thread::spawn): you can use the returned handle if you want, but if you don't, the task is still managed by the runtime.

Understandable. I have never been fond of these APIs for this exact reason, but it is a measured trade-off in terms of UX.

Now the init_* functions leak this weird link stuff I'm supposed to always keep in a vec (like - I literally never want them to be dropped). This seems a strictly worse UX to me. We're pushing some tedious bookkeeping work we're now doing internally back to the user, and I'm not sure what the benefit is?

For the use case of "attach-once-and-leave it for the entire program's / Bpf's lifetime", there is indeed minimal benefit. When one has to frequently attach/detach, or has a fixed-size use case (we can get away with an array of MaybeUninit or Option), then we don't have to deal with as much dynamic memory allocation or pointer indirection under-the-hood.
A concrete example, if a little raw:
`
enum Event
{
AttachLink(usize),
DetachLink(usize),
}
fn event_stream() -> impl Stream<Item = Event + Unpin>
{
unimplemented!()
}
async fn event_loop(mut bpf: Bpf)
{
let links = [None, None];
let programs = [ "xdp0", "xdp1" ];
let interfaces = [ "eth0", "eth1" ];
while let Some(ev) = stream.next().await
{
    match ev
    {
        Event::AttachLink(index) =>
        {
            let xdp: &mut Xdp = unsafe {
                bpf.program_mut(programs[index])
                    .unwrap_unchecked()
                    .try_into()
                    .unwrap_unchecked()
            };
            let link = match xdp.attach(interfaces[index], XdpFlags::SkbMode)
            {
                Ok(l) => l,
                Err(_) => continue,
            };
            links[index].replace(link);
        },
        Event::DetachLink(index) =>
        {
            links[index].take();
        },
    }
}

}
`
This change puts the onus on the caller to decide what trade-off to make, but at least allows the caller to make that trade-off rather than deciding for them. "The Rust way" has always come across to me as opting in to any sort of additional runtime costs for convenience.
Having said all of that, you're the maintainer here, and we can continue to maintain our own fork of aya for this specific change.
There is still a real problem with the links vector growing without bound in the current implementation, but I don't have a definitive answer for how to uniquely key on a link in such a way that it covers all potential use cases.

Now the init_* functions leak this weird link stuff I'm supposed to always keep in a vec (like - I literally never want them to be dropped). This seems a strictly worse UX to me. We're pushing some tedious bookkeeping work we're now doing internally back to the user, and I'm not sure what the benefit is?

For the use case of "attach-once-and-leave it for the entire program's / Bpf's lifetime", there is indeed minimal benefit. When one has to frequently attach/detach, or has a fixed-size use case (we can get away with an array of MaybeUninit or Option), then we don't have to deal with as much dynamic memory allocation or pointer indirection under-the-hood.

Even for cases where one has to frequently attach and detach, I'd argue that not having to keep links around is preferable. Attach and detach do syscalls, so those will trump whatever performance cost the tiny heap allocation might have. Unless you have very limited heap space - in which case fair enough this change might make a difference - I doubt you'd see those link Rcs in any perf profile.

This change puts the onus on the caller to decide what trade-off to make, but at least allows the caller to make that trade-off rather than deciding for them. "The Rust way" has always come across to me as opting in to any sort of additional runtime costs for convenience.

If this was in a performance sensitive area I would agree with you, and it's for example why the PerfEventArray API looks the way it does. It could be made a lot easier to use, but only at the expense of performance. Since it is an API that is often going to be in a hot path, it makes sense to prioritise performance over ease of use.

Having said all of that, you're the maintainer here, and we can continue to maintain our own fork of aya for this specific change.

I think that might be the best course of action. That said if you do measure and see that the allocation does have a significant performance cost on your system, I'd certainly be open to changing my mind.

There is still a real problem with the links vector growing without bound in the current implementation, but I don't have a definitive answer for how to uniquely key on a link in such a way that it covers all potential use cases.

Yep this is definitely something we need to fix. Not sure exactly how either, will require some thinking.

This morning I had a shower thought: to properly clean up ProgramData::links on link.detach(), links will probably have to include some kind of cookie that uniquely identifies them. If we change the API from link.detach() to program.detach(link) then we can remove the Rcs, make the links Copy and do nothing on Drop

I like this; we'll probably stick to our approach on our end, but this would probably be good on the upstream side. Something like:
pub type LinkId = usize;

pub(crate) struct ProgramData {
    pub(crate) obj: obj::Program,
    pub(crate) fd: Option<RawFd>,
    pub(crate) links: HashMap<LinkId, OwnedLink>,
    link_ind: LinkId,
    pub(crate) expected_attach_type: Option<bpf_attach_type>,
    pub(crate) attach_btf_obj_fd: Option<u32>,
    pub(crate) attach_btf_id: Option<u32>,
}

impl ProgramData {
    pub fn attach<T: Into<OwnedLink>>(&mut self, link: T) -> LinkId {
        let id = self.link_id;
        self.links.insert(id, link.into());
        self.link_id += 1;
        id
    }

    pub fn detach(&mut self, id: LinkId) {
        self.links.remove(id);
    }

I like this; we'll probably stick to our approach on our end, but this would probably be good on the upstream side. Something like:

Yep something like that

It does feel silly to me for each Program / ProgramData to track its own links, since we're ultimately trying to tie them to the Bpf's lifetime, right? Perhaps we can migrate this map there...

It does feel silly to me for each Program / ProgramData to track its own links, since we're ultimately trying to tie them to the Bpf's lifetime, right? Perhaps we can migrate this map there...

It seems logical to me? Programs are owned by Bpf, so transitively the links are too. And it also provides a kind of namespacing which will simplify things once we add support for unloading programs, as we'll be able to iterate ProgramData::links and detach all of a program's links before unloading.
Also keep in mind that while everything happens inside of Bpf at the moment, it probably won't always be like that. For example see https://github.com/aya-rs/aya/issues/2. There are eBPF APIs that we don't yet (fully) support that allow working with maps and programs loaded by other user space programs. It's not clear yet whether those APIs will go through Bpf or not.

I've pushed a new commit which includes this change if you want to play around with it. I think that my initial enthusiasm is substantially dulled.
The caller has to now keep track of which program a given LinkRef belongs to, which makes manual detaching a lot more painful with many programs to manage. But we do retain the advantage of keeping the API for the simplest, likely most common, use case trivial.

I've pushed a new commit which includes this change if you want to play around with it.

Thank you! I'll play a bit with it today

The caller has to now keep track of which program a given LinkRef belongs to, which makes manual detaching a lot more painful with many programs to manage. But we do retain the advantage of keeping the API for the simplest, likely most common, use case trivial.

I see, in which case doing bpf.detach(link) would make sense. I guess it wouldn't be hard to make LinkRef work for both bpf.detach(link) and program.detach(link) tho? (LinkRef would have to store the program name too)

If we go that route, I think that it ends up being more expensive than the original Rc<RefCell<dyn Link>> implementation due to having to clone Strings for each LinkRef.

This change breaks lirc. With lirc, you want to be able to load a BPF program, and then the program must be able to exit without detaching the program.
This is the whole point of BPF IR decoding: you load the BPF program once during boot, and then the BPF program keeps on decoding. There is no user-space program running.
This is the problem:
impl Drop for OwnedLink {
    fn drop(&mut self) {
        let _ = self.detach();
    }
}

Please remove it.

This change breaks lirc. With lirc, you want to be able to load a BPF program, and then the program must be able to exit without detaching the program.

Many other programs work that way too, say you want an userspace program to launch, setup some eBPF programs then exit leaving the programs attached. It's a very common scenario.
I think that needs to be explicitly handled tho, see https://github.com/aya-rs/aya/issues/51

I am not following. For lirc, this currently work in aya. I think this PR breaks that. You're saying that is ok?

I am not following. For lirc, this currently work in aya. I think this PR breaks that. You're saying that is ok?

I'm saying if this ever goes in it's probably fine as it is, but before it can go in in order not to break lirc (and enable other people to exit userspace leaving eBPF programs attached), we should implement #51.

I'm saying if this ever goes in it's probably fine as it is, but before it can go in in order not to break lirc (and enable other people to exit userspace leaving eBPF programs attached), we should implement #51.

And I'd also require that we add some documentation to the book that explains eBPF program lifecycle in Aya and what your options are for managing it :wink:

@seanyoung It is a breaking change, but I think that detach on drop is the better default behavior and what most users would expect. In order to intentionally leak a value, one can always forget(link). You're right that this needs to be documented in more detail.
Looking at lirc and similar programs specifically, I think that their Drop implementation should be changed to detach, and that we should implement https://github.com/aya-rs/aya/issues/51.

If we go that route, I think that it ends up being more expensive than the original Rc<RefCell<dyn Link>> implementation due to having to clone Strings for each LinkRef.

We could probably store programs by their name hash, then include the hash in the link ref?

Makes sense.
This is probably superfluous, but if a link is created via bpf_prog_query() then the default should not be to detach on drop.

@seanyoung I don't think that aya currently has a data type for links from queries, but I would definitely qualify it as a separate data type from the OwnedLink.

I've drafted an example forget implementation building off the concept of OwnedLink at https://github.com/aya-rs/aya/pull/124. It should be possible to apply this to the LinkRef approach, but I don't think that I want to invest the energy in figuring that out; the API seems worse to me than OwnedLink.
In light of this, I think that there is currently a footgun: certain classes of eBPF programs (anything but LIRC2) are detached when dropping or terminating the program, while others are merely forgotten unless explicitly detached (LIRC2). I think that the behavior should be to consistently detach on drop unless explicitly forgotten (my preference), or vice-versa.

@seanyoung I don't think that aya currently has a data type for links from queries, but I would definitely qualify it as a separate data type from the OwnedLink.

LircLink can be used exactly for this, see: https://github.com/aya-rs/aya/blob/main/aya/src/programs/lirc_mode2.rs#L83-L104

Closing as replaced by #249


Light source and receiver functionality

Light sources and receivers can be placed in a level and the receiver will "charge up" with light


add tracking allocator to xutils

this is a little different from how Eigen implements this feature: they add an assert(true) if allocation occurs when the tracking is enabled (ie. the program is crashed).
Also, this obviously only works if no custom allocator is used.

fixes #658


fix the overlapped leftside nav entries

see the wrapped lines in the red rectangular for example

@thockin @RichieEscarez @bgrant0607

Definitely need to fix this. I'll let @thockin review and merge, though, since I'm not at all familiar with the formatting.

Hi Chao, per our conversation, ill add this to my PR which will include a fix for the sub menu as well as all the work ive done to the nav menu. As discussed, ive added these changes to my branch and will close this.

Richie - is your PR about ready for testing?

@thockin, yes, just opened #11566


Fix macOS and iOS builds on older versions of Xcode

I'm not sure this is the right fix, yet, but it gets the build working for BabylonNative on older versions of Xcode.

Better way of fixing this would be to define all those for version where they don't exist, so that code is just using logical name instead hardcoded value.

Like this?
enum GPUFamily {
	Apple4 = 1004,
	Apple5 = 1005,
	Apple6 = 1006,
	Apple7 = 1007,
	Apple8 = 1008
};

...

if ([m_device respondsToSelector: @selector(supportsFamily:)])
{
	if ([m_device supportsFamily: MTLGPUFamily(Apple4)])
	{
		g_caps.vendorId = BGFX_PCI_ID_APPLE;

		if ([m_device supportsFamily: MTLGPUFamily(Apple8)])
		{
			g_caps.deviceId = 1008;
		}
		else if ([m_device supportsFamily: MTLGPUFamily(Apple7)])
		{
			g_caps.deviceId = 1007;
		}
		else if ([m_device supportsFamily: MTLGPUFamily(Apple6)])
		{
			g_caps.deviceId = 1006;
		}
		else if ([m_device supportsFamily: MTLGPUFamily(Apple5)])
		{
			g_caps.deviceId = 1005;
		}
		else
		{
			g_caps.deviceId = 1004;
		}
	}
}

Like this?

Curiously this was how it was done before the commit that broke the build
https://github.com/bkaradzic/bgfx/commit/c43c4479de1ba4290b8e9d4c4df1ed4b760e84dc

@bkaradzic Are you thinking this should be done like this comment suggests?
https://github.com/bkaradzic/bgfx/issues/3067#issuecomment-1519101402

On SDK version where MTLGPUFamilyAppleX doesn't exist, define those in header. Where they exist don't do anything.

On SDK version where MTLGPUFamilyAppleX doesn't exist, define those in header. Where they exist don't do anything.

Thanks. Please take a look at the recent commits and let me know if anything else should be changed.

I'm also having this issue. Will checkout this PR to test and report back!

Yes! SilverNode works with this patch! I'm on macOS 12.

Yes! SilverNode works with this patch! I'm on macOS 12.

Cool, thanks for testing! Just waiting on @bkaradzic to merge it in for us.

@docEdub I'm waiting for this change: https://github.com/bkaradzic/bgfx/pull/3116#issuecomment-1650227521

Define those missing defines in header file on platforms that don't have it. And then leave code that testing against defines as is.

I would do this, but I have no ability to test on older versions. So you have to do it, make sure it works.

@docEdub I'm waiting for this change: #3116 (comment)

As far as I can tell, the PR as it stands, does exactly that. @bkaradzic

Build still seems to fail when new xcode is used to target older macos:
https://github.com/belegdol/mame/actions/runs/5989591652/job/16245941482

The failing config has XCode 14 on macOS 12.

https://github.com/bkaradzic/bgfx/commit/5f564db0d5338c7204efc060ba82b41d80f3682a  This commit use MTLBinding  instead of MTLArgument.
https://developer.apple.com/documentation/metal/MTLArgument?language=objc  MTLArgument is available on iOS 8.0–16.0
https://developer.apple.com/documentation/metal/mtlbinding?changes=_5 MTLBinding is available on iOS 16+
https://developer.apple.com/documentation/metal/mtlbinding/3929840-isused?changes=_5  MTLBinding.isUsed is not well documented. Its behaviour is strange on my iPhone (iOS 16.6/iPhone 12 mini), it returns true only when I launch app from the XCode , and if I launch app outside of XCode, it always returns false.
I doubt it is not the equivalent to the MTLArgument.isActive (https://developer.apple.com/documentation/metal/mtlargument/1461891-active?language=objc)

This got integrated in #3284. Can be closed.

This got integrated in #3284. Can be closed.

Thank you!