Execute code in a sandboxed environment

Still a long way to go but opening this early for getting feedback. The more eyes look at it the better. My approach has been to run the user-provided code in a thread pool that is sandboxed and doesn't allow them to access dangerous APIs (System.exit) or change the VM's settings. This is achieved with a inheritable thread local boolean flag that controls which threads are sandboxed which i currently can break and have included a test for fixing.

[x] Evaluate which permissions the sandbox must have

Currently no permissions are granted to the sandbox, we can build a safelist as we go.

- [ ] Implement the sandbox described in this paper
References:

Permissions in the JDK
Evaluating the flexibility of the Java Sandbox
JDK 8 security enhancements
Self-protecting sandbox using SecurityManager
A per-thread Java Security Manager

Amazing work, looking forward to have this enabled so we can finally bring some sanity to the evaluation process.

@dialelo What is the status of this PR? Anything we can do to help push this forward? @franciscodr may be able to help here too if you need him to. Thanks!

@raulraja I took a look at this, but TBH I was struggling to get much past what @dialelo had already done

I've tried several approaches to sandboxing code, all of which have been unsuccessfull. I struggled because things don't work as documented, and the examples in the articles i consulted simply didn't work.

Using a custom security manager: you can replace the JVM's security manager, and intercept every permission check to implement your own policy.

Tried sandboxing only certain threads and it worked, but the sandbox can easily be escaped by replacing the security manager from a privileged thread in, for instance, a global thread pool like scala.concurrent....implicits.global
Tried discerning if the permission check was for code loaded at runtime after compilation, but i couldn't distinguish between the stack frames of our code in the application and the code we load and code. Permissions across a stack seem to be computed as the intersection of all the permissions in the code that is in such stack.

Using a custom classloader

Tried setting a custom protection domain, which in theory can have certain permissions attached, to load the code. Didn't work as documented. Even when using a custom domain with no permission i was able to do anything from the loaded code.
Tried combining a custom classloader + Policy approach, but both the stack frames of the loaded code and of the application end up having the custom classloader. This surprised me because the custom classloader used for compilation is not a descendant of the system classloader.

I've also tried using a custom Policy, and combinations of the approaches outlined above, with no success. By the end of this i was thinking whether it wouldn't be easier to move the code execution to a separate, non-privileged JVM, and comunicate with it through messaging.

The concern with moving evaluation to it's own JVM it's bootstraping of dependencies + start + shutdown time and potentially a decrease of concurrent evaluations given the JVM will reserve RAM that may be never be used. If say there is 10 concurrent evaluations and each process takes 500m ram we can quickly exhaust the Heroku dyno allocated mem.

@dialelo @noelmarkham can this be closed now?

I think so. I'll leave the final decision up to @dialelo, considering the final solution was a workaround rather than "doing it properly," but I don't think I have any more energy to give this task :-)

Let's close this for the moment and revisit in the future if we need to.

I'll keep the branch around just in case.


Escaped spaces in applet path

Applet path in ‘binary’ variable caused path not found error when
options.name included spaces
After I updated to latest I found the new MacOpen method was breaking since my options.name used spaces. Escaping the spaces fixed it. It might not be exhaustive, and I've only dealt with OSX but this little update sorted it for me.
If you see anything that refers to or implies wrapping the path in quotes that's because I changed approach from wrapping in quotes to escaping the spaces.

Thanks @tenKinetic
I think this may be a bug in child_process.exec(). It supports relative paths with spaces, but not absolute paths with spaces.
See: https://github.com/nodejs/node/issues/6803

Yeah ok that could be a better place to address the issue.
I haven't managed to find where a relative path gets treated differently to an absolute one though. Unless it's in libuv which I haven't looked at much but surely it can't be in there without worldwide explosions.
Assuming that using the shell argument would potentially result in a valid path, I tried that but it still produced the same error.
While writing this, I did come across this issue: uv_spawn env+path semantics which alludes to the difference in how absolute and relative paths are treated by libuv. Whilst I'm definitely clutching at straws here, it sounds like it might be possible that both actually fail however if the path is relative then $CWD\filename (which is only tried by libuv for non-absolute paths) does work whereas with an absolute path then that explicit path is the only one tried.
So maybe it is in fact libuv's treatment of absolute vs relative paths that is making us think the relative path works when it's actually by providing cwd that makes the relative path work.
So, since we are setting the cwd option, passing './applet' as the command is working for me. Probably a better workaround within sudo-prompt that I will use for now.

I see you found the precise source of the problem. Closing this which can serve as a possible workaround until an update to node.

Thanks for your help @tenKinetic. I will try and see if I can use the ./ trick for Mac and Linux in the meantime. It should work fine.


PDF Viewer crash

RQ2A.210505.002.2<IP_ADDRESS>
App constantly crashes
Different files attempted

There's no actionable information in this report. You need to provide an example of what crashes and explain what you mean by crash.

If you've disabled the WebView, that's a user error. Apps aren't expected to handle the WebView not being present since it's a guaranteed component of the base system for Android.


fix container init

Container initialization is broken because of schema file rename in #5db5de7
+
Container initialization is broken because of aiida removal in #bf11bfb

@knopki AiiDA was just temporarily removed due to very cumbersome installation, generally, we’re still relying on it in the long term

Currently the best option would be not to account for AiiDA but still check if it’s available (and be able to install it, if the user requires its workflows).

@knopki AiiDA was just temporarily removed due to very cumbersome installation, generally, we’re still relying on it in the long term

When AiiDA returns, we will return the scripts to install it. Right now the installation is broken anyway (although it can be fixed).

Currently the best option would be not to account for AiiDA but still check if it’s available (and be able to install it, if the user requires its workflows).

I don't think it's any better. It adds to the complexity.


Consider using the name kytt instead of ytt?

Since the name of almost all tools in k14s starts with 'k', do we ever consider using the name kytt instead of ytt? kytt is also easy to pronounce comparing to the acronym ytt?

Since the name of almost all tools in k14s starts with 'k'

that's mostly an artifact of those tools being specific to kubernetes. ytt, however, is a generic tool (doesnt even know what k8s is), and so it doesnt have a k in front.

do we ever consider using the name kytt instead of ytt

possibly, though probably not in the short term.

closing for now.


please add the ability to select and then copy text.

it's absolutely necessary for me to be able to translate the whole chapter.

Hmmm good idea, will try in future releases


Search parent packages for error handler processes

When a runtime error is encountered, currently the search for package-level handlers includes only the package of the process instance.  However, if a handler is not found here, we should continue to traverse the parent package hierarchy to locate potential handlers.
This logic can be incorporated into ProcessEngineDriver.getPackageHandler().

Rather than making a heirarchical structure (as there is no "extends" concept and all processes are visible to every package ) , can we rather have a exception handler "type" package. And every business package can be configured to select one of these packages (among all available exception type package names).

I think in 6.1.09 we can readily extend the existing hierarchical pattern for error handling to search in parent packages.
We can also open a separate issue to cover your more novel approach of having exception handler type packages.  I would like to explore this in more depth.

Feel free to open an issue describing how you'd like exception handler type packages to work.


.rct-header has a bounce effect

Thanks for creating an issue! Please fill out the sections below so that we can accommodate your request:
What kind of issue are you experiencing?

[x] Bug

Actual Behavior:
When I used 0.15.5 the header scrolled along the timeline. Now with version 0.16.1 there is this bounce effect know from apple/safari/iphone, which produces some bad UX.
Steps to reproduce the issue:
limit scroll of timeline to a fixed range, for example from today 0:00 untill tomorrow 0:00 . zoom into timeline and scroll to start or end (I'm using a touchpad on macbook). the timeline events scroll normally, but the header (rct-header) "overshoots" the end and bounces back (see screenshot attached)
Versions:
0.15.5
Additional Information:
Chrome,  66.0.3359.139, OS X

EDIT: Simple Repro to see the effect is here: https://codesandbox.io/s/lx52v97507
I just added a scroll limiting method to onTimeChange

Sure… I can give it a try

Hi @mcMickJuice this is blocking me at the moment. Could you please give me some pointers on how to solve this?

@mcMickJuice Also interested…

Related issue - https://github.com/namespace-ee/react-calendar-timeline/issues/357

@mcMickJuice I havent found any workaround for this yet so I think I am going to tackle it later this week

Hi, I was trying to solve this bug and I found out that the problem in my app was from a timeline plugin I wrote. Sorry, but I am not going to continue with this bug since the problem doesn't affect me anymore. For anyone who is interested in the future, I solved it by applying oveflow: auto on my plugin divs.

@Ilaiwi Interesting, I also had a timeline plugin activated. However even when I disabled all timeline plugins completely the bug still occurs as well as when applying css overflow property as you suggested.

@florianbepunkt in my application the plugin was responsible for some customized drag and drop behavior. This bug appeared for me when I was dragging the item close to the edges so after researching that the solution was for me is to do overflow: auto. This was a very specific case. I also have mouse dragging disabled on the timeline so it is very rare for this bug to appear for me.

I think I can solve this after finishing #349. For now, my new headers bounce because of canvasTimeStart and canvasTimeEnd change where they shouldn't, because of the limit enforced by onTimeChange

this is solved in 0.20.0


Bernoulli distribution for MNIST

Hi, I'm getting an error when running the MNIST example (I'm using torch version 1.12.1).
Traceback (most recent call last):
  File "<input>", line 637, in <module>
    log_prob_p_x_z = p_x_z.log_prob(x)  # e.g. torch.Size([8])
    log_prob = self.base_dist.log_prob(value)
    raise ValueError(
ValueError: Expected value argument (Tensor of shape (512, 784)) to be within the support (Boolean()) of the distribution Bernoulli(probs: torch.Size([512, 784])), but found invalid values:

I think it's because the torch.distributions.Bernoulli expects the MNIST data to be either 0 or 1, instead of in [0,1].
I replaced lines 426-431 in mfcvae.py with the following and the error goes away:
# term 1: compute log p(x|z), the MC estimate of E_{q(z,c|x)}[log p(x|z)] where z~q(z|x)
if self.likelihood_model == 'Bernoulli':
    # p_x_z = D.Independent(D.Bernoulli(probs=torch.clamp(x_hat, min=1e-10, max=1-(1e-10))), 1)
    # log_prob_p_x_z = p_x_z.log_prob(x)  # e.g. torch.Size([8])
    # can't use D.Bernoulli here because x is continuous between [0,1]

    bernoulli_loss = torch.nn.BCELoss(reduction='none')
    log_prob_p_x_z = -torch.sum(bernoulli_loss(torch.clamp(x_hat, min=1e-10, max=1 - (1e-10)), x), axis=1)
elif self.likelihood_model == 'Gaussian':
    p_x_z = D.Independent(D.Normal(loc=x_hat, scale=torch.ones_like(x_hat) * self.sigma_multiplier_p_x_z), 1)
    log_prob_p_x_z = p_x_z.log_prob(x)  # e.g. torch.Size([8])

Sounds good, thank you!! Yes results are still reproducible with this fix!


Functions deploy does not work on Windows

Bug report
Describe the bug
When trying to deploy a function on Windows, it throws the following error message:
PS C:\Users\panam\Development\supabase-test> supabase functions deploy hello-world
Error: exec: "C:\\Users\\panam/.supabase/deno": file does not exist

I checked the directory and the file does actually exist, so it might be an issue with the slashes in the path?
To Reproduce

Follow the Getting Started guide in the docs for Edge functions.
Run supabase functions deploy hello-world.
The process fails.

Expected behavior
The function should be deployed.
System information

OS: Windows 11
Shell: PowerShell
Version of cli: 0.23.6

I've been stuck on this awhile today...
They issued 0.23.7 which got past the deploy ...
but then I hit this:
mkdir ~/.supabase: The system cannot find the path specified
But looks like they reverted 0.23.7 back to the old code and now 0.23.8

good catch @soedirgo is working on a patch now 👍

I guess I'll just wait as it was reverted... I think.  0.23.7 got further, but seemed still broke.

Waiting for scoop to get it the new version...
Do you have to update that too?
Never used scoop before but the guide showed using that to get the cli...

Error: exec: "C:\Users\gabbo\.supabase\deno": file does not exist
No Go.

Error: exec: "C:\\Users\\gabbo\\.supabase\\deno": file does not exist No Go. edited to have the \\ come thru. 0.23.9

Same for me.

Better have it working by hackathon tomorrow or you are going to have to handicap the Linux guys somehow... ;)

😅 I don't have a Windows machine, so this will be tricky... can you try v0.23.10 once it's out?

The hackathon can proceed!

Yep, it's working with v0.23.10. Thanks for the quick fix @soedirgo


BAD_POINTER_COMPARISON   Implicitly checking whether NSNumber pointer value is nil at line.

(BOOL)setValue:(NSNumber *)value forUnit:(NSString *)unit
{
if(value)
{
//some code here
}
}
I am getting the above error in my code. What is the expectation here, Can some one give an example. I am getting lot errors like this. Please suggest.

A common mistake is to write if(value) when intending to test whether value is a non-zero integer, while the code actually tests whether value is nil.  If you instead write if(value != nil) explicitly, then the warning should not be reported.

Thank you @jberdine

Some documentation: http://fbinfer.com/docs/infer-bug-types.html#BAD_POINTER_COMPARISON


Update featured section on homepage

What
Change 3rd feature box link and description.
Why
Changes requested.
Visual Differences
Before

After

Has OK from Jen and Joe to go live.


Support<EMAIL_ADDRESS>

Support<EMAIL_ADDRESS>

is doesnt work on 6 now?

@mafintosh
<EMAIL_ADDRESS>by brew install
system: OS X EI Capitan V10.11.4

could you try again? just bumped some of the dependencies

@mafintosh  ; ) Thanks!! Just like u say.


⚠ Turnos 24x7 has degraded performance

In 0d922c6, Turnos 24x7 (https://reservas-dev.alternativasinteligentes.com/) experienced degraded performance:

HTTP code: 200
Response time: 3428 ms

Resolved: Turnos 24x7 performance has improved in 76a7620.


Replace deprecated methods in ParquetNativeRecordReader

Replace deprecated read footer method in Parquet record reader
Also, provide localFS hadoop configuration to Native reader (already provided to avro record reader)

Codecov Report

Merging #9106 (d21651e) into master (bbec0a8) will increase coverage by 0.06%.
The diff coverage is 0.00%.

@@             Coverage Diff              @@
##             master    #9106      +/-   ##
============================================
+ Coverage     26.40%   26.47%   +0.06%
  Complexity       42       42
============================================
  Files          1826     1826
  Lines         96934    96958      +24
  Branches      14621    14627       +6
============================================
+ Hits          25599    25669      +70
+ Misses        68835    68788      -47
- Partials       2500     2501       +1

Flag
Coverage Δ

integration1
26.47% <0.00%> (+0.06%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...inputformat/parquet/ParquetNativeRecordReader.java
0.00% <0.00%> (ø)

...pinot/plugin/inputformat/parquet/ParquetUtils.java
0.00% <0.00%> (ø)

...pinot/controller/util/CompletionServiceHelper.java
71.42% <0.00%> (-20.00%)
:arrow_down:

...troller/helix/core/retention/RetentionManager.java
26.61% <0.00%> (-17.75%)
:arrow_down:

.../apache/pinot/controller/util/TableSizeReader.java
77.86% <0.00%> (-17.22%)
:arrow_down:

.../org/apache/pinot/common/http/MultiGetRequest.java
85.00% <0.00%> (-15.00%)
:arrow_down:

...ache/pinot/core/operator/docidsets/OrDocIdSet.java
75.00% <0.00%> (-13.64%)
:arrow_down:

...ter/predicate/EqualsPredicateEvaluatorFactory.java
51.11% <0.00%> (-12.23%)
:arrow_down:

...pache/pinot/core/util/SortedRangeIntersection.java
65.21% <0.00%> (-10.15%)
:arrow_down:

...core/operator/dociditerators/NotDocIdIterator.java
90.00% <0.00%> (-10.00%)
:arrow_down:

... and 49 more

Help us with your feedback. Take ten seconds to tell us how you rate us.


Re-enable checkpointing with parameterized functions

This feature works again.
Describe your changes
 Backward/forward compatibility checks

Check these boxes or delete any item (or this section) if not relevant for this PR.

[x] Client+Server: this change is compatible with old servers
[x] Client forward compatibility: this change ensures client can accept data intended for later versions of itself

Note on protobuf: protobuf message changes in one place may have impact to
multiple entities (client, server, worker, database). See points above.

Changelog
Memory snapshotting can now be used with parameterized functions.

@prbot approve


FOLIO-2562 initial repo setup

Need a first PR, to finalise FOLIO-2562.

INFO: lint-raml: Using lint-raml version: 1.4.2
INFO: lint-raml: Investigating and determining configuration: mod-organizations/ramls
INFO: lint-raml: Assessing RAML files (https://dev.folio.org/guides/raml-cop/):
ERROR: lint-raml: No RAML files found in ./ramls
ERROR: lint-raml: There were processing errors. See list above.

Note: When those errors are fixed, then the INFO messages will be gone too. See Jenkins "Artifacts" tab.


Questions in Observation submit

I created a mb-fhir-form to submit blood pressure FHIR observation. Although it works I am not sure how to specify the subject and value unit as described below.
Subject
FHIR example
"subject": {
	"reference": "Patient/{{ _.patientFHIRId }}",
	"display": ""
}

Here is the code to specify the subject reference using mb-context but doesn't seem to have any effect:
 <mb-context path="subject.reference" bind="Patient/1" />
 <mb-context path="subject.reference" bind="Patient/{fhir_id}" />

Quantity
FHIR example
"valueQuantity": {
	"value": 115.3,
	"unit": "mmHg",
	"system": "http: //unitsofmeasure.org",
	"code": "mm[Hg]"
}

Here is how I setup quantity
 <mb-input label="Systolic" path="valueQuantity.value" />

and that maps to this
"valueQuantity": {
	"value": 115.3,
}

How can I set the unit, system and code?

I figured out you can set the unit using mb-context like this
  <mb-context path="valueQuantity.unit" bind="mmHg" />
  <mb-context path="valueQuantity.system" bind="http://unitsofmeasure.org" />
  <mb-context path="valueQuantity.code" bind="mm[Hg]" />

But it seems only work because valueQuantity was referenced in mb-input. The same mechanism wouldn't work for subject.


WIP BibCheck: new ticket creation policies

Adds the --ticket-creation-policy option which can be either
per-rule, per-record, per-rule-per-record or no-tickets. These
control how tickets are dispatched, if at all.  --no-tickets is
preserved for backwards compatibility.
(closes #2383)

Turns errors, amendments and warnings of records into instances of
the new Issue class. This allows for a more coherent way of storing and
extracting this information.

Changes the call to BIBCATALOG_SYSTEM.ticket_submit to pass recordid
only if the ticket concerns a single record.

Fixes a typo in a comment.

@dset0x Can you please add appropriate Signed-off-by and Reviewed-by directives to the commit message?  It seems that at least @jmartinm reviewed your branch, so you could add him.
@jmartinm @kaplun Was this deployed on INSPIRE already? Can you set appropriate milestone?

Nope not yet deployed. I believe there is still some pending discussion as well with @ksachs . To be checked.

I believe there is still some pending discussion as well with @ksachs

Maybe you are referring to #2474 (that I merged already), or was there something else?

I have no idea what you are talking about ...
----- Original Message -----
From: "Tibor Simko"<EMAIL_ADDRESS>To: "inveniosoftware/invenio"<EMAIL_ADDRESS>Cc: "ksachs"<EMAIL_ADDRESS>Sent: Tuesday, 6 January, 2015 3:05:07 PM
Subject: Re: [invenio] WIP BibCheck: new ticket creation policies (#2397)
I believe there is still some pending discussion as well with @ksachs
Maybe you are referring to #2474 (that I merged already), or was there something else?
—
Reply to this email directly or view it on GitHub .

@ksachs This is the patch that allows for submitting tickets in bundles (eg: One ticket in the HoldingPen per check).
@kaplun I can't recall any unaddressed issue, other than that we have not ran this through some usability test.

This is superseeded by #2841


add support for XIM

This provides with a tentative implementation to add support for XIM. The major modifications are made to winit so that the alacritty side is minimal. Temporarily redirect the glutin repo to my branch because we want glutin to use my current version of winit (https://github.com/tomaka/winit/pull/223). This directly addresses issue https://github.com/jwilm/alacritty/issues/44. It would be better to open another PR to change @jwilm own  glutin repo to aggregate the changes.

if you want to update this PR to be against master, it should have the latest winit.

@jwilm Must be drunk at the time of writing last comment (misread your comment), Lol. Working on the PR against master now.


QuantifiedConstraints parse error

Hlint reports a parse error at the line:
deriving instance (forall a. Show (t a)) => Show (E t c)

Despite trying everything I can to tell hlint to enable QuantifiedConstraints.
GHC is fine with it.
It's very possible this has the same cause as #850.

Ok, I was running an older version of hlint than I realised. This appears to have been fixed.

Well actually, there is something odd going on here, but maybe it's an issue with hie.
On version 2.2.10, hlint is able to parse the file when I use the standalone executable, but not when used as part of hie (the parser reports 'Illegal class assertion').
I have checked, and my hie executable was built with hlint-2.2.10.

I'd bring the issue over to hie - if it works standalone then I guess it's something hie is doing? Or maybe they are using the API which is doing the wrong thing?

HIE depended on an old version, so they can fix it.


Add the MCFM qqb/gg->GG ME

A Dbkgkin for gamma gamma channel now possible.

Are we missing some MELA reference files?  Because I don't understand how the tests passed - you modified testME_v2 but not any references.

Yes, we miss JHUGenMCFM Ping. I will add those references later since I want to work a bit more on this new feature.


Add support for preventing opening menu when vertically scrolling

For Participant app fix.

Coverage decreased (-0.5%) to 69.412% when pulling 32814039c471a357f2fd0220f9f02c5a18b40975 on FocusVision:feature/km/vertical-prevent into f60dfd731c7a67e32619dfceb2e9dd715f660a07 on Mango:master.


I believe that this project will participate in the JB financial support program.

This is the application link : https://docs.google.com/forms/d/e/1FAIpQLSfihYy9ot3RPmiaMrEVmE_xgJlGnYg8zXewnF1-r2YXqi3CKg/viewform
https://www.youtube.com/@Kotlin/community

Thank you for making sure I knew about this! I actually applied thanks to people on my Discord server pointing it out when it became available. Otherwise, I would have missed it for sure.
If you use Discord, feel free to join us there! (Link is at the top of the README if you're interested)


Export transformers and destinations as apps

Provide an ability to generate Apps from existing destinations or transformers in the platform.

Updated README at https://github.com/cptn-io/apps with instructions for sharing apps with community


Doctor Appointment App back end PR

In this Capstone Project Milestone we have implemented the below -:
Link to front end PR front end pr

[ ] Set up Rails back end for the application
[ ] Create Models
[ ] Setup controllers and actions
[ ] Created Migrations and schema according to Project requirements.
[ ] Defined routes in routes.rb
[ ] Setup devise and devise-jwt for authentication.
[ ] Deployed back end application on render to expose apis for the front end.
[ ] API and model specs created for back end (Passing).

Please Note; If you want to run rails back end app please comment out line no 33 and line no 40 in puma.rb file. Screen shot attached below for your kind reference.

All suggested changes have been implemented. Sending it for your review again.

Linters errors removed.
All suggested changes have been implemented.


exec not working can't get variables and pulling all key /values

I'm trying to pass on env. variables to  a program. To simplify the issue, I was able to replicate it with the Linux command "echo". This works in my docker ubuntu container:
vaultenv  env = list all variables from the container and from Vault
this also works vaultenv  /bin/echo $LANG (inherited variable)
This returns nothing 'vaultenv  /bin/echo $DB_HOST'.
2nd question:
Is there a way to load all key /values from with default names from VAULT with specifying all in the secret file like:
VERSION 2
MOUNT dev
env#* (all K/V under dev )
Thank you

Hi @raiknet,
Thansk for your interest in vaultenv!
To answer your first question, it seems that the problem here is not so much vaultenv, but your shell. I managed to reproduce your problem and it looks to me like your shell is replacing your environment variable before the call to vaultenv.
Take vaultenv /bin/echo $DB_HOST. The first step would be to do a lookup of the environment variable in the current environment, which results in the call vaultenv /bin/echo ''. After this, no more environment variables exists in you echo call, so none will be printed from the vaultenv environment. To solve this, you could add a separate script, like echo_env:
#!/usr/bin/env bash

/bin/echo $DB_HOST

After this, the script is called using vaultenv ./echo_env. This will evaluate the environment variable in the correct environment, meaning the environment vaultenv created for you.
To answer your second questions, in short, no, there is no way to currently do this. The vault API does allow us to list all keys in some path, which can be used recursively to create a tree of all possible keys, but only individual secrets can be retrieved. The combination of these 2 endpoints would in theory allow us to retrieve all keys or keys with a wildcard, but that is currently not yet possible.
Hope this helps!

Thank you again, it's working now.


fix: only include declarations in docs if the file is ambient

Previously, the declare keyword was taken to indicate a global variable and shown in the documentation. However, that is only be the case for ambient declarations - in modules, this just indicates a private value.
Partially fixes #57

This actually includes all the changes from #59, which I guess it probably shouldn't, but the goal was to fix #57 and those changes are needed to fix it. Maybe I should undo that?

I think I'll leave it to avoid merge conflicts.

@Liamolucko please rebase

@Liamolucko please rebase


Installing artifactory startup issue

After Installing jfrog Not able to brning up service on Ubuntu machine getting below issue.
I am able to check the artifactory .
root@ip-172-31-3-232:/home/ubuntu/artifactory-oss-6.0.1/bin# ./artifactoryctl check
Checking arguments to Artifactory:
ARTIFACTORY_HOME     =  /home/ubuntu/artifactory-oss-6.0.1
ARTIFACTORY_USER     =  artifactory
TOMCAT_HOME          =  /home/ubuntu/artifactory-oss-6.0.1/tomcat
ARTIFACTORY_PID      =  /home/ubuntu/artifactory-oss-6.0.1/run/artifactory.pid
JAVA_HOME            =  /usr/lib/jvm/java-8-oracle/
JAVA_OPTIONS         =  -server -Xms512m -Xmx2g -Xss256k -XX:+UseG1GC -XX:OnOutOfMemoryError="kill -9 %p" -Djruby.compile.invokedynamic=false -Dfile.encoding=UTF8 -Dartdist=zip -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true -Djava.security.egd=file:/dev/./urandom
root@ip-172-31-3-232:/home/ubuntu/artifactory-oss-6.0.1/bin# ./artifactoryctl start
found java executable in JAVA_HOME
Starting Artifactory tomcat as user artifactory...
/home/ubuntu/artifactory-oss-6.0.1/tomcat/bin/startup.sh: 1: /home/ubuntu/artifactory-oss-6.0.1/tomcat/bin/startup.sh: uname: not found
/home/ubuntu/artifactory-oss-6.0.1/tomcat/bin/startup.sh: 1: /home/ubuntu/artifactory-oss-6.0.1/tomcat/bin/startup.sh: dirname: not found
Cannot find /catalina.sh
The file is absent or does not have execute permission
This file is needed to run this program
** ERROR: Artifactory Tomcat server did not start. Please check the logs
But logs have not generated in tomcat logs folder

Can you list all the steps you have followed before reaching this stage

First copied

jfrog-artifactory-oss-6.0.1.zip on to  Ubuntu server.
Unzipped and installed jfrog using Executed below command from artifactory bin location
./artifactory.sh
Then I have set JAVA_Home variable.
Tried to  check ./artifactoryctl check it completed succesfully.
When I tried to start this command  ./artifactoryctl start I am getting below issue .

root@ip-172-31-3-232:/home/ubuntu/artifactory-oss-6.0.1/bin# ./artifactoryctl start
found java executable in JAVA_HOME
Starting Artifactory tomcat as user artifactory...
/home/ubuntu/artifactory-oss-6.0.1/tomcat/bin/startup.sh: 1: /home/ubuntu/artifactory-oss-6.0.1/tomcat/bin/startup.sh: uname: not found
/home/ubuntu/artifactory-oss-6.0.1/tomcat/bin/startup.sh: 1: /home/ubuntu/artifactory-oss-6.0.1/tomcat/bin/startup.sh: dirname: not found
Cannot find /catalina.sh
The file is absent or does not have execute permission.
But files are presenting in that location and it has rewrite executable permission

Issue have been fixed after Resetting java and tomcat home variables in default script

Issue fixed


Drafting calculation of tbtc auction value in percent

WIP.. this PR is addressing the following comment: https://github.com/keep-network/coverage-pools/pull/53#issuecomment-849686389
More prod. code and tests will be added after we agree on the general approach.
We need to calculate the percentage value of tbtc auction in liquidation with signer bonds in exchange for TBTC. Signer bonds are seized for the deposits in liquidation and we cannot use deposit.collateralizationPercentage() because it will return 0. The idea in this PR is to use deposit.auctionValue() instead and read the current market ETH/BTC ratio. Based on the current collateral percentage value we can decide if a coverage pool auction should be opened or not.

Closing without merging in favor of the approach proposed in #75.


ref(cd): simplify deployment pipelines

we've since shipped a few changes on our end that enable these simplifications!

outdated


DetailView::$_id could be usefull

Hi,
unlike GridView implementation, the Widget::$_id is not used as an option when the table is created. I am not nostalgic of CDetailView but I liked this feature.
A small change in code, a great step for testers and consistency :)
Thanks.

Done.


fix: Cleanup single value metadata

Ran Docs Authoring Pack "Cleanup: Single Value metadata"

Docs Build status updates of commit a307bec:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Replaced by #538


Crashes on HASS 2024.5.0b1

I tried to upgrade to the latest beta of HA (2024.5.0b1) and this caused the integration (and all my media center automations) to crash with the following error message:

RuntimeError: Detected that custom integration 'smartthings_soundbar' calls async_write_ha_state from a thread at custom_components/smartthings_soundbar/api.py, line 104: self.async_schedule_update_ha_state(). Please report it to the author of the 'smartthings_soundbar' custom integration.

I've reverted back to 2024.4 for now, and the integration is working again.
I also see this notice at startup, but assume that this is currently a minor issue:

2024-04-28 10:05:28.355 WARNING (MainThread) [homeassistant.helpers.entity] Entity media_player.soundbar (<class 'custom_components.smartthings_soundbar.media_player.SmartThingsSoundbarMediaPlayer'>) is using deprecated supported features values which will be removed in HA Core 2025.1. Instead it should use <MediaPlayerEntityFeature.PAUSE|VOLUME_SET|VOLUME_MUTE|TURN_ON|TURN_OFF|VOLUME_STEP|SELECT_SOURCE|PLAY: 19853>, please create a bug report at https://github.com/PiotrMachowski/Home-Assistant-custom-components-SmartThings-Soundbar/issues and reference https://developers.home-assistant.io/blog/2023/12/28/support-feature-magic-numbers-deprecation

Fixed in v1.0.3

Brilliant, thanks for the quick fix. Will upgrade and test it soon.

Confirmed working on 2024.5.0b4

@aherbjornsen great, thank you!


remove old content matching code and assets

Jira Issue: https://issues.redhat.com/browse/AAP-24568

[!WARNING]
Review this PR carefully; it removes a lot of code and has non-obvious/complex interactions with seat checking:

A seat check was removed from the code that sends the content matching request to WCA, but the check was left in for telemetry
For the WCA case, the codematch function is invoked with a payload that contains the seat attribute, but it's never sent to WCA.

Description

Remove old "attributions" endpoint; content matching is currently being handled by WCA.
Notes

I removed the seat check from the content matching invocation, but not the segment event code. The assumption is that the segment code will treat every content matching telemetry event as if the user has a seat, but pay attention to this case during review

Testing

Steps to test

Pull down the PR
Start the service in local mode and commercial model
Submit completion requests through VS Code

Scenarios tested

Production deployment

[ ] This code change is ready for production on its own
[X] This code change requires the following considerations before going to production:

[ ] Cleanup unused AI_SEARCH env vars in infra files and secrets

@gebhardtr sorry for the conflicts. To resolve them, you can:

Remove skip-string-normalization = true from the pyproject.toml
Run re-commit run --all-files
Commit everything
Squash all your commits in one single change
git fetch --all
Do the git pull --rebase

PR Deployment created in namespace wisdom-pr-1123
Look up the wisdom-service url in the cluster


cp: correctly copy ancestor dirs in --parents mode

Fix a bug where cp failed to copy ancestor directories when using
the --parents option. For example, before this commit:
$ mkdir -p a/b/c d
$ cp --parents a/b/c d
$ find d
d
d/c

After this commit
$ mkdir -p a/b/c d
$ cp --parents a/b/c d
$ find d
d
d/a
d/a/b
d/a/b/c

This commit also adds the correct messages for --verbose mode:
$ cp -r --parents --verbose a/b/c d
a -> d/a
a/b -> d/a/b
'a/b/c' -> 'd/a/b/c'

Fixes #3332.

Okay there are some WIndows issues I need to resolve, but there is another failure reported in the GNU test suite that I want to explain.
The test case tests/cp/src-base-dot.sh claims to verify that
mkdir x y
cd y
cp --verbose -ab ../x/. .

succeeds, prints nothing to stdout, and is a no-op. However, it doesn't actually verify that the operation is a no-op, it only verifies that --verbose prints nothing. It just assumes that the directory x does not get copied into the directory y.
On the main branch, uutils cp will cause the directory x to be copied into the directory y, and nothing will be printed by --verbose (this is incorrect behavior). On this new branch, uutils cp will cause the directory x to be copied into the directory y but now --verbose will print the message ../x/. -> ./x (this is still incorrect behavior).
So the tests/cp/src-base-dot.sh test case was incorrectly passing on the main branch, and now it is correctly marked as a failure.

nice:
Warning: Congrats! The gnu test tests/cp/cp-parents is no longer ERROR!

Did you see
Error: GNU test failed: tests/cp/src-base-dot. tests/cp/src-base-dot is passing on 'main'. Maybe you have to rebase?

Did you see Error: GNU test failed: tests/cp/src-base-dot. tests/cp/src-base-dot is passing on 'main'. Maybe you have to rebase?

Yes, it is an expected failure. The explanation is in my comment above https://github.com/uutils/coreutils/pull/3894#issuecomment-1236041009 and I created a new issue for that here https://github.com/uutils/coreutils/issues/3897

sorry about that :(

Seems that it broke a few tests ?

Error: GNU test failed: tests/cp/cp-i. tests/cp/cp-i is passing on 'main'. Maybe you have to rebase?
Error: GNU test failed: tests/cp/src-base-dot. tests/cp/src-base-dot is passing on 'main'. Maybe you have to rebase?

tests/cp/src-base-dot is an expected new failure, see my comments above. However, tests/cp/cp-i does seem to be an unexpected new failure, so I'll fix that.

I'm going to mark this as a draft pull request because there's still some weird behaviors of GNU cp that I haven't quite figure out yet. Specifically, I need to figure out how to get the quoting behavior for cp -v to work in both the cp-i.sh test case and the cp-parents.sh case.

Okay, this is ready or review again.

I'm going to close this and come back with a pull request that doesn't attempt to implement the --verbose behavior.


NIFI-3671: Ensure that we use the existing ResourceClaim (if it exist…

…s) when swapping data in, instead of creating a new one. Otherwise, if the ResourceClaim is still writable, then we may archive the data and then write to it, which can cause a NullPointerException in FileSystemRepository
Thank you for submitting a contribution to Apache NiFi.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?

[ ] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

[ ] Has your PR been rebased against the latest commit within the target branch (typically master)?

[ ] Is your initial contribution a single, squashed commit?

For code changes:

[ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
[ ] Have you written or updated unit tests to verify your changes?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
[ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
[ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?

For documentation related changes:

[ ] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

reviewing


Concat operator

Concat operator implementation

Already implemented in https://github.com/ReactiveX/RxGo/pull/117


How to solve circular deps with event subscriber/dispatcher

This is probably more me being daft rather than a bug, but if I try to:

Inject a subscriber
That depends on a service
That itself depends on the dispatcher

I end up with what I think is a circular ref issue:
https://gist.github.com/EdwardIII/9ecc9fdb00f0822349ad
I don't think this happens in Symfony so is there a way I can make ->extend play nice?

Maybe in silex 2.0 you can do it in the dedicated subscribe() method on a service provider. But currently you have to do it in a boot() method or similiar.

@henrikbjorn That's great, subscribe() on 2.0 works perfectly.
I've updated the README in my test project, I'll try to pop a blog post up somewhere:
https://gist.github.com/EdwardIII/9ecc9fdb00f0822349ad#solution
Thanks!


update composer dependencies

Add PHP 7.2
🆙 graze/standards to v2
🆙 phpunit up to v7
🆙 symfony with v4

@biggianteye Thoughts on new baseline config for testing multiple versions locally? (also supports nightly php versions)


Update the README.md file to describe the repo

Updated the README.md file to better describe my repo

LGTM


feat(api): Consolidate pipette configuration

overview
This PR is also acting as a ticket. Pipette configuration data is duplicated in several places in both data and code, and the config loading system is unnecessarily complex.
This PR will consolidate pipette configuration to only use the version from shared-data, and update multi-channel pick up currents based on test data from @Carlos-fernandez.
changelog

Bundle pipette-config.json from shared-data into the api wheel
Use bundled pipette-config.json for pipette configs (no other copies of data)
Update multi-channel pick up current values based on test data

review requests
Tested on 🌔 🌔 --loads correctly with different pipettes (p300_multi_v1 and p300_single_v1).  @Carlos-fernandez please verify the new current values.

Got confirmation on current values from @Carlos-fernandez


Shadow volume culling and tighter shadow caster culling

Existing shadow caster culling takes no account of the camera. This PR adds the highly encapsulated class RenderingLightCuller which can cut down the casters in the shadow volume to only those which can cast shadows on the camera frustum.
Forward port of #82584
This is used to:

More accurately defer dirty updates to shadows when the shadow volume does not intersect the camera frustum.
Tighter cull shadow casters to the view frustum.

Lights dirty state is now automatically managed:

Continuous (tighter caster culling)
Static (all casters are rendered)

Explanation
You can see roughly how it works in this old video of mine (ignore the rooms and portals, that is a separate system):
https://www.youtube.com/watch?v=1WT5AXZlsDc
The blue lines from the light sources to the camera frustum show the extra culling planes.
How does it work?
At runtime, the routine checks each plane of the camera frustum, and finds whether it is facing either towards or away from the light (0 or 1). These bits for the 6 planes form a 6 bit number, which is the lookup.
The lookup tells us a list of corner points from the camera frustum which form a silhouette, which can be used to generate culling planes together with the light origin (3 points form a culling plane).
References:
http://lspiroengine.com/?p=153
http://www.terathon.com/gdc06_lengyel.pdf
Performance
In tests in TPS demo, without GI and just using shadows, in many areas this halves the number of drawcalls / vertex count, in some cases reduces drawcalls by a factor of 10x. This can lead to 10-300% increase in FPS (the increase in FPS depends on settings used, if fill rate is bottlenecking then improving shadows has less dramatic effect and vice versa).
In WroughtFlesh, which uses directional light only, I get a more modest 10% or so improvement if FPS, due to the tighter caster culling with the directional light. So it seems like the benefits are higher the more omnis / spots are used.
Notes

This is based on a resurrection of #33340 . That PR's approach wasn't compatible with the dirty flag optimization for omnis and spots, but this PR automatically manages dynamic lights to handle tighter culling, while reverting to rendering all casters when used in static manner.
This PR also adds the major optimization that shadow maps outside the view frustum don't need to be updated at all. There was some existing culling based on AABBs but this PR is far more accurate. This can lead to major performance gains where a lot of shadowed lights are in use (e.g. TPS demo).
The project setting is initially for testing, it can detect changes at runtime (if you e.g. change project setting from script). If it works correctly there may be no need to have it switchable, as it should be virtually always a win and the calculations are very little, and outweighed by any gains.

Tighter caster culling and Multiple Cameras
There is one more situation in which tighter caster culling is problematic: when multiple viewports are in use, and the shadow volume intersects multiple cameras.
In this situation tighter caster culling will work - it will do a tight cull on the first camera, and a full render for the second camera. The problem is that it will do 2 shadow renders per frame instead of one.
The answer used here is to detect this situation (in detect_light_intersects_multiple_cameras()) and switch to a different mode light_intersects_multiple_cameras.
This reverts to the legacy approach of doing a full render on the first update. However, we still want to detect the situation where it changes back to a single camera. This is done by means of a timeout after a certain number of frames without a double update.
Directional Lights
Directional lights are handled separately, they are always updated, and with different shadow maps if multiple cameras are used with viewports. Therefore they can always do the tighter caster cull.
There are some differences in the directional light culling compared with #82584, as the culling in 4.x relies on a brute force approach.

Production edit: This closes https://github.com/godotengine/godot/issues/57549.

I tested quickly on a high complexity scene and it spammed the following error:

I tested quickly on a high complexity scene and it spammed the following error:

I'll take a look. This may be due to some slight differences in cameras from 3.x (bear in mind this is a straight port and has had much more testing on 3.x). :+1:

Quick debugging check, changing MAX_PLANES to 32 makes the error go away.
The scene has multiple omnilights and a directionallight3D. The error spams while multiple omnilights are within the camera frustum. The error continues when the DirectionalLight3D is made not visible.
I'll try to make a reproduction project

Also worth checking is if setting MAX_PLANES to 17 makes it go away.
I've just pushed a slightly updated version with the lookup table showing all 8 entries (there was a strange half finished bit for only writing 7 entries on purpose, but I wrote this 3 years ago so can only try and remember why I did that, maybe it was a bug).
I'm not sure this would have created your bug but it is just possible, so try the latest version as well.

Okay, I've prepared an MRP that reliably reproduces the issue.
clip-plane-mrp.zip

Fantastic! Thanks so much for spending time on this. I can immediately see that light culler contains some garbage values so some kind of corruption is going on.

This should be fixed now.

Preparing the directional lights has been moved outside the multithreaded code, only the culling is performed there now, which should be thread safe.
I also fixed it to work with multiple directional lights. These are prepared in advance, so the cull_directional_light() can chop and change between the different directional lights (this is the pattern currently used in the code).

Tested locally, it works as expected. Visuals look correct too from my testing in various demo projects.
Benchmark on tps-demo
OS: Fedora 38
CPU: Intel Core i9-13900K
GPU: GeForce RTX 4090 (NVIDIA 535.113.01)
The project is modified to disable V-Sync. The FPS reported is the highest FPS attained over a period of 10 seconds after loading the level. Unfortunately, unlike the 3.x branch, I can't notice any performance improvement with this PR. This also applies to other demos like Truck Town and 3D platformer which I've tested as well.

Type
Before
After

4K Maximum Forward+
232
234

4K Minimum Forward+
961
962

720p Maximum Forward+[^1]
579
568

720p Minimum Forward+[^1]
1044
1033

4K Minimum Compatibility[^1]
471
472

720p Minimum Compatibility[^1]
466
476

[^1]: CPU is the bottleneck, with GPU sitting at less than 50% utilization.

Unfortunately, unlike the 3.x branch, I can't notice any performance improvement with this PR.

Oh I must have made a booboo in the fixup then, as there should be an improvement. I'll do a debug to check it's working correctly (I don't have many test projects in 4.x).

I tested again on the same scene and can confirm that the errors are gone.
In a section of the level that is mostly outdoors, I noticed a decrease in draw calls by about 25% (2500 -> 1900).
However, in an indoor section with multiple omnilights, I noticed an increase in the number of draw calls (14,300 -> 14,700). My guess is that something is forcing the omnilights to update every frame instead of just when changed. I'll try to isolate the case and confirm the theory

However, in an indoor section with multiple omnilights, I noticed an increase in the number of draw calls (14,300 -> 14,700). My guess is that something is forcing the omnilights to update every frame instead of just when changed. I'll try to isolate the case and confirm the theory

I'll have a look too in the mean time and see if I can work out what is going on.

After testing the large MRP @clayjohn mentioned above, my results are as follows:
Drawcalls (tight culling first, legacy second):
Outdoor 2530, 2873
Indoor 19990, 21440
Panoramic 3487, 4713
Free 1977, 7191
Character 1972, 2315

So improvement in all cases.
I haven't figured out yet how to move the indoor camera, these are the default views.
BTW I tested by switching on the LIGHT_CULLER_DEBUG_FLASH define, and changing the frequency to 64 (I got very low frame rate on my PC).
Then recorded the drawcalls given in the game.
Also I set LIGHT_CULLER_DEBUG_LOGGING, it tells you when it flashes the culling on and off.
Also I realised there was an out of date variable name in the debugging defined section, had to swap an active to light_culling_active (due to a variable rename) to get it to compile. I'll fix that in the PR and repush.

Thanks!

Tighter Shadow caster culling causes the shadows to disappear in some cases...
Run this test scene, and press home a handful of times to rotate the lights one increment at a time (end to rotate in reverse)
With tighter shadow caster culling enabled, shadows will disappear for a frame a few rotations in, only to pop back in, but this is not the case with it disabled
Shadows.zip

Tighter Shadow caster culling causes the shadows to disappear in some cases...

This could be a bug, usually better reported in an issue. I'll try out the project. :+1:

Sorry about that, was kind of in a rush yesterday and it was a bit late, The PR happened to be the 1st thing I was thinking about when I observed and quickly pinned down the issue.
What doesn't make sense is that it only disappears for 1 frame, only to reaper the next despite the fact there should be no update to lights that next frame, as they're static... So somehow they're dirtied one frame after the rotation, probably by the offending model (Female26 for me) and this has nothing to do with skinning as yea, a cube of roughly the AABB of the dummy also gets the same issue.
I'll open an issue now..


[SPARK-23937][SQL] Add map_filter SQL function

What changes were proposed in this pull request?
The PR adds the high order function map_filter, which filters the entries of a map and returns a new map which contains only the entries which satisfied the filter function.
How was this patch tested?
added UTs

cc @ueshin

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1762/
Test PASSed.

Merged build finished. Test PASSed.

Test build #94141 has started for PR 21986 at commit 3f88e2a.

Test build #94141 has finished for PR 21986 at commit 3f88e2a.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
trait UnaryHigherOrderFunction extends HigherOrderFunction with ExpectsInputTypes
trait ArrayBasedUnaryHigherOrderFunction extends UnaryHigherOrderFunction
trait MapBasedUnaryHigherOrderFunction extends UnaryHigherOrderFunction
case class MapFilter(

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94141/
Test FAILed.

retest this please

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1772/
Test PASSed.

Test build #94170 has started for PR 21986 at commit 3f88e2a.

retest this please

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1791/
Test PASSed.

Test build #94199 has started for PR 21986 at commit 3f88e2a.

Test build #94199 has finished for PR 21986 at commit 3f88e2a.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds the following public classes (experimental):
trait UnaryHigherOrderFunction extends HigherOrderFunction with ExpectsInputTypes
trait ArrayBasedUnaryHigherOrderFunction extends UnaryHigherOrderFunction
trait MapBasedUnaryHigherOrderFunction extends UnaryHigherOrderFunction
case class MapFilter(

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94199/
Test FAILed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1838/
Test PASSed.

Build finished. Test PASSed.

Test build #94272 has started for PR 21986 at commit 9bbaa3b.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1839/
Test PASSed.

Merged build finished. Test PASSed.

Test build #94273 has started for PR 21986 at commit 37e221c.

Test build #94273 has finished for PR 21986 at commit 37e221c.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94273/
Test FAILed.

retest this please

Test build #94272 has finished for PR 21986 at commit 9bbaa3b.

This patch passes all tests.
This patch does not merge cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94272/
Test PASSed.

Build finished. Test PASSed.

Test build #94280 has started for PR 21986 at commit 37e221c.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1842/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1849/
Test PASSed.

Test build #94289 has started for PR 21986 at commit b58a1de.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1851/
Test PASSed.

Test build #94291 has started for PR 21986 at commit 9c25ae6.

Test build #94280 has finished for PR 21986 at commit 37e221c.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94280/
Test PASSed.

Test build #94291 has finished for PR 21986 at commit 9c25ae6.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94291/
Test FAILed.

Test build #94289 has finished for PR 21986 at commit b58a1de.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94289/
Test PASSed.

Merged build finished. Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1904/
Test PASSed.

Test build #94363 has started for PR 21986 at commit 1823fb2.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1908/
Test PASSed.

Test build #94367 has started for PR 21986 at commit af79644.

LGTM.

Test build #94367 has finished for PR 21986 at commit af79644.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94367/
Test FAILed.

retest this please

Test build #94371 has started for PR 21986 at commit af79644.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1911/
Test PASSed.

Test build #94363 has finished for PR 21986 at commit 1823fb2.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
trait SimpleHigherOrderFunction extends HigherOrderFunction with ExpectsInputTypes
trait ArrayBasedSimpleHigherOrderFunction extends SimpleHigherOrderFunction
trait MapBasedSimpleHigherOrderFunction extends SimpleHigherOrderFunction

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94363/
Test PASSed.

Test build #94371 has finished for PR 21986 at commit af79644.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94371/
Test PASSed.

Thanks! merging to master.


IsInt gives Warning: NumberFormatter::parse()

When letter is parsed for validation IsInt, a warning is generated instead of issuing error self::INVALID. The warning is Warning: NumberFormatter::parse(): Number parsing failed in ..\vendor\zendframework\zendframework\library\Zend\I18n\Validator\IsInt.php on line 121

This was fixed with version 2.3.4. See: https://github.com/zendframework/zendframework/pull/6598


Backend needs documentation on how to run...

This repo looks interesting. Would you mind adding some more documentation on how to run the backend too? I would also like to contribute if possible.

@ytimocin Yeah, sure thing. Sorry, I didn't have time to document yet but I will write the document detail soon when I finish my work.


1402 postgis

https://github.com/bcgov/cas-registration/issues/1402

Probably (?) moving away from postgis, closing


Return true on successful bundle send

not a big problem but kinda confusing when your sendbundle returns null

This is a good change, but I'm going to hold off on merging til the v0.2 release just in case this breaks someone's workflow.

This is a good change, but I'm going to hold off on merging til the v0.2 release just in case this breaks someone's workflow.

that was fast lmao

This is superseded by 84778d1953fe4d0c4027d9d2762e3d4ba5b2b658, thanks for this PR though, it got us thinking about what to return here.


Incorrect Encounter Slots for Haina Desert

Dugtrio is incorrectly listed for Encounter slots 2, 4, 7, 8, 9, and 10.
Both Real96's Guide and Serebii lists Dugtrio for slots 2 and 4 only.
I only noticed when I encountered a Sandile for slot 10 in the desert.
I'm using the newest release v1.1.1
Edit: Resolution Cave (B1F) lists Zubat and Diglett for both Day and Night, when it should Golbat and Dugtrio.

Fixed. thanks


Add Renderer

This adds a simple point cloud renderer using wgpu and fixes issues when compiling it (and the pasture core libs) for wasm. See the readme for details on building for wasm and future improvements.

Thanks for the PR! I'll need some time to review this. The only thing that I saw right away is that we probably shouldn't include the SPIR-V files in the repository, since you are creating them at build time using the custom build script.

I agree with all of your points. Should be fixed now.

There is one missing test, if this is fixed I will merge this


problem with dcm_to_quat for simple rotation

A matrix represented a simple rotation; swapping the x and y.
julia> dcm = [0 1 0;1 0 0;0 0 1] |> SMatrix{3,3,Float64}
3×3 SMatrix{3, 3, Float64, 9} with indices SOneTo(3)×SOneTo(3):
 0.0  1.0  0.0
 1.0  0.0  0.0
 0.0  0.0  1.0

It doesn't rotate back.
julia> dcm_to_quat(dcm) |> quat_to_dcm
3×3 SMatrix{3, 3, Float64, 9} with indices SOneTo(3)×SOneTo(3):
 0.5  0.0  0.0
 0.0  0.5  0.0
 0.0  0.0  0.5

hmm, I guess it is not a "rotation"

Hi @cchderrick!
You are right. This DCM has determinant -1, indicating it has mirroring.


Skeleton not working... missing theme files

I think the title explain the pb already :)

The skeleton must be installed. See the readme https://github.com/flaviocopes/grav-skeleton-gravcart/blob/develop/README.md#installation

Sorry, didn't read the bottom note on readme... tks 👍


Error in author names in publication lists

Rémi's name is incorrectly formatted in all "Related Publication" lists for the KBase apps. The unicode is U+00E9.
See https://appdev.kbase.us/legacy/appcatalog/app/EFIToolsKBase/run_EFI_EST_Families/dev for example.

If the output HTML document is encoded as UTF-8, the correct character can be inserted directly without encoding.  If not, then using é will insert the correct character.


Archive TDWG 2012 conference

Archive TDWG 2012 conference: http://www.tdwg.org/conference2012/

[ ] Host: Keping Ma?
[ ] Venue: China People's Palace Hotel in Beijing, China (could not find functional website)
[x] Date
[ ] Programme: programme booklet or print of programme page or TDWG_Program_Final.pdf or file server
[ ] Proceedings: There are links to the abstract (e.g. https://mbgserv18.mobot.org/ocs/index.php/tdwg/2011/paper/view/83) but they are all broken. Do we have proceedings?
[ ] Presentations: Linked from this pdf but not on the website. Will have to download from file server.

[x] Conference page created
[x] Cover image added
[ ] Files in staging on Google Drive
[ ] Files on file server
[ ] Conference page reviewed by @stanblum

Conference page now available at http://localhost:9000/conferences/2012/

@stanblum @gkampmeier can you verify host, venue and track proceedings?


neuropil correction factor doesn't match with subquantities name

In imaging.Calcium, the neuropil correction factor seems to be set with the constant NEUROPIL_CORRECTION_FACTOR which is set to 0.8.
However, in analysis.read_NWB, in the function list_subquantities, the description of the neuropil corrected fluorescence trace doesn't match with the chosen parameter ("F-0.7Fneu", "d(F-0.7Fneu)").

Yes, that's a deprecated strategy. I was extracting the neuropil correction factor from the subquantity string. I just removed those deprecated settings in d15b0.
Now, the dFoF is thought to be always built by calling data.build_dFoF, like this:
data = analysis.read_NWB(filename)
data.build_dFoF() # without arguments, the default params are from imaging/Calcium.py

Or, with arguments:
data.build_dFoF(
                   roi_to_neuropil_fluo_inclusion_factor=1.20,
                   neuropil_correction_factor=0.9,
                   method_for_F0='sliding-percentile',
                   percentile=5.,
                   sliding_window=600.)


[Feature Request]: Re Designing FAQ page

Is there an existing issue for this?

[X] I have searched the existing issues

Feature Description
I would like to re design the FAQ section, this section seems not looking good.
I want re design this to better.
##Screenshots

Use Case
Make the section looking good and proper.
Benefits
No response
Add ScreenShots
No response
Priority
High
Record

[X] I have read the Contributing Guidelines
[X] I'm a GSSOC'24 contributor
[X] I have starred the repository

I want to resolve the issue, kindly assign me the task


Code Triage auth failure

Hi,
I still receive emails about auth failure, even after following the "the logout, login" instructions.
Christophe

Sorry for the delay. I'll be able to check into it tomorrow

I received a code triage mail this morning, so it seems to work now ;)
Thanks anyway !

Great. I'm seeing that your account has an active token right now so you should be set. Let me know if you get another email about auth failure.


fix(plex): remove traefik annotation on service

Description of the change
Removed traefik annotation on plex service

Thanks! <3


add go.mod, update to Python 3 syntax and add bash shebash

Hi @ryanbressler,
Thanks for sharing the numpy2go PoC project! It was exactly what I was looking for :)
This PR updates the repo to use Go modules, adds a Bash shebang to run.sh so that it may be invoked from other shells, and updates the Python code to Python 3 syntax.
Wish you a lovely summer.
Cheers,
Robin

Thanks! Are you doing anything cool with python and go?

Thanks! Are you doing anything cool with python and go?

I'm doing a neuroscience course currently and ran into performance issues with Numpy, so want to see if Go could speed up the computations.


Selectproperties parameter does not filter the managed properties requested instead returning all properties

In SP.SimpleDataTable xml response, search api is returning below properties, eventough passing Selectproperties parameters with just Title as property required response.
Rank,DocId,Title,SiteId,WebId,UniqueId,PartitionId,UrlZone,Culture,ResultTypeId,RenderTemplateId,GeoLocationSource
[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 3b3e4019-1df2-2317-7a63-dbdfba66a3f4
Version Independent ID: 7f409188-f8ae-94cd-d174-b0c7b5183096
Content: SharePoint Search REST API overview
Content Source: docs/general-development/sharepoint-search-rest-api-overview.md
Product: sharepoint
Technology: sharepoint-framework
GitHub Login: @spdevdocs
Microsoft Alias: spdevdocs

Ping @wobba

By design as a mimimal set of properties is always returned with the SP REST API.


Renaming .txt file to .glsl while it is open in the editor has unpredictable results

Godot version
v4.0.beta.custom_build [3c9bf4bc2]
System information
Fedora 36 KDE, RTX 3080
Issue description
If I create a TextFile in Godot and change the file extension to .glsl I get different behaviour if the file was open in the editor at the time of renaming.
Steps to reproduce
First attempt, file not open when renaming:

Create a TextFile in Godot and name it "test1.txt"
Right click it in the FileSystem dock and rename it to "test1.glsl"
Double click the file -> ShaderFile tab opens.
In a script, both load and preload of the file produces a Resource / RDShaderFile

Everything works as expected.
Second attempt, file open when renaming:

Create a TextFile in Godot and name it "test2.txt"
Double click the file to open it in the Godot editor.
Right click it in the FileSystem dock and rename it to "test2.glsl"
Close the file in the Godot editor.
Double click the file -> It opens in the editor.
In a script, load the file produces a Resource / RDShaderFile
In a script, preload-ing the file produces a TextFile

Minimal reproduction project
I think it's easier to see the strange behaviour by reproducing the steps in the editor.

I'm not sure if my issue is related; But, when I open test.glsl to edit using the built-in Godot text editor, then load("test.glsl") gives me a TextFile, even when the .import says RDShaderFile. When I close the file in the built-in Godot text editor, it fixes.
Was a little weird to figure out, but now it's no issue. I just hit "close" when I'm done editing and the code works.


Fix publishing of static binaries

Fixes: #21
See the successful publish github action: https://github.com/fulcrumgenomics/tmp-fqgrep-githubactions/actions/runs/4239758449.
See the binaries automatically generated by the publish github action: https://github.com/fulcrumgenomics/tmp-fqgrep-githubactions/releases/tag/v16

fqgrep-x86_64-apple-darwin runs with no errors on MacBook Air (M1, Ventura 13.0.1)

fqgrep-x86_64-apple-darwin runs without errors on a MacBook Air (M2, Ventura 13.2.1)
Using docker on the same computer, I can also run fqgrep-x86_64-unknown-linux-musl with no issues (docker images based on alpine and ubuntu both work).
When I try to run fqgrep-arm-unknown-linux-gnueabihf in an ubuntu container I get this error: "qemu-arm: Could not open '/lib/ld-linux-armhf.so.3': No such file or directory"

@bnovak32 I think the arm release you need to use the arm32v7/ubuntu docker image (32-bit).  I don't have an arm64 target (e.g. aarch64-unknown-linux-gnu).  I'd be glad for a PR if folks want one.

@bnovak32 I think the arm release you need to use the arm32v7/ubuntu docker image (32-bit). I don't have an arm64 target (e.g. aarch64-unknown-linux-gnu). I'd be glad for a PR if folks want one.

I think it would be great to have an arm64 version because that's what you get when running docker on an m1/m2 Mac. I try to always build amd64/arm64 multi-platform docker images to easily transfer workflows across different systems. I can just install from source, though, so it's not a big deal.

@bnovak32 if you want to test out a aarch64-unknown-linux-gnu, it's a simple as creating a new repo (not forking), pushing the main branch here up there, then update the publish with the github actions (add the aarch64-unknown-linux-gnu target), then pushing a tag to kick off the publish github action.


fix: eslint errors not fixable with --fix

Jira
n/a
Summary
Manually fix eslint errors not fixable with --fix and a few that are.
How to test
Review file changed.

PR was released with v2.14.0


Poor performance in single-buffer test

We test run single hash jobs on one core:

ISA-L

*_ctx_mgr_init(mgr_);
*_ctx_mgr_submit(mgr_, &ctxpool_, buf, text_length, HASH_ENTIRE);
*_ctx_mgr_flush(mgr_);

OpenSSL

*_Init(&ctx_);
*_Update(&ctx_, buf, text_length);
*_Final(digest_, &ctx_);

OpenSSL is better than ISA-L(same result in update test):

Library
Algorithm
100B(us)
1KB(us)
256KB(us)
512KB(us)
1MB(us)

OpenSSL
MD5
0.259577
1.973022
466.580400
930.553100
1858.246400

ISA-L
MD5
0.510208
3.434382
788.836000
1580.523900
3185.874900

OpenSSL
SHA1
0.243943
1.435872
329.519600
654.810200
1309.367800

ISA-L
SHA1
0.326654
1.885040
393.532000
786.502300
1573.971600

OpenSSL
SHA256
0.432444
3.031160
714.659800
1429.176200
2858.618900

ISA-L
SHA256
0.597858
4.251419
997.934900
1993.451800
3989.759000

OpenSSL
SHA512
0.337439
2.207246
477.429400
953.789300
1908.231500

ISA-L
SHA512
0.598538
4.924967
1119.788800
2237.286500
4411.418500

If the test is wrong, please look through my steps above and tell me what I am doing wrong.
If not, please tell me why this result.
Thanks

Hi @blackbird52.  I can't say what is wrong with your test.  As you may know, the biggest benefit for using the multi-buffer hashing comes from submitting multiple independent jobs at once.  We do have some special single-buffer or even two-buffer optimizations for when we detect the number of lanes filled is only at 1 or 2.

@blackbird52 , could you give more information like CPU, memory and other related system configuration?

Have the same problem.
We try to test openssl and isa-l，found the performance of isa-l single buffer for SHA256 is worse than Openssl.

Hardware & Software Ingredients

Item
Description

CPU
Intel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHzNumber of cores 12, Number of threads 48

Memory
128 GB

Linux kernel version
3.10.107

GCC version
4.8.5

OpenSSL version
1.0.1e

ISA-L version
2.22.0

@blackbird52 , could you give more information like CPU, memory and other related system configuration?

Hardware & Software Ingredients

Item
Description

CPU
Intel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHzNumber of cores 12, Number of threads 48

Memory
128 GB

Linux kernel version
3.10.107

GCC version
4.8.5

OpenSSL version
1.0.1e

ISA-L version
2.22.0

I see the same on mac and linux containers.
And this is my machine:
macOS Catalina 10.15.4 (19E287)
CPU: 2.6 GHz 6-Core Intel Core i7
Memory: 16 GB 2400 MHz DDR4
gcc: Apple clang version 11.0.3
yasm: 1.3.0
isa-l_crypto: tag v2.22.0
openssl: OpenSSL 1.1.1g  21 Apr 2020

Here is my only diff in md5_mb/md5_mb_vs_ossl_perf.c:
 // Set number of outstanding jobs
-#define TEST_BUFS 32
+#define TEST_BUFS 1

building:
make DEFINES="-I/usr/local/opt/openssl/include/" LDFLAGS="-L/usr/local/opt/openssl/lib/" -f Makefile.unx md5_mb_vs_ossl_perf

running:
md5_openssl_cold: runtime =    4260941 usecs, bandwidth 3200 MB in 4.2609 sec = 787.49 MB/s
multibinary_md5_cold: runtime =    7198752 usecs, bandwidth 3200 MB in 7.1988 sec = 466.11 MB/s
Multi-buffer md5 test complete 1 buffers of 33554432 B with 100 iterations
 multibinary_md5_ossl_perf: Pass

I can also get similar 2x slower times in linux VM.
Let me know if I can provide more information.

@guymguym I think there is an obvious issue with your results.

md5_openssl_cold: runtime =    4260941 usecs, bandwidth 3200 MB in 4.2609 sec = 787.49 MB/s
multibinary_md5_cold: runtime =    7198752 usecs, bandwidth 3200 MB in 7.1988 sec = 466.11 MB/s

I would expect multiple gigabytes/s here.  Could it be that the host is not passing on all the native instruction set support to the VM?  If configured to pass native, the VM should see AVX2.
@blackbird52, is the included multi-buffer test showing expected results?

@guymguym , yes, multi-buffer test got expected results.

@gbtucker @blackbird52 did you notice my change tO #define TEST_BUFS 1? I meant to test a single buffer so I wouldn’t expect to get multiple gigabytes. But I do expect to get similar results to single buffer of openssl and I get half the performance...

@gbtucker
I'm concerned about the performance in single-buffer test and want to know why we got these results.
Is OpenSSL better than ISA-L in this case, or is there any optimization to run ISA-L effectively?
Thanks!

Sorry @guymguym, I did miss the #define TEST_BUFS 1 meaning that you had modified the test.
As I said there are some optimizations for single and dual buffer but many of these are for later CPUs.  If your system will primarily only utilize a single buffer at a time you may find the integration for multi-buffer hashing is not worth it.  It may be difficult to say where the crossover point is for your integration without some experimentation.

thanks @gbtucker. Even with 2 buffers I already get better performance:
md5_openssl_cold: runtime =    4323584 usecs, bandwidth 3200 MB in 4.3236 sec = 776.08 MB/s
multibinary_md5_cold: runtime =    3604363 usecs, bandwidth 3200 MB in 3.6044 sec = 930.94 MB/s
Multi-buffer md5 test complete 2 buffers of 16777216 B with 100 iterations
 multibinary_md5_ossl_perf: Pass

However, I am not sure how to use multibuffers in my case - perhaps you can help me understand. My server is processing multiple streams of md5 concurrently (an S3 endpoint). When a stream starts it initializes an md5 context, and then whenever a buffer is read from the socket it is submitted to the mgr with it's stream's context. But then I immediately have to call flush, because I want to submit the next buffer of that stream, so essentially to use more than one multibuffer per flush, what kind of event loop synchronization or "tick" should I implement? Is there a reference project that uses multibuffer for data streaming?

@guymguym, it sounds like you want to split hash jobs into partial updates.  This is entirely possible with the multi-buffer interface without having to flush after each.  Typically the updates are tracked in the same job context with ctx.user_data, filled with info to act like a callback to track, and resubmit a job with the next update.  Some of the examples and unit tests do this.
I would suggest having a context pool worker that manages this.  Also rolling_hash/chunking_with_mb_hash.c is a good example although this one doesn't use updates.

Just a late bit of (hopefully useful) info for anyone who stumbles across this:
From my quick scan of the code, it seems like the 1 or 2 buffer optimisations @gbtucker mentions only applies to SHA1/SHA256. For MD5, which is what @guymguym is testing here, there doesn't appear to be any optimisations for fewer than the ideal number of buffers, so ISA-L will always use the full SIMD width with two interleaved vectors regardless of how many lanes are active.
OpenSSL's implementation is quite good (though it can be beaten), and will be much faster than ISA-L's current "max-throughput"-only approach for single buffer scenarios.


python3Packages.jaxlib: fix #282184 and migrate to cuda redist packages

Description of changes
This change snowballed to cover a number of improvements:

Fix https://github.com/NixOS/nixpkgs/issues/282184. The --config=cuda flag was lost at some point, disabling CUDA builds even with cudaSupport = true.
Migrate to using the CUDA redist packages instead of cudatoolkit.
Unify stdenv behind effectiveStdenv following a pattern that has precedent in OpenCV's derivation and was recommended in https://github.com/NixOS/nixpkgs/pull/288857#discussion_r1490003016.

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Currently blocked on https://github.com/google/jax/issues/19814

@SomeoneSerge @GaetanLepage @ndl This fixes the CUDA jaxlib build, and is now ready for review. Let me know if you have any feedback!
Only remaining TODO AFAIK is to get the aarch64-linux fetchAttrs hash.

I don't currently have access to an aarch64-linux machine to fetch the hash with. Anyone who might be able to help out? If not, I propose that we drop support for the time being.

Result of nixpkgs-review pr 288857 run on x86_64-linux 1

10 packages marked as broken and skipped:

python311Packages.dm-sonnet
python311Packages.dm-sonnet.dist
python311Packages.elegy
python311Packages.elegy.dist
python311Packages.tensorflow-datasets
python311Packages.tensorflow-datasets.dist
python311Packages.treeo
python311Packages.treeo.dist
python311Packages.treex
python311Packages.treex.dist

26 packages failed to build:

libretranslate (python311Packages.libretranslate)
libretranslate.dist (python311Packages.libretranslate.dist)
python311Packages.aeppl
python311Packages.aeppl.dist
python311Packages.aesara
python311Packages.aesara.dist
python311Packages.augmax
python311Packages.augmax.dist
python311Packages.coffea
python311Packages.coffea.dist
python311Packages.dalle-mini
python311Packages.dalle-mini.dist
python311Packages.distrax
python311Packages.distrax.dist
python311Packages.dm-haiku
python311Packages.dm-haiku.dist
python311Packages.flax
python311Packages.flax.dist
python311Packages.objax
python311Packages.objax.dist
python311Packages.rlax
python311Packages.rlax.dist
python311Packages.vqgan-jax
python311Packages.vqgan-jax.dist
tts
tts.dist

63 packages built:

python311Packages.argos-translate-files
python311Packages.argos-translate-files.dist
python311Packages.argostranslate
python311Packages.argostranslate.dist
python311Packages.arviz
python311Packages.arviz.dist
python311Packages.awkward
python311Packages.awkward.dist
python311Packages.bambi
python311Packages.bambi.dist
python311Packages.blackjax
python311Packages.blackjax.dist
python311Packages.chex
python311Packages.chex.dist
python311Packages.correctionlib
python311Packages.correctionlib.dist
python311Packages.ctranslate2
python311Packages.ctranslate2.dist
python311Packages.dask-awkward
python311Packages.dask-awkward.dist
python311Packages.equinox
python311Packages.equinox.dist
python311Packages.faster-whisper
python311Packages.faster-whisper.dist
python311Packages.gymnasium
python311Packages.gymnasium.dist
python311Packages.jax
python311Packages.jax.dist
python311Packages.jaxlib (python311Packages.jaxlib-build ,python311Packages.jaxlibWithoutCuda)
python311Packages.jaxlib.dist (python311Packages.jaxlib-build.dist ,python311Packages.jaxlibWithoutCuda.dist)
python311Packages.jaxlibWithCuda
python311Packages.jaxlibWithCuda.dist
python311Packages.jaxopt
python311Packages.jaxopt.dist
python311Packages.jmp
python311Packages.jmp.dist
python311Packages.mplhep
python311Packages.mplhep.dist
python311Packages.numpyro
python311Packages.numpyro.dist
python311Packages.optax
python311Packages.optax.dist
python311Packages.optax.testsout
python311Packages.pymc
python311Packages.pymc.dist
python311Packages.pytensor
python311Packages.pytensor.dist
python311Packages.skrl
python311Packages.skrl.dist
python311Packages.tensorflow-bin
python311Packages.tensorflow-bin.dist
python311Packages.torchrl
python311Packages.torchrl.dist
python311Packages.translatehtml
python311Packages.translatehtml.dist
python311Packages.uproot
python311Packages.uproot.dist
python311Packages.vector
python311Packages.vector.dist
whisper-ctranslate2
whisper-ctranslate2.dist
wyoming-faster-whisper
wyoming-faster-whisper.dist

Amazing work ! Thanks so much @samuela !
Hash for aarch64-linux is: sha256-NehnpA4m+Fynvh0S6WKy/v9ab81487NE9ahvbS70wjY=

Thanks @GaetanLepage! Just pushed a commit with the aarch64-linux fetchAttrs hash, but is otherwise identical to the previous one

but is otherwise identical to the previous one

This is not suprising as you have just changed cuda-related stuff right ?

Result of nixpkgs-review pr 288857 run on x86_64-linux 1

10 packages marked as broken and skipped:

python311Packages.dm-sonnet
python311Packages.dm-sonnet.dist
python311Packages.elegy
python311Packages.elegy.dist
python311Packages.tensorflow-datasets
python311Packages.tensorflow-datasets.dist
python311Packages.treeo
python311Packages.treeo.dist
python311Packages.treex
python311Packages.treex.dist

26 packages failed to build:

libretranslate (python311Packages.libretranslate)
libretranslate.dist (python311Packages.libretranslate.dist)
python311Packages.aeppl
python311Packages.aeppl.dist
python311Packages.aesara
python311Packages.aesara.dist
python311Packages.augmax
python311Packages.augmax.dist
python311Packages.coffea
python311Packages.coffea.dist
python311Packages.dalle-mini
python311Packages.dalle-mini.dist
python311Packages.distrax
python311Packages.distrax.dist
python311Packages.dm-haiku
python311Packages.dm-haiku.dist
python311Packages.flax
python311Packages.flax.dist
python311Packages.objax
python311Packages.objax.dist
python311Packages.rlax
python311Packages.rlax.dist
python311Packages.vqgan-jax
python311Packages.vqgan-jax.dist
tts
tts.dist

63 packages built:

python311Packages.argos-translate-files
python311Packages.argos-translate-files.dist
python311Packages.argostranslate
python311Packages.argostranslate.dist
python311Packages.arviz
python311Packages.arviz.dist
python311Packages.awkward
python311Packages.awkward.dist
python311Packages.bambi
python311Packages.bambi.dist
python311Packages.blackjax
python311Packages.blackjax.dist
python311Packages.chex
python311Packages.chex.dist
python311Packages.correctionlib
python311Packages.correctionlib.dist
python311Packages.ctranslate2
python311Packages.ctranslate2.dist
python311Packages.dask-awkward
python311Packages.dask-awkward.dist
python311Packages.equinox
python311Packages.equinox.dist
python311Packages.faster-whisper
python311Packages.faster-whisper.dist
python311Packages.gymnasium
python311Packages.gymnasium.dist
python311Packages.jax
python311Packages.jax.dist
python311Packages.jaxlib (python311Packages.jaxlib-build ,python311Packages.jaxlibWithoutCuda)
python311Packages.jaxlib.dist (python311Packages.jaxlib-build.dist ,python311Packages.jaxlibWithoutCuda.dist)
python311Packages.jaxlibWithCuda
python311Packages.jaxlibWithCuda.dist
python311Packages.jaxopt
python311Packages.jaxopt.dist
python311Packages.jmp
python311Packages.jmp.dist
python311Packages.mplhep
python311Packages.mplhep.dist
python311Packages.numpyro
python311Packages.numpyro.dist
python311Packages.optax
python311Packages.optax.dist
python311Packages.optax.testsout
python311Packages.pymc
python311Packages.pymc.dist
python311Packages.pytensor
python311Packages.pytensor.dist
python311Packages.skrl
python311Packages.skrl.dist
python311Packages.tensorflow-bin
python311Packages.tensorflow-bin.dist
python311Packages.torchrl
python311Packages.torchrl.dist
python311Packages.translatehtml
python311Packages.translatehtml.dist
python311Packages.uproot
python311Packages.uproot.dist
python311Packages.vector
python311Packages.vector.dist
whisper-ctranslate2
whisper-ctranslate2.dist
wyoming-faster-whisper
wyoming-faster-whisper.dist

This is not suprising as you have just changed cuda-related stuff right ?

Yup! just wanted to clarify to ease reviewing since I was force pushing

Result of nixpkgs-review pr 288857 run on aarch64-linux 1

12 packages marked as broken and skipped:

python311Packages.aeppl
python311Packages.aeppl.dist
python311Packages.aesara
python311Packages.aesara.dist
python311Packages.bambi
python311Packages.bambi.dist
python311Packages.pymc
python311Packages.pymc.dist
python311Packages.pytensor
python311Packages.pytensor.dist
python311Packages.treeo
python311Packages.treeo.dist

24 packages failed to build:

python311Packages.arviz
python311Packages.arviz.dist
python311Packages.augmax
python311Packages.augmax.dist
python311Packages.awkward
python311Packages.awkward.dist
python311Packages.blackjax
python311Packages.blackjax.dist
python311Packages.coffea
python311Packages.coffea.dist
python311Packages.correctionlib
python311Packages.correctionlib.dist
python311Packages.dask-awkward
python311Packages.dask-awkward.dist
python311Packages.mplhep
python311Packages.mplhep.dist
python311Packages.objax
python311Packages.objax.dist
python311Packages.torchrl
python311Packages.torchrl.dist
python311Packages.uproot
python311Packages.uproot.dist
python311Packages.vector
python311Packages.vector.dist

21 packages built:

python311Packages.chex
python311Packages.chex.dist
python311Packages.equinox
python311Packages.equinox.dist
python311Packages.gymnasium
python311Packages.gymnasium.dist
python311Packages.jax
python311Packages.jax.dist
python311Packages.jaxlib (python311Packages.jaxlib-build ,python311Packages.jaxlibWithoutCuda)
python311Packages.jaxlib.dist (python311Packages.jaxlib-build.dist ,python311Packages.jaxlibWithoutCuda.dist)
python311Packages.jaxopt
python311Packages.jaxopt.dist
python311Packages.jmp
python311Packages.jmp.dist
python311Packages.numpyro
python311Packages.numpyro.dist
python311Packages.optax
python311Packages.optax.dist
python311Packages.optax.testsout
python311Packages.skrl
python311Packages.skrl.dist

Result of nixpkgs-review pr 288857 run on x86_64-linux 1

10 packages marked as broken and skipped:

python311Packages.dm-sonnet
python311Packages.dm-sonnet.dist
python311Packages.elegy
python311Packages.elegy.dist
python311Packages.tensorflow-datasets
python311Packages.tensorflow-datasets.dist
python311Packages.treeo
python311Packages.treeo.dist
python311Packages.treex
python311Packages.treex.dist

24 packages failed to build:

libretranslate (python311Packages.libretranslate)
libretranslate.dist (python311Packages.libretranslate.dist)
python311Packages.aeppl
python311Packages.aeppl.dist
python311Packages.aesara
python311Packages.aesara.dist
python311Packages.augmax
python311Packages.augmax.dist
python311Packages.dalle-mini
python311Packages.dalle-mini.dist
python311Packages.distrax
python311Packages.distrax.dist
python311Packages.dm-haiku
python311Packages.dm-haiku.dist
python311Packages.flax
python311Packages.flax.dist
python311Packages.objax
python311Packages.objax.dist
python311Packages.rlax
python311Packages.rlax.dist
python311Packages.vqgan-jax
python311Packages.vqgan-jax.dist
tts
tts.dist

65 packages built:

python311Packages.argos-translate-files
python311Packages.argos-translate-files.dist
python311Packages.argostranslate
python311Packages.argostranslate.dist
python311Packages.arviz
python311Packages.arviz.dist
python311Packages.awkward
python311Packages.awkward.dist
python311Packages.bambi
python311Packages.bambi.dist
python311Packages.blackjax
python311Packages.blackjax.dist
python311Packages.chex
python311Packages.chex.dist
python311Packages.coffea
python311Packages.coffea.dist
python311Packages.correctionlib
python311Packages.correctionlib.dist
python311Packages.ctranslate2
python311Packages.ctranslate2.dist
python311Packages.dask-awkward
python311Packages.dask-awkward.dist
python311Packages.equinox
python311Packages.equinox.dist
python311Packages.faster-whisper
python311Packages.faster-whisper.dist
python311Packages.gymnasium
python311Packages.gymnasium.dist
python311Packages.jax
python311Packages.jax.dist
python311Packages.jaxlib (python311Packages.jaxlib-build ,python311Packages.jaxlibWithoutCuda)
python311Packages.jaxlib.dist (python311Packages.jaxlib-build.dist ,python311Packages.jaxlibWithoutCuda.dist)
python311Packages.jaxlibWithCuda
python311Packages.jaxlibWithCuda.dist
python311Packages.jaxopt
python311Packages.jaxopt.dist
python311Packages.jmp
python311Packages.jmp.dist
python311Packages.mplhep
python311Packages.mplhep.dist
python311Packages.numpyro
python311Packages.numpyro.dist
python311Packages.optax
python311Packages.optax.dist
python311Packages.optax.testsout
python311Packages.pymc
python311Packages.pymc.dist
python311Packages.pytensor
python311Packages.pytensor.dist
python311Packages.skrl
python311Packages.skrl.dist
python311Packages.tensorflow-bin
python311Packages.tensorflow-bin.dist
python311Packages.torchrl
python311Packages.torchrl.dist
python311Packages.translatehtml
python311Packages.translatehtml.dist
python311Packages.uproot
python311Packages.uproot.dist
python311Packages.vector
python311Packages.vector.dist
whisper-ctranslate2
whisper-ctranslate2.dist
wyoming-faster-whisper
wyoming-faster-whisper.dist

Thanks for the reviews @SomeoneSerge and @GaetanLepage! I'll proceed to merge tomorrow unless there are any objections?


CRUD support for properties and meta data in SDK

Created by: @bayegan Jun 07 2021 at 07:11 CET
Last updated: Nov 19 2021 at 18:21 CET
ClickUp ID: IK-960
As a user of the SDK (e.g., developer), I should be able to perform CRUD on all user properties and meta data supported by the SDK.
Acceptance criteria:
Properties and meta data as defined by https://github.com/indykite/jarvis-proto/blob/develop/proto/identity/v1/attributes.proto#L111
The SDK provides methods to create properties and meta data
The SDK provides methods to read properties and meta data data
The SDK provides methods to update properties and meta data
The SDK provides methods to delete properties and meta data

Commented by: @bayegan Jun 07 2021 at 07:17 CET
@ Krisztian Marko @marvinkite : are the properties and meta data we are referring to here profile properties and meta data ? where can i find a complete list of the properties and meta data that we support?

Commented by: @marvinkite Jun 07 2021 at 07:29 CET
https://github.com/indykite/jarvis-proto/blob/develop/proto/identity/v1/attributes.proto#L111https://github.com/indykite/jarvis-proto/blob/develop/proto/identity/v1/attributes.proto# L111
message PropertyMetadata {
  // Primary is true if this instance of a multi-value property is the default to pick from array at first.
  bool primary = 1;
  // AssuranceLevel shows the level of assurance of the property value.
  AssuranceLevel assurance_level = 2;
  // Issuer is the identifier of the source of property value.
  //
  // Example: ["indykite.id", "aW5keWtpRGWAAAIPAAAAAA"]
  string issuer = 3;

  // VerificationTime represent the time when AssuranceLevel was changed for Verifier.
  google.protobuf.Timestamp verification_time = 4;

  // Verifier is the identifier of provider carried out the verification process.
  //
  // Example: ["indykite.id", "aW5keWtpRGWAAAIPAAAAAA"]
  string verifier = 5;
}

Commented by: @bayegan Jun 07 2021 at 08:08 CET
tnx @marvinkite !


make distclean does not remove dSYM cruft on macOS

The makefile on the rule_count_breakout branch does not re remove compiled cruft on macOS.
We suggest something along the lines of:
diff --git a/makefile b/makefile
index 17c67a3..f14a580 100644
--- a/makefile
+++ b/makefile
@@ -40,6 +40,7 @@ clean:

 distclean: clean
        -rm -fr ${PROJ}$E test a.out VERSION 2>/dev/null
+       -rm -fr ${PROJ}.dSYM 2>/dev/null

 test: build
        ./${PROJ}-test.sh -v

You may wish to wait to address this until we have a pull request (hopefully later tonight).  That pull request will contain a similar makefile hack (with additional makefile mods) (without the XXX's) that you can modify later on if you wish.

This will be addressed in the pull request we are submitting.


Documentation: Update security.https.md

Reflect renaming of 2 tables from OC 3.0 -> OC 4.0 DB migration script.

Looks good, thanks for the contribution @pawohl!


Ability to write TSDB sequences of any Serializable, rather than just primitive elements

Suggested additional feature for Streamline/Druid.  Metron gets a lot of value from writing time series of STATS statistical sketch objects (T-Digests), then re-processing to derive primitive TS as desired at query time.

Happy to arrange a demo if you would like.


fix(render): handle large messages when calculating line width

Some misbehaving plugin code was sending very large messages to vim.notify(), which would trigger an exception when calculating the max line width for rendering:
[...]/notify/render/default.lua:6: too many results to unpack

To fix this, I updated the implementation to calculate the max line width in-place. This ultimately ends up being much more efficient because it is not creating a new table with vim.tbl_map().
Here's some benchmark data:
https://gist.github.com/flrgh/349aecd8ca35d20cf0526ec8b218657c
In the large input benchmarks, the new function is ~2x faster. For small inputs (which are probably most common for vim.notify()), the speedup is even larger at 3-4x.
Other changes:

I noticed the same snippet appeared in multiple render modules, so I factored it out into notify.util.max_line_width() and added a unit test
I updated the calculation to use vim.api.nvim_strwidth() when available (see #247).

Awesome thanks for the PRs! And apologies for the delay :sweat_smile:


Lostfilm doesn't work

Few days (or maybe weeks) I can't get results from lostfilm. Nothing was changed on my side.

2019-06-23 16:48:25.150 T:3007275888  NOTICE: [script.elementum.nova] >    LostFilm search URL: http://www.lostfilm.tv/series/escape_at_dannemora/season_1/episode_2/
2019-06-23 16:48:26.403 T:3007275888  NOTICE: [script.elementum.nova] >>   LostFilm returned  0 results in 4.2 seconds

Try relogin on web. After some incorrect login lostfilm need enter captcha.

@Nemiroff Hello. I tried this before my post. It doesnt helped.

Some change in response break check auth. Will be fixed in new release

Thanks a lot!