Maximum number of keys and multiple dicts in a database?

Hi, I was wondering about (a) what is the maximum number of keys that can be used in sqlitedict? and (b) how to use to use multiple dicts in the same database (that's a feature you listed in the readme I guess) and what is the maximum number of such dicts that can be present in the database?
Typically, is it possible to have like 20 billion keys whose values are lists of integers in a single dict? Also can I have hundreds of such dicts in the same db file?
Thanks,
Ayaan

Hi @ayaanhossain
(a) All limitation from sqlite, look at this page Maximum Number Of Rows In A Table and Maximum Database Size
(b) Yes, it's possible through set tablename parameter, looks at example
from sqlitedict import SqliteDict

fname = "file.sqlite"
dicts = ["first", "second"]  # two names for our dicts

with SqliteDict(filename=fname, tablename=dicts[0], autocommit=True) as dct:
# Write data to 'first' dict
    dct[1] = 100
    dct[2] = 200

with SqliteDict(filename=fname, tablename=dicts[1], autocommit=True) as dct:
# Write data to 'second' dict
    dct[1] = 123
    dct[2] = 456
    dct[3] = 3

# Read all dicts
for dictionary_name in dicts:
    with SqliteDict(filename=fname, tablename=dictionary_name) as dct:
        print(dictionary_name, dct.items())

"""
Output:
('fist', [('1', 100), ('2', 200)])
('second', [('1', 123), ('2', 456), ('3', 3)])
"""

Thanks. Learnt a lot.


inherit user changes before deleting user

Environment

Python version:  Python 2.7.5
NetBox version:  Netbox 2.4.0

Proposed Functionality
Hi,
When deleting user, all changes done by him are disappearing. It will be nice that there will be function that will inherit that changes to any other user.
Thanks beforehand.

Changes cannot be reassigned to other users. If you need to retain the change history for a user, disable the account instead of deleting it.


Add "uncertified PPE okay?" yes/no box

Some suppliers are wondering if hospitals are open to PPE from non-standard hospital supply chains.
These folks may lack FDA certification but are producing almost-equivalent products (think N95 for painters vs. for medical use). The FDA did relax N95 certification requirements a few weeks ago to allow 3M to use all of their lines to produce medical PPE, but this doesn't extend to other items like gowns, booties, gloves, etc.
Some of our hospitals explicitly state that they ARE looking for industrial masks in addition to medical N95's. They probably would be fine with other non-certified items like hairnets or booties.
If we can add an "Okay with non-FDA certified PPE?" yes/no, the suppliers will have clarity about what hospitals want, and the hospitals can open up the potential sources of donors.

Uncertified PPE apparently comes in different kinds. "origami" , Wisconsin, etc.  Could leave a response field open for them?

Added masks in #340, including homemade option.
I suggest we be very specific about what uncertified PPE hospitals would accept. For example, N95 masks for construction might be OK, but booties or gowns for other efforts, like manufacturing, might not. I saw guidance from the CDC on different masks being acceptable, but not for other PPE.
@cchang-adi  - is there a directive or some other FDA/CDC guidance advising hospitals to use other PPE?

@cchang-adi @4kbt - is this ticket still valid?

Yes, I just shared it again in our new form design flow thread.
It's actually even more relevant now than before, since the PN95 intake by default does not mention homemade PPE, and the makers are actually becoming more important as a domestic source, since imports are pretty messed up right now.


first-timers-only: Add test to make sure resultNoMatch callbacks returns possible sentences

first-timers-only
This issue is tagged :octocat: first-timers-only. It is only for people who have never contributed to open source before, and are looking for an easy way take their first steps.
Thank you for your help :heart:
Background
Callbacks added using annyang.addCallback('resultMatch', aFunction) will be called when a sentence a user said does match any of the currently registered commands (see docs).
The callback function should be called with an array of the possible sentences the user may have said as the first argument.
The Issue
The test specs do not currently test to verify that the callback function is called with the possible sentences the user may have said
Add a test spec that verify this spec:
  it('should call the callback with the the first argument containing an array of all possible Speech Recognition Alternatives the user may have said', func1);
}

This should be added into the following test suite:
describe("annyang.addCallback('resultNoMatch')", function() {

The last test described inside describe("annyang.addCallback('resultMatch')", function() {…}); in test/spec/BasicSpec.js, can be used as a template for adding the new spec described above.
Contributing to annyang

[ ] Fork the repository from the annyang GitHub page.
[ ] Clone a copy to your local machine with $ git clone git@github.com:YOUR-GITHUB-USER-NAME/annyang.git
[ ] Make sure you have node.js and npm installed on your machine. You can use this guide for help.
[ ] Install all of annyang's development dependencies with npm. $ cd annyang; npm install
[ ] Run grunt to make sure everything runs smoothly $ grunt
[ ] Code, code, code.
[ ] Run $ grunt after making changes to verify that everything still works and run the build process. Alternatively, you can just run $ grunt watch once, and leave that process running. It will continuously run all the tests and build the files every time you make a change to one of annyang's files. It will even beep if you make an error, and help you debug it.:+1:
[ ] Before committing your changes, the last step must always be running $ grunt. This makes sure everything works, and all files are kept up to date with your changes.
[ ] Once you've made sure all your changes work correctly, and committed all your changes, push your local changes back to github with $ git push -u origin master
[ ] Visit your fork on GitHub.com (https://github.com/YOUR-USER-NAME/annyang) and create a pull request for your changes.
[ ] Makes sure your pull request describes exactly what you changed and references this issue (just include the issue number in the title like this: #49)

About the tests
annyang is tested using Jasmine.
BasicSpec.js contains a series of spec groups (e.g. describe('a spec group', function() {})) which each contain 1 or more specs (e.g. it('should do stuff', function() {})). Some of the spec groups also contain some code which runs before each spec (beforeEach(function() {})).
To simulate Speech Recognition in the testing environment, annyang uses a mock object called Corti which mocks the browser's SpeechRecognition object. Corti also adds a number of utility functions to the SpeechRecognition object which simulate user actions (e.g. say('Hello there')), and allow checking the SpeechRecognition's status (e.g. isListening() === true).

Trying to tackle this issue now. I'm just wondering why annyang is adding the words "and so on" for each possible sentence.

Hi @timotius02 , thanks for pitching in! :fist:
That is actually added by Corti, the mock object annyang's tests use to simulate speech recognition in a testing environment.
Because you don't have support for Speech Recognition in the command line, or in the headless browser used by the tests, I created a mock object called Corti which mocks the browser's SpeechRecognition functionality.
Corti duplicates the behaviour of many of SpeechRecognition's features, as well as adding a number of utility functions to the SpeechRecognition object which simulate user actions (e.g. recognition.say('Hello there')), instead of speaking into the microphone and recognition.isListening() to check SpeechRecognition's status (instead of the blinking red dot in the browser.
Since recognition.say() doesn't actually use real Speech Recognition, I simulated different speech recognition alternatives by taking the original sentence and adding "and so on", "and so on" to each alternative.

Fixed in #169 by @timotius02
Thank you for the help! :fist:


Homepage https://www.fstar-lang.org/ broken

Papers are missing, menu items don't move you down the page.

Thanks for reporting, this should be fixed now.


spinner runner aborts when an empty directory is given

$ mkdir empty_dir
$ ruby spec/mspec/bin/mspec-run -B spec/default.mspec empty_dir
ruby 2.5.0dev (2017-08-02 trunk 59463) [universal.x86_64-darwin15]
Traceback (most recent call last):
	10: from spec/mspec/bin/mspec-run:7:in `<main>'
	 9: from spec/mspec/lib/mspec/utils/script.rb:265:in `main'
	 8: from spec/mspec/lib/mspec/commands/mspec-run.rb:83:in `run'
	 7: from spec/mspec/lib/mspec/runner/mspec.rb:51:in `process'
	 6: from spec/mspec/lib/mspec/runner/mspec.rb:96:in `actions'
	 5: from spec/mspec/lib/mspec/runner/mspec.rb:96:in `each'
	 4: from spec/mspec/lib/mspec/runner/mspec.rb:96:in `block in actions'
	 3: from spec/mspec/lib/mspec/runner/formatters/spinner.rb:84:in `start'
	 2: from spec/mspec/lib/mspec/runner/formatters/spinner.rb:62:in `compute_progress'
	 1: from spec/mspec/lib/mspec/runner/formatters/spinner.rb:54:in `compute_percentage'
spec/mspec/lib/mspec/runner/formatters/spinner.rb:54:in `/': divided by 0 (ZeroDivisionError)

MSpecScript#entries returns an empty array if partial is a directory name which has no spec files.
I'm uncertain where and how it should be dealt with.

abort in MSpecScript#files or MSpecScript#files_from_patterns when the list is empty.
same in MSpec.register_files.
same or ignore in MSpec.process.

abort if no files matched in MSpecScript#entries.

3cf0b6c56e70d89cd20071a7ad591f99dbad4c75
Author: Nobuyoshi Nakada<EMAIL_ADDRESS>Date:   Thu Aug 3 10:19:08 2017

    script.rb: skip empty directories

    * spec/mspec/lib/mspec/utils/script.rb (entries): skip empty
      directories so that at least one file would run.

1 file changed, 2 insertions(+), 1 deletion(-)
 spec/mspec/lib/mspec/utils/script.rb | 3 ++-

	Modified   spec/mspec/lib/mspec/utils/script.rb
diff --git a/spec/mspec/lib/mspec/utils/script.rb b/spec/mspec/lib/mspec/utils/script.rb
index 146e3367af..3ce3b2fd3b 100644
--- a/spec/mspec/lib/mspec/utils/script.rb
+++ b/spec/mspec/lib/mspec/utils/script.rb
@@ -188,7 +188,8 @@
       if File.file?(expanded) && expanded.end_with?('.rb')
         return [expanded]
       elsif File.directory?(expanded)
-        return Dir["#{expanded}/**/*_spec.rb"].sort
+        specs = Dir["#{expanded}/**/*_spec.rb"]
+        return specs.sort unless specs.empty?
       end
     end

Thank you for the bug report! I'll have a look this week.
Indeed if the set of files is empty it's OK to abort mspec, I think such a check already exists somewhere.

I applied your patch for MSpecScript#entries, thank you!


feat: add robots.txt server component

Description
Fixes #533

I think that would be good to add (TIL you can reference sitemaps from robots). The problem is that it needs to be a fully-qualified production URL, and we don't have access to that in Hydrogen. We have a similar problem with building SEO tags for canonical URLs, etc.
We should probably investigate a way to define these as at the config level or use an environmental variable. I'll open up a separate issue.

@jplhomer want to merge this as a starter template and I can add more functionalities after?


Fix parameter documentation inconsistencies

fix indentation - otherwise list items are rendered incorrectly
consistency: no spaces inside square brackets

Codecov Report

Merging #2584 into master will increase coverage by <.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #2584      +/-   ##
==========================================
+ Coverage    35.1%   35.11%   +<.01%
==========================================
  Files          79       79
  Lines        6973     6972       -1
  Branches      680      680
==========================================
  Hits         2448     2448
  Misses       4424     4424
+ Partials      101      100       -1

Impacted Files
Coverage Δ

src/metric/rank_metric.cc
90.81% <0%> (-0.55%)
:arrow_down:

src/c_api/c_api.cc
17.77% <0%> (-0.19%)
:arrow_down:

src/objective/regression_obj.cc
86.88% <0%> (+0.54%)
:arrow_up:

src/data/simple_dmatrix.cc
57.33% <0%> (+0.66%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2b3a431...e598d65. Read the comment docs.


duroc_hog should be recursive by default

Summary
duroc_hog should be recursive by default.
Desired Behavior
duroc_hog provides --recursive, which I would guess is what you almost always want. The default should be --recursive and it should provide --no-recursive, or whatever the standard inversion name format is.

finished


Wrong result for path rewriting with dot inside

For example,
/version/8.5.x/

will be recognized as a file.
This is a known issue.
This line of code should not use path.extname to test a path
https://github.com/vercel/serve-handler/blob/master/src/index.js#L608

shameless self-promotion alert:this feature is fixed in my @warren-bank/serve fork of serve


advice around https vs ssh for repo additions

Right now, I've got this working wonderfully if I add projects using https. But, I use bitbucket and the url is something of the form<EMAIL_ADDRESS>I'd like to force my development staff to use this if possible, but they'd all have to change the username in the .meta;
I did get it to work with ssh when I added the project with the format git@bitbucket.org:organization/repo.git but it was challenging to say the least and I had ssh whining at me for quite a bit.
Perhaps I'm just ignorant to an easier way. Perhaps I should just write a script that's inside my metarepo; But I'm very fond of just using meta git clone metarepourl to clone everything. If I had to script the updation of the .meta file to hot-swap the end-users username, it wouldn't work etc..
Perhaps you can point me to some best practice around using this in teams?
For what it's worth, I'd love to figure this out and help you document the add project module a bit more; It was hard to tell if I just entered the repo wrong, or it was an ssh problem etc...

I always try to use ssh, as https will often force you to enter a password when performing operations. So, you'll have to constantly re-enter your password. ssh will use your key pair, and avoid this.
That said, I had this same issue with a client who was using an internally hosted bitbucket. Your .meta file can actually be a js file, if you like. So, you can have
const username = process.env.USERNAME
module.exports = {
  "project"<EMAIL_ADDRESS>}

These guys have a similar tool but I like the simplicity of meta better: https://github.com/prezi/pride . However they do have a global config file in the user home directory where you can set up your base repo url like<EMAIL_ADDRESS>or https://bitbucket.org and their metadata file just has the suffix.   Might be an idea for dealing with this.   Surely we could use a bit of javascript code to handle that - but might be a nice core feature.

Looks like their tool is specific to projects that use gradle. I don't know why people prefer to bundle concerns with tools like this. pride and lerna would both probably better if they worked for any platform.

Yeah, I agree.. I just thought the idea was useful, how they store the repository path in a user config file whether it be ssh or https end point and that way different team members can choose their preference.

Sounds like more code to maintain, and an easy way to make there need to be a magic file your devs have to exchange to make things work. With meta and a single Makefile, your devs can completely onboard with a single command. Here's one I use for a current client: npm i -g meta && meta git clone git@github.com:my-client/my-client-meta.git && cd my-client-meta && make onboard. That will install all common tools the client's engineers use, clone all repos, npm install all repos, clone a QA database, and run everything in docker containers.  When you externalize a single file required for your repo to work, you now have one more external dependency for all that to work.

All you do there is  pride config repo.base.url git@bitbucket.org:whatever and it just drops a hidden config file in your $HOME.    And it uses that to prefix your repos, which lets devs set that whether they want to use ssh or https and can easily be chained into your single command.     I too would prefer everyone just use ssh.  I was just offering this idea I saw over there as a suggestion so you can leave it open for developers to choose which way to go and address the original question on the ticket.


Sozo CLI --version displaying more info

Creation of fn generate_version that gathers versions of cairo, scarb, sierra and sozo and rewrite the --version default behavior given by library clap, looking now very similar to the output of scarb --version.
closes #1696
closes DOJ-290

@glihm  Thank you very much for the guidance and the opportunity :)


Custom logo support

Adds custom logo support for WordPress 4.5. http://codex.wordpress.org/Theme_Logo

no longer making changes/updates to sage 8. we can consider this for sage 9, but we wouldn't support WP < 4.5 for this feature so the function_exists check isn't necessary
i'm also unsure about supporting this out of the box considering wordpress still doesn't support SVG uploads

Hey @retlehs - supporting this out of box does no real harm. Most people are totally fine with having logo as png (or worse - jpg) image. Those that need full svg support, can and will install plugin for svg support (i'd simply add one as a dependency in composer.json).

supporting this out of box does no real harm

agreed

Most people are totally fine with having logo as png (or worse - jpg) image.

stats on this statement? i believe most sage users are doing logos as SVG from the theme files based on what we see from new sites being shared on discourse

Those that need full svg support, can and will install plugin for svg support (i'd simply add one as a dependency in composer.json).

sure... but these users are most likely to not use the custom logo WP implementation and will continue to serve SVGs from the theme for logos.
this makes a lot more sense for people making themes to sell or distribute, which isn't our primary demographic.

stats on this statement? i believe most sage users are doing logos as SVG from the theme files based on what we see from new sites being shared on discourse

Most people as in people using themes, not developing them. I had my share of customers wanting to upload graphics, logos etc after buying theme. However, surely made-for-customer theme, with perfect branding etc, it makes more sense to blend logo right in theme.

sure... but these users are most likely to not use the custom logo WP implementation and will continue to serve SVGs from the theme for logos.

You mean developers, and I fully agree. I do that too.

this makes a lot more sense for people making themes to sell or distribute, which isn't our primary demographic.

Agreed. still - no harm in it simply being there, just to satisfy those wanting "all new Wordpress features". I mean, without function_exists wrapper this would be 2 lines additional in total (excluding possible fallbacks) and one could simply remove them if not needing them or add them if needing (if this or similar PR won't make it into sage 9).
Unrelated fun side-note: i'm arguing here about no-harm of feature i'd likely disable or wouldn't use for ~50% of themes i plan on doing once sage 9 is released... Or would use, but it won't be used by customer because fall-back will be "exactly what is needed". So... Only sake of this is "completeness" I guess?

Closing this.

Not making changes to Sage 8.
Fails CI.
Echoing a function that returns void.
Lack of interest.


Add uv export --format requirements.txt

Summary
The interface here is intentionally a bit more limited than uv pip compile, because we don't want requirements.txt to be a system of record -- it's just an export format. So, we don't write annotation comments (i.e., which dependency is requested from which), we don't allow writing extras, etc. It's just a flat list of requirements, with their markers and hashes.
Closes #6007.
Closes #6668.
Closes #6670.

Pretty cool, this will help migrate to uv.lock because users will have a way to revert easily if needed. 🎉


How to use Angular modules with differents paramters in same single page application

Hello everyone, I'm using AngularJS in my single page application, I have to use diiferents angular modules. I declared one module like that:
var app1 = angular.module('Myapp1',[]);
app1.controller('MainCtrl', function($scope) {
...}
And I want to use the second one with this paramter which is leaflet directive:
var app2 = angular.module('Myapp2', ['leaflet-directive']);
app2.controller('BasicCenterController', [ '$scope', function($scope) {
....}
But when I approach like that the map not appear, that's mean the seconf module didn't work.
Did you have any idea how to fix it?
Thankx for advance

On your HTML are you calling ng-app="app2" and ng-controller="BasicCenterController" ?

Hi, yes. I found a solution right now, I combined the two modules by adding this line: angular.module("CombineModule", ["Myapp1", "Myapp2"]). But thank you so much for your time :)


InvalidOperationException

Hi.
A new server-based project was created using this template.
When I start building and debugging (without any editing !),
the sample website appears fine and appears to be working as expected.
However, when I look at the output window of the integrated development environment, I see the following error messages
Exception thrown: 'System.InvalidOperationException' in Microsoft.AspNetCore.Components.Server.dll
Exception thrown: 'System.InvalidOperationException' in Microsoft.JSInterop.dll
Exception thrown: 'System.InvalidOperationException' in System.Private.CoreLib.dll
Exception thrown: 'System.InvalidOperationException' in System.Private.CoreLib.dll
Is this the behavior intended by the developers?

Yes, this is indeed. This is swallowed exception from the prerendering when JSInterop is not available:
https://github.com/MudBlazor/MudBlazor/blob/c4915b813bf6770dc5f0a114c392de410ce37cf8/src/MudBlazor/Services/IIJSRuntimeExtentions.cs#L67


Can I use this with SWIFT?

I tried to add this lib into my swift project via the bridging header but I don't know how to use it to create a string.
Using the syntax as described like this "Obfuscate.T.h.i.s..i.s..a..t.e.s.t.dot" does not work?
Many thanks!

Hey guys, check out this solution someone posted in another issue thread  - Obfuscate has been re-written in Swift here.

Hey @nchodelski. Did you know we've actually met.
I always check out everyone who stars my repos and knew we had met a few years back.

Hey there, thanks for the awesome ios tools you've built!  I'm  still figuring out if i can use Obfuscate in my in app purchase code (app is entirely swift atm) but it looks really useful. :)
Did we meet somewhere in Australia?  I used to live in Melbourne.

Yes in Melb. Long story though.
Of course you can use the obfuscator for ANY strings.
The most important thing about in app purchase is to do receipt validation. Most people don't do that.

I'm doing receipt validation but I dont have access to a 3rd party server so I'm trying to check a variety of hardcoded & obfuscated strings in addition to validating with Apple, to add more security to the validation process.

You don't need 3rd party server (although it is recommended). You can get app to do that in the same way the server does it.

You do know you can now use swift for server programming? $5 digital ocean server (if you are just doing receipt validation it may be enough).
Some good frameworks are perfect.org or vapour

I did look at local/ app side validation with some cocoapods and Open SSL
but the scale to set up was beyond the budget for my project.  Making my
own server is a good idea though. I have heard a bit about Perfect...might
try setting it up on a Linux box at home... it would be a good way to learn
about server side swift.
On 13 May 2016 at 17:33, pjebs<EMAIL_ADDRESS>wrote:

You do know you can now use swift for server programming? $5 digital ocean
server (if you are just doing receipt validation it may be enough).
Some good frameworks are perfect.org or vapour
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/UrbanApps/UAObfuscatedString/issues/15#issuecomment-219189188

When did you start doing apps. When I met you you were focussing on ants

App side validation with cocoapods + openssl + beyond budget.  I think you were doing it wrong

Those keywords suggest that

Hello @Foddy
Yes, you can use this library with Swift. Please close the issue.

Oh! In the meantime all my "secret" keys have been leaked across the internet... 😅
(kidding, just in case a client jumps into that issue)


Start on windows without window

rechearing...


Updates version of Torque to 2.11.2

Changes since last replacement:
- Added error handling to Torque
- Fixed 'remove' event triggering when removing a Torque layer.
- Value is now capped at 255 instead of performing modulo on higher values.

@javierarce ;)

:flags:


Offset based pagination component

In a conversation with @johnlcox we have discussed a possibility of adding a common offset based pagination component to the beadledom. The component would decorate a list of objects returned from target methods with all necessary pagination elements.
Using John’s implementation of pagination as part of one of the Cerner projects as a model the component would include:

a subclass of WriterInterceptor applied to target methods with a specific return type (e.g.
OffsetPaginatedList)
a response DTO decorated by the interceptor with:
Items – list of items of type T
totalResults – total number of results across all the pages
hasMore - an indication wheter there are additional items on another page
firstLink – link to the first page
lastLink – link to the last page
nextLink – link to a page after current (null if current is the last page)
prevLink – link to a page before current (null if current is the first page)

I'm opening this issue to discuss design of the solution and feedback would be appreciated.

I think the hasMore field is not needed on the actual response object since that information can be derived by looking for a nextLink. The hasMore field should be present on the internal type that will be intercepted though since it can be used to determine the necessary links when total results may not be known.
lastLink should also be nullable since a last link can only exist in the case that total results are known.
One thing that needs to be decided is how to handle default values for the limit query parameter since that will be used for generating links when an explicit limit value is not set on a request.

Services are already duplicating the limit values between @DefaultValue and the swagger docs. The @DefaultValue is especially troublesome because it requires a String even though the limit param is usually a long or an int.
Since the value is already being duplicated it might be best to allow the WriterInterceptor to be injected with a value for the default pagination limit. Then each service could bind that limit to whatever they want, but make sure that it matches the swagger and JAX-RS values. Since it is a primitive type (long) it would need to be bound using a binding annotation.

@DefaultValue is used by both jaxrs and swagger so I am not sure there is duplication.  If we were able to, via ResourceInfo, read the DefaultValue annotation value, we could avoid the need for injection of the value via a binding annotation.
The only limitation would be that the field would need to be named "limit" so that we could find the DefaultValue.
The other option would be to create a named binding that would be applied to the parameter that is the limit and then that parameter could have any arbitrary name but that feels like overkill.

Is there an example that could be linked on how ResourceInfo could be used? Could an instance of ResourceInfo be injected into a WriterInterceptor via the @Contexst of the request or did you have something else in mind?

I was able to inject it using the @Context and the ResourceInfo has a way to get to the parameters and the parameter annotations.
Annotation[][] resourceInfo1 = resourceInfo.getResourceMethod().getParameterAnnotations();
When I evaluated that it shows the following:

We could then move through all of the parameters and find the one called "limit" and use the "DefaultValue" annotation to set the default limit. The only thing that would need to be looked into beyond that is...What do we do if they don't create a default value via the annotation.  Still have some research to do to verify we can tie them all together but I thought it was very promising.

That's pretty sweet that ResourceInfo can be injected using @Context. I looked in both the Resteasy and Jersey projects on github and it looks like both of them support injecting it, so we should be able to build this to be JAX-RS implementation agnostic.
I'm not sure we want to look for a parameter called "limit" as the means for finding the right default value. I can imagine other services, that might used beadledom, wanting to use "limit" for something besides pagination.
Having a custom annotation for the pagination limit parameter is probably the safer approach. We could also have a fallback default that is used if the @DefaultValue parameter is not present. However, this may also require a custom annotation for the offset query parameter as well since we will need to be able to look up the actual query param values, if set, when generating the pagination links.
I'm not sure I'm a huge fan of needing two additional query params for any method that will use pagination. If we instead did want to use "limit" and "offset" I think we would have to provide a way to configure what those values are rather than having them hard coded. We could have "limit" and "offset" as defaults, but I think we are getting back to needing to inject those values somewhere.

So here's what I was thinking about...

Create two custom name bindings one for limit and one for offset which would serve as markers within the method that indicated which parameter was the limit and the offset.
Then in the intercepter we can use the resourceinfo to find the markers and use them to resolve the parameter names as well as get the default values if also annotated.
If the markers are not present in any of the query parameters, then we will look for "offset" and "limit" and use those.  This way they would only mark the parameters if they wished to have a parameter name other than "limit" or "offset".
If the default value is not set via the annotations in the method then we can have a hard coded value behind the scenes.

I will be in big room planning on Monday but if we agree to continue down this path I will attempt to prioritize it for completion at the beginning of Q4.

Something like this:
{code:java}
@ApiParam(value = "Paging parameter to retrieve specialty list starting at the given index."
+ " Must be greater than or equal to 0, default value of 0", allowableValues = "range[0, Infinity]")
@Offset @DefaultValue("0") @QueryParam("offset") Integer offset,
@ApiParam(
value = "Paging parameter to limit the number of returned specialty list" +
" Must be greater than or equal to 1 with a maximum value of 100, "
+ "default value of 20",
allowableValues = "range[1, 100]")
@Limit @DefaultValue("20") @QueryParam("limit") Integer limit {code}

I'm still not sure I like needing to add those two annotations everywhere.
Is there anyway with JAX-RS to have those be used in place of @QueryParam and @DefaultValue? If we could combine those two into a single annotation for each pagination parameter that would be pretty amazing.

If the markers are not present in any of the query parameters, then we will look for "offset" and "limit" and use those. This way they would only mark the parameters if they wished to have a parameter name other than "limit" or "offset".

We would also need to provide a mechanism to override the expected query parameter names.

I have been able to create a common pagination implementation based on your project, added the stuff we talked about and was feeling pretty good about it - however, I ran into an issue in the beadledom based java client.
The issue has to do with return type change in the interceptor. com.cerner.beadledom.client.proxy.GenericClientProxy uses reflection when determining how to deserialize the response json.
From the method signature it is not getting OffsetPaginatedListDto rather it gets OffsetPaginatedList and therefore loses any fields that are not in the latter:
ParameterizedType genericReturnType = (ParameterizedType)method.getGenericReturnType();
Type bodyType = genericReturnType.getActualTypeArguments()[0];
entity = response.readEntity(new GenericType(bodyType));

Considering that our endpoints are all returning a response of type GenericResponse we have to come up with some other way of determining the response type for response deserialization.
Perhaps we can create an interface that OffsetPaginatedList would implement. We will then be able to use this interface to control the flow in GenericClientProxy::invoke and return an Object or have a field in this interface defining a return type. In either case it appears that we will need to make changes to GenericClientProxy to allow for response type change in the interceptor.

With the service we have been building we have started coming to the conclusion that it's not worth it to share a JAX-RS interface between the client and server.

Pagination doesn't work as you mentioned
PATCH will require a very different method signature on the server-side than the client-side
Adding additional query params on the server-side is totally passive if they have a default, but is breaking for clients if an interface is shared

OK.  We will have to put the pagination stuff on hold then until the client changes are in place and then re-visit it.

Hey @johnlcox and @elliott311, I've been working on a pagination implementation based largely off the work that @johnlcox had done with the addition of a BeanParam for easily adding and identifying the query params used for pagination. I also added some mechanisms to allow for configurable default limit/offset values as well as configurable field names. I've consumed the module in an internal project and it seems to be working pretty well, but I had a question about configurable field names within the BeanParam. Right now the BeanParam will always use "offset" and "limit" for the query parameter names since it is using the @QueryParam annotation to name the param. Because of this, if a consumer were to want to use a different field name, they would not be able to use the BeanParam in an intuitive way (and subsequently the convenience methods that come with it to use the configured default field values). I was thinking about using reflection at injection time to modify the values of the @QueryParam annotations, but wasn't sure if this was the best strategy to go with or even worth the overhead for such a tiny convenience. What approach do you guys think would be the best? If you want to see what I've added, my feature branch is here.

@lulzWill Does your implementation allow for endpoints to have different defaults for limit? For example, one endpoint have a default limit of 100, and another have a limit of 1000?

Would using a custom ParamConverter instead of a BeanParam make it possible to customize the parameter names by injecting the names into the converter?

@shiftyeyes you can do this if you do not use the bean param. You would just have to populate the corresponding values within the list metadata object. The list metadata values will take precedence over the injected values if they are provided so you are able to have endpoints with varying default values as well as default field names.
@johnlcox I'm not very familiar with using ParamConverter, but I'll look into it and check back. From the quick glance I did of the documentation, it looks promising though.

@johnlcox got tied up working on some other stuff the past couple weeks, but got a chance to dive further into using the ParamConverter today. Unless I'm missing something, it looks like it would have the same issue as the bean param since both are just using the @QueryParam annotation to specify the field name. I'm gonna play around with the reflection approach a bit and see if I can come up with something that isn't really ugly.

I took up @lulzWill work and added some tests/documentation and put out a PR. Some future work related to pagination that was explicitly not covered by that PR is:

Cursor based pagination - it can be added at a later date to the same module
pagination is not compatible with generated clients - as discussed above the server representation of pagination is not directly compatible with the client representation. To work around this consumers can create an interface specifically for client consumption. To see an example of this in action, see the integration tests in the PR


How to get an instance inside Grape::API class

Since #1803 Grape:API instance stopped presenting the data inside.
Previously we used parent_name in order to rewrite standard errors classes to change errors  format.
With grape 1.1.0
api = @scope.instance_variable_get(:@api)
module_name = api.parent.name.humanize.demodulize
So, since i've update grape to 1.2.0+ it stopped to work
How can i get the data now?
I will appreciate any help, thanks :)

Hi, as explained in the Upgrading guide: https://github.com/ruby-grape/grape/blob/master/UPGRADING.md
We've deprecated name and you should now be using .to_s or base.name - doesn't this work for you?

I'm not sure what do you have in @scope
I think you should try us

Yes, i missed that. Well, it works pretty well.
I was looking for smth like this in specs, while looking for solution.
@myxoh Is there any spec describing getting access to the named class from the instance?
P.S. with the answer feel free to close issue, it solved, thanks

I'll close this.
I there's no spec that shows this, appreciate a PR!


Credential Caching for the process-creds command

Very much a work in progress, creating PR now for visibility.

My question: can we move a lot of the logic outside the cmd package? Or explain (in the README.md) what the command is & how to use it.

My question: can we move a lot of the logic outside the cmd package? Or explain (in the README.md) what the command is & how to use it.

I agree! I think the plan is actually to move a lot of this logic outside of aws-oidc altogether (ie into go-misc). These changes to alter the command or how it's used at all, just provide transparent caching behind the scenes.


open_mfdataset does not handle single pathlib.Path objects but handles single str just fine

What happened:
An exception was raised
What you expected to happen:
That a dataset object would be created.
Minimal Complete Verifiable Example:
from pathlib import Path

import xarray as xr

dataset = "xarray/tests/data/example_1.nc"

with xr.open_mfdataset(dataset) as actual:
    assert isinstance(actual, xr.Dataset)

with xr.open_mfdataset(Path(dataset)) as actual:
    assert isinstance(actual, xr.Dataset)  # this fails

with the following traceback
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/panos/Prog/git/xarray/xarray/backends/api.py", line 869, in open_mfdataset
    paths = [str(p) if isinstance(p, Path) else p for p in paths]
TypeError: 'PosixPath' object is not iterable

Anything else we need to know?:
This seems to be fixing it, but I am not familiar with the codebase to know if this is the proper way to fix it
diff --git a/xarray/backends/api.py b/xarray/backends/api.py
index 6d73946b..596ac249 100644
--- a/xarray/backends/api.py
+++ b/xarray/backends/api.py
@@ -838,6 +838,8 @@ def open_mfdataset(
     .. [1] http://xarray.pydata.org/en/stable/dask.html
     ..
I would probably just fix it with
diff --git a/xarray/backends/api.py b/xarray/backends/api.py
index 6d73946b..00619d68 100644
--- a/xarray/backends/api.py
+++ b/xarray/backends/api.py
@@ -865,6 +865,8 @@ def open_mfdataset(
             )
         else:
             paths = sorted(glob(_normalize_path(paths)))
+    elif isinstance(paths, Path):
+        paths = [str(paths)]
     else:
         paths = [str(p) if isinstance(p, Path) else p for p in paths]

to emulate the list comprehension we already have.
Would you be interested in submitting this little fix as a PR? Or would you rather I did it?


Failed to install sbf-tools: HTTP status client error (404 Not Found)

I'm trying to build my rust program, but I'm getting this. The 1.32 release is removed and the link is dead (404).
I'm using the latest version of solana-program in my cargo toml file.
cargo build-bpf --manifest-path=./src/program-rust/Cargo.toml --bpf-out-dir=dist/program

[2023-02-18T11:36:12.267149000Z ERROR cargo_build_sbf] Failed to install sbf-tools: HTTP status client error (404 Not Found) for url (https://github.com/solana-labs/sbf-tools/releases/download/v1.32/solana-bpf-tools-osx.tar.bz2)

Here's a snippet from my cargo toml
[dependencies]
borsh = "0.10.2"
borsh-derive = "0.10.2"
solana-program = "1.15.2"

[dev-dependencies]
solana-program-test = "1.15.2"
solana-sdk = "1.15.2"

Same outcome with tests on macOS and WSL2.

Same issue here. On MacOS M1 with latest release of solana.

Please use --tools-version v1.30 command line option with cargo build-bpf command. The later versions of sbf-tools are currently broken.

I am getting the same error when I try to build my anchor app with anchor build
[2023-02-18T23:24:01.364357000Z ERROR cargo_build_sbf]
Failed to install sbf-tools:HTTP status client error (404 Not Found)
for url (https://github.com/solana-labs/sbf-tools/releases/download/v1.32/solana-bpf-tools-osx.tar.bz2)

The releases are restored.


double prompt from command line

openssl apps defines own UI_method.
In ui_write(apps.c) fail-back to "reader" instead to writer.

Sorry - master branch only.

Addressed in #4198


only run github ci checks once

📝 Summary
Before they were always run twice (once for push and once for PR). This PR makes the checks run only once.

✅ I have run these commands

[x] make lint
[x] make test-race
[x] go mod tidy

Codecov Report

Merging #494 (3d9f29c) into main (9acaf56) will increase coverage by 0.23%.
The diff coverage is n/a.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@            Coverage Diff             @@
##             main     #494      +/-   ##
==========================================
+ Coverage   68.79%   69.03%   +0.23%
==========================================
  Files           8        8
  Lines        1269     1269
==========================================
+ Hits          873      876       +3
+ Misses        346      345       -1
+ Partials       50       48       -2

Flag
Coverage Δ

unittests
69.03% <ø> (+0.23%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.
see 1 file with indirect coverage changes
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Though, are you sure this is the desired behavior? Without this, I don't believe it will run CI checks for PRs. And I think it's pretty useful to see the CI checks for in PRs.

yeah if anything I'd rather enforce the checks at the PR, then not bother with running them "twice" on the main branch
that being said I am also curious why we need to change from the status quo in the first place?

We need to change from status quo because in the current configuration, checks are only run for PRs to the main branch, but they should be run on any PR (i.e. you open a PR to another PR/another branch).
The new configuration is correct -- run checks on every PR, and on any push to main branch too

Okay, sounds good to me 👍


Add Viper LLL descriptions

Added how many arguments Viper's LLL takes and descriptions for Viper's LLL

@DavidKnott thanks! Did some minor updates, merging it now.


Release 1.2

This PR is not to be merged until the release of cert-manager 1.2

/override dco

/hold
/override dco

Do we still need to /hold this one?

/override dco

/lgtm

/hold cancel


Wrong nullable fields

SafeQl recognizes fields as nullable when inner joining with a materialized view:

The View:
CREATE MATERIALIZED VIEW issues_base AS
    SELECT issue.id AS issue_id,
        issue.cve_code,
        image.name AS image_name,
        image.id AS image_id,
        jira_issue_key,
        array_agg(issue_instance.id) AS issue_instance_ids,
        client_id,
        issue.ignored,
        bool_or(
            CASE
                WHEN (is_native)
                    THEN state = '2'
                ELSE
                    last_executed_at IS NOT NULL
            END
        ) AS is_executed
  FROM issue,
      image,
      issue_instance,
      client_library_instance,
      live_k8s_image_builds
  WHERE image.id = issue.image_id
  AND image.id = live_k8s_image_builds.image_id
  AND issue_instance.issue_id = issue.id
  AND issue_instance.image_build_id = live_k8s_image_builds.id
  AND issue_instance.client_library_instance_id = client_library_instance.id
  AND issue.resolved IS NOT TRUE
  GROUP BY issue.id, image.id;

The query:
          SELECT issue_id AS "issueId",
              cve_code AS "cveCode",
              image_name AS "imageName",
              image_id AS "imageId",
              jira_issue_key AS "jiraIssueKey",
              issue_instance_ids AS "issueInstanceIds",
              ignored
          FROM issues_base
          WHERE client_id = ${clientId};

TL;DR - SafeQL (currently) doesn't support nullability checks for SQL views.
SafeQL's approach is "everything is nullable until proven otherwise" (for safety reasons). It infers whether a target result (e.g., selected column) is nullable or not, by first analyzing what it is. If it's a computation, it's not null for sure. If it's a table column, it's not null as long as that table column is NOT NULL as well, and so on.
Unlike querying columns from tables, PostgresSQL doesn't know that the column cve_code that was called from view issues_base is from table issue.  SafeQL already traverses throw the queries for more precise checks, but currently it doesn't traverse over the views to determine nullability checks.
As a workaround for now, SafeQL is smart enough to know that a column is not nullable if you do one of the following options:

add WHERE issue_id IS NOT NULL
add a COALESCE with a fallback value.

Thank you!


fix cv bug

#268

Hi @wxchan, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
The agreement was validated by Microsoft and real humans are currently evaluating your PR.
TTYL, MSBOT;

@guolinke What is the train_set argument in booster.update() for? Seem never used?

@wxchan , used for changing training data during training.
Not used in high level interface now.

Ok, got it. Users can call booster.update themselves.


vector_to_regular_points: Drop or duplicate vector properties for independent points?

We recently added a "group" parameter to vector_to_regular_points so that either MultiPoints or individual points are generated.
For MultiPoints it's easy to keep vector properties.
For Points you'd need to duplicate the vector properties. Is that intended? Right now it's documented to drop the properties. The program would also need to generate new IDs then.
Origin: https://github.com/Open-EO/openeo-processes/pull/315#commitcomment-68730976 and https://github.com/Open-EO/openeo-processes/pull/315#commitcomment-68731148

What a user probably wants is to keep properties like landcover class, scene classification, etc
What an implementer probably wants to drop/replace is a property that acts as unique identifier.
What might help is to extend the vector cube definition to have a dedicated identifier property (a unique identifier associated with each geometry).  At load time it can be populated from a user specified property, or, if that is missing, autogenerated (e.g. autoincrement int)
For the "ungrouped" version of "vector_to_regular_points" you can then specify:

the id property is dropped and fully replaced with new ids
all the other properties are kept/duplicated

This is actually a much broader discussion that we need to have, regardless of this process. Essentially, what do you do with the metadata that is not part of the data cube? This also applies for raster cubes, it's also basically undefined there.

I've changed to "vector properties are generally preserved", but for MultiPoint the ID is also preserved while for Points new IDs must be assigned.

I do not see why the ID must be different from other properties, in my opinion we should just duplicate all of them. Or we need to define it as a mandatory field of a vector-cube.
The back-end can still have another indexing column which can be kept internally for working with the vector-cubes.
I've added a comment here https://github.com/Open-EO/openeo-processes/pull/352/files

Can't see the comment in the PR, but the ID must change for points because otherwise, it's not unique anymore. I assume IDs should be always unique?

But the question is, does a feature/geometry always need to have an unique ID? If yes, I've missed this.
The geoJSON samples that I've sent around some time ago didn't have any ID field but only geometries plus some properties.

No, indeed, an ID is likely not required.


payer_id is not pulled along with subscriber details when looking for a subscription instance

I'd like to use the subscription's payer_id, and it comes back in the JSON response, but it's not actually put into the returned object:
thesub = PayPal::SDK::Subscriptions::Subscription.find("I-UF77YVRH12WE")
thesub.subscriber.name.given_name # joe
thesub.subscriber.payer_id # undefined method

I'll try to work up a PR.

fixed by #2

#2 manually merged.


[WIP] feature: update TFlint #2418

Description
Updates TFlint to v0.44.1
Fixes #2410 #2418 #2420

TODOs
Read the Gruntwork contribution guidelines.

[x] Update the docs.
[ ] Run the relevant tests successfully, including pre-commit checks.
[x] Include release notes. If this PR is backward incompatible, include a migration guide.

Release Notes (draft)

Updated TFlint to v0.44.1
Removed forced --module check for TFlint
Migration Guide
If you want to have the TFlint module check enabled, simply add module = true to your .tflint.config.hcl
config {
  module = true
}

Hi,
I'm not sure if this PR is ready, but it fails now to build because of issues with go.sum
7 errors occurred:
--> linux/amd64 error: exit status 1
Stderr: ../go/pkg/mod/github.com/hashicorp/terraform@v0.15.3/lang/funcs/crypto.go:21:2: missing go.sum entry for module providing package golang.org/x/crypto/bcrypt (imported by github.com/hashicorp/terraform/lang/funcs); to add:
	go get<EMAIL_ADDRESS>../go/pkg/mod/filippo.io/age@v1.0.0-beta7/internal/stream/stream.go:15:2: missing go.sum entry for module providing package golang.org/x/crypto/chacha20poly1305 (imported by filippo.io/age); to add:
	go get<EMAIL_ADDRESS>../go/pkg/mod/filippo.io/age@v1.0.0-beta7/x25519.go:20:2: missing go.sum entry for module providing package golang.org/x/crypto/curve25519 (imported by filippo.io/age); to add:
	go get<EMAIL_ADDRESS>../go/pkg/mod/gopkg.in/square/go-jose.v2@v2.5.1/asymmetric.go:31:2: missing go.sum entry for module providing package golang.org/x/crypto/ed25519 (imported by gopkg.in/square/go-jose.v2); to add:
	go get<EMAIL_ADDRESS>../go/pkg/mod/filippo.io/age@v1.0.0-beta7/primitives.go:17:2: missing go.sum entry for module providing package golang.org/x/crypto/hkdf (imported by filippo.io/age); to add:
	go get<EMAIL_ADDRESS>../go/pkg/mod/go.mozilla.org/sops/v3@v3.7.2/pgp/keysource.go:25:2: missing go.sum entry for module providing package golang.org/x/crypto/openpgp (imported by github.com/terraform-linters/tflint/plugin); to add:
	go get<EMAIL_ADDRESS>../go/pkg/mod/go.mozilla.org/sops/v3@v3.7.2/pgp/keysource.go:26:2: missing go.sum entry for module providing package golang.org/x/crypto/openpgp/armor (imported by go.mozilla.org/sops/v3/pgp); to add:
	go get<EMAIL_ADDRESS>../go/pkg/mod/gopkg.in/square/go-jose.v2@v2.5.1/symmetric.go:33:2: missing go.sum entry for module providing package golang.org/x/crypto/pbkdf2 (imported by gopkg.in/square/go-jose.v2); to add:
	go get<EMAIL_ADDRESS>../go/pkg/mod/github.com/!azure/go-autorest/autorest/adal@v0.9.13/persist.go:27:2: missing go.sum entry for module providing package golang.org/x/crypto/pkcs12 (imported by github.com/Azure/go-autorest/autorest/adal); to add:
	go get<EMAIL_ADDRESS>../go/pkg/mod/filippo.io/age@v1.0.0-beta7/internal/stream/stream.go:16:2: missing go.sum entry for module providing package golang.org/x/crypto/poly1305 (imported by filippo.io/age/internal/stream); to add:
	go get<EMAIL_ADDRESS>../go/pkg/mod/filippo.io/age@v1.0.0-beta7/scrypt.go:17:2: missing go.sum entry for module providing package golang.org/x/crypto/scrypt (imported by filippo.io/age); to add:
	go get<EMAIL_ADDRESS>../go/pkg/mod/github.com/hashicorp/terraform@v0.15.3/lang/funcs/crypto.go:22:2: missing go.sum entry for module providing package golang.org/x/crypto/ssh (imported by github.com/hashicorp/terraform/lang/funcs); to add:

Hi @denis256. Thanks for the hint! I ran a god mod tidy and pushed the clean go.sum - let's see how it goes.
Guess I resolved the merge conflict wrong yesterday :-/
Besides that, the PR is ready from my side  - I will mark it as ready if the tests pass successfully.

I am fixing the failing test. Deleted too much content in the tf files. -.-'

--- PASS: TestTflintFindsNoIssuesWithValidCode (2.33s)
=== RUN   TestTflintFindsModule
    integration_test.go:3882: Copying fixture-tflint/module-found to /tmp/terragrunt-test3238892953
runTerragruntVersionCommand after split
[terragrunt plan --terragrunt-working-dir /tmp/terragrunt-test3238892953/fixture-tflint/module-found]
    integration_test.go:5254:
        	Error Trace:	integration_test.go:5254
        	Error:      	Received unexpected error:
        	            	1 error occurred:
        	            		* Tflint found issues in the project. Check for the tflint logs.

        	Test:       	TestTflintFindsModule
--- FAIL: TestTflintFindsModule (4.88s)

[INFO] Initializing environment for https://github.com/gruntwork-io/pre-commit.
Terraform fmt............................................................Failed
- hook id: terraform-fmt
- files were modified by this hook

main.tf

build and testis green 🥳

https://github.com/gruntwork-io/terragrunt/releases/tag/v0.43.0


cant see container full log on argo UI

On version v2.6.0-rc3 UI  when i enter the log of one of the containers in the workflow i cant see the full log (i can see only last logs like tail -f)

Fixed in https://github.com/argoproj/argo/pull/2330


add mini player on mobile

Similar to Android app
bonus:
some simple controls, or at least an X to close?

the cordova branch has the pause + x, this would be great to get in at least for regular videos until player is ready.


Support Platform View creation

https://docs.flutter.dev/platform-integration/ios/platform-views
I have to write a plugin to wrap a native library and, for reasons not easily avoidable, that includes writing a Platform View. It would be nice if I was able to do that in Klutter. AIUI it's mainly a matter of exposing the FlutterPlatformView and FlutterPlatformViewFactory classes in the Kotlin API? And then more importantly, automagically handling registration. Of course if you want to make it even fancier and generate the factory too, nobody will complain.
Caveats: I believe platform views only exist for Android and iOS. Not sure what Klutter would do in this case when building for other platforms.

I'm not sure it's even possible, just throwing it out there...

Interesting idea. Everything is possible of course. The main thing would be to expose FlutterPlatformView(Factory) in Kotlin. I'm not sure if that's doable. I had a lot of issues wrapping Event-channel Flutter code in Kotlin Multiplatform. To make it work is one thing but to maintain it is another.


AP-2227 Chances of success for Multiple Proceedings

AP-2227 Chances of success for Multiple Proceedings
Link to story
LegalAidApplication model changes

Add #lead_application_proceeding_type method
Add a has_many :chances_of_success, through: :application_proceeding_types
Delegate #chances_of_success to the above  #lead_application_proceeding_type
Add merits_complete! method to update the merits_submitted_at attribute . This is used when the merits have been submitted for the application.

Update ChancesOfSuccess controller and views to take in an application_proceeding_type instead of legal_aid_application

Update success_prospects and proceedings_before_the_courts controller and view to follow the above because they use the same ChancesOfSuccess model which has been moved under the ApplicationProceedingType

Update calculation_date_service to use the merits_submitted_at from the LegalAidApplication model

reports/mis/application_detail_csv_line.rb has the merits_submitted_at under the chances_of_success, which isn't accurate but it can be done in a following ticket e.g. to update the excel sheet and test it.

Add tests.

Update broken tests.

Database changes:

Add migration to update ChancesOfSuccess with a reference to ApplicationProceedingType

Update ChancesOfSuccess table to not require a legal aid application id on creation, this means we don't have to get
rid of the field for now.

Add a rake task to transfer existing associations with LegalAidApplication to ApplicationProceedingType for the ChancesOfSuccess

Add #merits_submitted_at field to LegalAidApplication . This takes over from ChancesOfSuccess#submitted_at

Add a rake task to transfer ChancesOfSuccess#submitted_at to LegalAidApplication#merits_submitted_at

Checklist
Before you ask people to review this PR:

[ ] Tests and rubocop should be passing: bundle exec rake
[ ] Github should not be reporting conflicts; you should have recently run git rebase master.
[ ] There should be no unnecessary whitespace changes. These make diffs harder to read and conflicts more likely.
[ ] The PR description should say what you changed and why, with a link to the JIRA story.
[ ] You should have looked at the diff against master and ensured that nothing unexpected is included in your changes.
[ ] You should have checked that the commit messages say why the change was made.

Codecov Report

Merging #2492 (e59fef1) into master (e5af328) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##            master     #2492   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files          506       506
  Lines         9869      9896   +27
=========================================
+ Hits          9869      9896   +27

Impacted Files
Coverage Δ

app/services/ccms/requestors/case_add_requestor.rb
100.00% <ø> (ø)

app/services/reports/merits_report_creator.rb
100.00% <ø> (ø)

...llers/providers/check_merits_answers_controller.rb
100.00% <100.00%> (ø)

...controllers/providers/merits_reports_controller.rb
100.00% <100.00%> (ø)

...eding_merits_task/chances_of_success_controller.rb
100.00% <100.00%> (ø)

...eeding_merits_task/success_prospects_controller.rb
100.00% <100.00%> (ø)

...viders/proceedings_before_the_courts_controller.rb
100.00% <100.00%> (ø)

...ers/providers/submitted_applications_controller.rb
100.00% <100.00%> (ø)

..._of_successes/proceedings_before_the_court_form.rb
100.00% <100.00%> (ø)

app/models/application_proceeding_type.rb
100.00% <100.00%> (ø)

... and 10 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e5af328...e59fef1. Read the comment docs.


hello how to contribute ?

Hi Im a rust developer i would like to contribute on this project
thanks

Hello @marcpar and welcome! If you are interested in contributing, we have a number of issues labeled as "good first issue" (and more are coming in time). The descriptions are not as detailed as one might wish, but they should be good enough for you to know if you are interested, in which case we are ready to answer any questions you might have on our discord channel.
Moreover, if you are interested in working in Rust on SSI full-time, Absa, currently the main sponsor of this project, is hiring:
https://www.hyperledger.org/job/absa-bank-czechia-or-eu-region-144-backend-software-engineer-rust
https://www.hyperledger.org/job/absa-bank-czechia-or-eu-region-144-fullstack-software-engineer-javascript-typescript
https://www.hyperledger.org/job/absa-bank-czechia-or-eu-region-144-backend-software-engineer-node-js
https://www.hyperledger.org/job/absa-bank-czechia-or-eu-region-144-frontend-software-engineer-react
https://www.hyperledger.org/job/absa-bank-czechia-or-eu-region-144-mobile-software-engineer-react-native


Request: Add support for non-buffered hash functions that take array inputs

Some hash functions use different constants depending upon the number of rounds (input elements). The current behaviour when hashing pairs of nodes is to concatenate the elements together before hashing them. This makes sense for hashing algorithms like SHA-2 where the hasher implementation takes care of decomposing the input into blocks of a fixed length, doing padding, etc.
For other hashing algorithms like Poseidon, which is used extensively in zero knowledge circuits, when hashing multiple elements the behaviour of the hashing algorithm differs according to the number of elements specified. As such, concatenating the elements before supplying them to the hasher makes that infeasible.
Put in a slightly more complicated way, elliptic curve compression functions accept "field elements", not strings of binary data. Combining two or more field elements is not done by concatenating their binary representations.
A backwards-compatible solution for this would be to allow the "concatenator function" to be specified as an option, with the default being Buffer.concat. However, it seems like the use of Buffers as inputs to hash functions is a bit baked in, and will need some careful untangling.

Awesome. That's looking very good. I'll test this out in a likely scenario (e.g. with Semaphore), and will report back on any issues.
In a quick skim of the code, I do see one thing that may be an issue. In combineHashes, you do hashes = hashes.filter(x => x.byteLength > 0).
I cannot find a consistent description of the byteLength property of Buffer objects, and it seems to be 0 always in every isolated test I've run.
Additionally, it would be pretty normal to have elements of a hash which are 0. Removing empty elements from the input would be unexpected behaviour.

@TheFrozenFire ah yea you're right that shouldn't be there. The redundant combineHashes method has been removed now so it just calls this.concatenator(...) when passing to hash function.


"Linking... " formatting nitpick

When rebar3 does this and reports what it's linking:
===> Linking /home/vagrant/irccloud/irccloud/_build/default/lib/erlydtl to /home/vagrant/irccloud/irccloud/_build/dev/lib/erlydtl

IMO it would be sufficient, and preferable, to remove the rebar.config dir from the path it prints, for neatness:
===> Linking _build/default/lib/erlydtl to _build/dev/lib/erlydtl

All these long lines make the console output really messy and make it hard to see what's going on.
Would you accept a patch to that effect?

or perhaps just don't print the Linking stuff at all, it doesn't really help.

Definitely would accept the patch. And yea, what I'd like is to only print that when it is actually made, so we should check if the symlink exists and not print if it does.

thanks


Priority based requests queue

Describe the feature
Add a { priority: number } parameter when enqueueing a request. I imagine it to be lowest number means higher priority.
Motivation
Make sure items are pushed to the dataset as soon as possible.
I've got a use case where:

a run might take 6-12 hours if not a day
an item pushed to the dataset after a nested list of crawled links
items have expiring content
the actor's user reads the dataset during the run to have the lest freshly pushed items

Right now I've got the case where 80 to 100% of these items end up being pushe to the dataset in the last 10% of time of the run, because all links end up  treated as last.
Constraints
I believe there is no simple drop-in solution with the requestQueue/queueing system as no kind of prioritizing is being done so far so it would require to rethink a little bit the system.
Thoughts?

Itsn't forefront: true enough for this?
https://sdk.apify.com/docs/api/request-queue#requestqueueaddrequestrequestlike-options

Forgot to mention that I'm using forefront: true as well.
The thing is, the flow is as follows:

Enqueue 25 requests
They all start (as I'm not restricting concurrency to 1 for example)
Then they all push subsequent requests to forefront (all of this several times, in a nested way)

I believe that they all race each other by placing the next requests in the forefront position.
This leads to the same as if I had not used forefront: true.
Does it make sense?

they all push subsequent requests to the forefront (all of this several times, in a nested way)

I see, so you basically have too much requests enqueued to the forefront, and you'd like to put priorities on that to have more predictability.
I believe this would need to be implemented in the API first, right @mnmkng?
Or would it be enough to handle this locally, making queueHeadDict priority aware and handle the priority only on what is already loaded?

Yeah, I would not add things like priority without API support. It is something that we discussed for a Request Queue redesign. But there are features that have higher priority now.
I'm not sure how priority would help in your case though @gahabeen. Would you go for priority:  1 - 25 ? Also, how's priority related to pushing to dataset? If you have to wait for a lot of requests to be processed before pushing anyway, then maybe it would be better to run this with only a single initial request and parallelizing over 25 actor runs?

Nice to know I'm not the first one to mention it.
As you've mentionned:

priority for the first entry request (and all its descendants) would be 1
priority for the second entry request (and all its descendants) would be 2
etc until priority 25 (for an example with 25 input requests)

Idea being that all requests with priority 1 would get threated first and some of priority 2 might start as well before new requests for priority 1 potentially join the queue.
I could solve this with 1 query / run but:

It's not our client specification (wishes to handle several inputs per API query)
It's sub optimal as we'll quickly end up with an enormous amount of runs in parallel which:

will be tough to observe
will mean from a few thousands to millions of runs per day
will be poor management against website as we may end up DDOS them

Then I would just do it serially. The only thing you would lose is "some of priority 2 might start as well before new requests for priority 1 potentially join", but given the fact that requests are generally added faster than processed - unless you're adding them one by one - the serial processing should not have a huge impact on performance.

Alright, it works really well 👍
The only hacky thing I had to do is reset the isRunninPromise before re-running the crawler.
crawler.isRunningPromise = null;

I'll share the pushed solution tomorrow once I merge it (in the router-framework).

Fyi, the way I have successfully implemented it is by maintaining a simple requests stack (serial-queue) and looping over these by feeding the next request to the requestQueue and running the crawler.
See here:
https://github.com/apify-projects/router-framework/blob/df5ceb8e77fb505de56a038dfb3f96f67652345f/src/router.ts#L232-L249

Looks good to me. I'll close this, since having a priority based queue really depends on the API, no the SDK.

I've updated the previous solution as I discovered that it would crash after too many calls.
To fix it, I switched to using a new crawler instance by serial request (using the same requestQueue).

Ok, after brainstorming with @pocesar, I've managed to make a PoC using the actual solution I was looking for. I simply handle the { priority: number } myself in a KV store that I keep in sync with the queue and if not, it defaults to normal behaviour of the queue.
https://github.com/apify-projects/felps/blob/master/src/overrides/request-queue.ts

@gahabeen If you want, you can do a simple class to https://github.com/metalwarrior665/apify-utils which is where we collect these advanced topics.

@mnmkng It's in fact mostly in-memory for now (only backing it up to KVS). I'll see to optimise it at scale when time comes.

Words of a senior dev 😄 👍


Support custom encryption keys for storage?

Can we support custom encryption keys for files uploaded?
Spec is here: https://cloud.google.com/compute/docs/disks/customer-supplied-encryption#top_of_page

Docs above are for Compute Engine, I believe this is the correct docs page for Storage:
https://cloud.google.com/storage/docs/encryption

Been looking at this. This looks really cool.

This appears to work just fine. I added the headers as follows to Bucket#create_file:
        headers = {
          "x-goog-encryption-algorithm" => "AES256",
          "x-goog-encryption-key" => Base64.encode64(options[:key]),   # TODO: delete entry for this key, since not an option
          "x-goog-encryption-key-sha256" => Digest::SHA256.base64digest(options[:key])
        } if options[:key]

        @client.execute api_method: @storage.objects.insert,
                        media: media,
                        parameters: params,
                        body_object: insert_file_request(options),
                        headers: headers         # TODO: conditionalize

Then I uploaded a file with a key, and reloaded it to see the response. As described, "the object's CRC32C checksum and MD5 hash" were no longer in the response. Downloading also failed, although I haven't yet inspected the reason.
irb(main):009:0> cipher = OpenSSL::Cipher.new('aes-256-cbc')
=> #<OpenSSL::Cipher:0x007fdc8a055370>
irb(main):010:0> cipher.encrypt
=> #<OpenSSL::Cipher:0x007fdc8a055370>
irb(main):011:0> key = cipher.random_key
=> "\xD5L\xFCA\xA2\x84mbA\xA95)T\xAA\x03\xAA\xEF\xD6\x9C\xBF\f\xDE\x97\x10\xDA\x00\x8Dd\xA3\xA6`s"
irb(main):012:0> file = bucket.create_file "./LICENSE", key: key
=> #<Gcloud::Storage::File:0x007fdc8b9d4800 @connection=Gcloud::Storage::Connection(bustling-kayak-91516), @gapi=#<Google::APIClient::Schema::Storage::V1::Object:0x3fee45cea478 DATA:{"kind"=>"storage#object", "id"=>"my-bucket123321/./LICENSE/1465254998004000", "selfLink"=>"https://www.googleapis.com/storage/v1/b/my-bucket123321/o/.%2FLICENSE", "name"=>"./LICENSE", "bucket"=>"my-bucket123321", "generation"=>"1465254998004000", "metageneration"=>"1", "timeCreated"=>"2016-06-06T23:16:37.982Z", "updated"=>"2016-06-06T23:16:37.982Z", "storageClass"=>"STANDARD", "size"=>"11357", "md5Hash"=>"htPzqVwyTJR5vYmGlo9DJw==", "mediaLink"=>"https://www.googleapis.com/download/storage/v1/b/my-bucket123321/o/.%2FLICENSE?generation=1465254998004000&alt=media", "crc32c"=>"Kqo3Ow==", "etag"=>"CKCipZ3ElM0CEAE=", "customerEncryption"=>{"encryptionAlgorithm"=>"AES256", "keySha256"=>"IrXFw/lSDiYNqaqQPXrHcCygAO8bRA49aBsfjQCiwxs="}}>>
irb(main):013:0> file.reload!
=> #<Google::APIClient::Schema::Storage::V1::Object:0x3fee4694ca68 DATA:{"kind"=>"storage#object", "id"=>"my-bucket123321/./LICENSE/1465254998004000", "selfLink"=>"https://www.googleapis.com/storage/v1/b/my-bucket123321/o/.%2FLICENSE", "name"=>"./LICENSE", "bucket"=>"my-bucket123321", "generation"=>"1465254998004000", "metageneration"=>"1", "timeCreated"=>"2016-06-06T23:16:37.982Z", "updated"=>"2016-06-06T23:16:37.982Z", "storageClass"=>"STANDARD", "size"=>"11357", "mediaLink"=>"https://www.googleapis.com/download/storage/v1/b/my-bucket123321/o/.%2FLICENSE?generation=1465254998004000&alt=media", "etag"=>"CKCipZ3ElM0CEAE=", "customerEncryption"=>{"encryptionAlgorithm"=>"AES256", "keySha256"=>"IrXFw/lSDiYNqaqQPXrHcCygAO8bRA49aBsfjQCiwxs="}}>
irb(main):014:0> file.download "./LICENSE-COPY"
NoMethodError: undefined method `[]' for nil:NilClass
	from /Users/quartzmo/code/google/codez/gcloud-ruby/lib/gcloud/storage/errors.rb:48:in `from_response'
	from /Users/quartzmo/code/google/codez/gcloud-ruby/lib/gcloud/storage/file.rb:355:in `download'
	from (irb):14

If you scroll over far enough to the right, you can see that the response contains:
"customerEncryption"=>{"encryptionAlgorithm"=>"AES256", "keySha256"=>"IrXFw/lSDiYNqaqQPXrHcCygAO8bRA49aBsfjQCiwxs="}

Awesome. I can't wait to have some design discussions on how to expose this in the ruby API!

The response is 400:
@response=#<Faraday::Response:0x007fe60b120ec8 ...> ... "content-length"=>"72", "server"=>"UploadServer", "alternate-protocol"=>"443:quic", "alt-svc"=>"quic=\":443\"; ma=2592000; v=\"34,33,32,31,30,29,28,27,26,25\"", "connection"=>"close"} @status=400>>

Its a bug in the error handling then. Good catch!

I successfully completed a roundtrip insert, get, and download using a customer-supplied key (below). My branch is here if anyone wants to try it.
irb(main):002:0> bucket = storage.bucket "my-bucket123321"
=> #<Gcloud::Storage::Bucket:0x007f91031206d8 @connection=Gcloud::Storage::Connection(bustling-kayak-91516), @gapi=#<Google::APIClient::Schema::Storage::V1::Bucket:0x3fc881890574 DATA:{"kind"=>"storage#bucket", "id"=>"my-bucket123321", "selfLink"=>"https://www.googleapis.com/storage/v1/b/my-bucket123321", "projectNumber"=>"664476911263", "name"=>"my-bucket123321", "timeCreated"=>"2016-06-06T23:07:41.263Z", "updated"=>"2016-06-06T23:07:41.263Z", "metageneration"=>"1", "location"=>"US", "storageClass"=>"STANDARD", "etag"=>"CAE="}>>
irb(main):003:0> key = "\xD5L\xFCA\xA2\x84mbA\xA95)T\xAA\x03\xAA\xEF\xD6\x9C\xBF\f\xDE\x97\x10\xDA\x00\x8Dd\xA3\xA6`s"
=> "\xD5L\xFCA\xA2\x84mbA\xA95)T\xAA\u0003\xAA\xEF֜\xBF\fޗ\u0010\xDA\u0000\x8Dd\xA3\xA6`s"
irb(main):004:0> file = bucket.file "./LICENSE", key: key
=> #<Gcloud::Storage::File:0x007f9103053f48 @connection=Gcloud::Storage::Connection(bustling-kayak-91516), @gapi=#<Google::APIClient::Schema::Storage::V1::Object:0x3fc88182c0d8 DATA:{"kind"=>"storage#object", "id"=>"my-bucket123321/./LICENSE/1465254998004000", "selfLink"=>"https://www.googleapis.com/storage/v1/b/my-bucket123321/o/.%2FLICENSE", "name"=>"./LICENSE", "bucket"=>"my-bucket123321", "generation"=>"1465254998004000", "metageneration"=>"1", "timeCreated"=>"2016-06-06T23:16:37.982Z", "updated"=>"2016-06-06T23:16:37.982Z", "storageClass"=>"STANDARD", "size"=>"11357", "md5Hash"=>"htPzqVwyTJR5vYmGlo9DJw==", "mediaLink"=>"https://www.googleapis.com/download/storage/v1/b/my-bucket123321/o/.%2FLICENSE?generation=1465254998004000&alt=media", "crc32c"=>"Kqo3Ow==", "etag"=>"CKCipZ3ElM0CEAE=", "customerEncryption"=>{"encryptionAlgorithm"=>"AES256", "keySha256"=>"IrXFw/lSDiYNqaqQPXrHcCygAO8bRA49aBsfjQCiwxs="}}>>
irb(main):005:0> file.download "./LICENSE-COPY", key: key
=> #<File:./LICENSE-COPY>

Thanks @blowmage for helping me debug some of this, or I would still be at it... :)

@jgeewax We can definitely do this. How urgent should we consider it?

I see three main options for specifying keys:

Globally
Per Session
Per API Call

Do we allow users to configure all 3?

@quartzmo Which API calls support custom encryption keys?

@blowmage They are not listed, but implicitly it seems every operation on the stored object (File):

You upload an object using a customer-supplied encryption key, and you attempt to perform another operation on the object (other than requesting or updating metadata) without providing the key.

-- Managing Data Encryption

Does listing files in a bucket behave differently if a custom key is provided? Like, do you get or not get the md5 hash when listing files?

I haven't tried it, but I would guess that the md5 would be omitted. I will try it in a bit.

I think people will want to specify a key per file -- and not per session or per API call.
In the future, they will want to use some key derivation function that they set at the top level, but to start the focus is "when I upload a file, let me give you a key to encrypt it with". Does that make life any easier?

@jgeewax Maybe we can work incrementally toward such higher-level conveniences? Should we start just by adding key-per-API request support? That's not hard to do, my spike branch already adds it for a few methods.

Makes sense. Thanks for clarifying the direction.

The key headers do not seem to be used in these API methods:

Objects: list - even if there is only one object in the bucket, passing the key used to insert it does not return a representation including md5, etc
Objects: delete - deleting an object is successful even without passing the key used to insert it


"chat_command" event missing?

I can find no way to get this event called.
Other events, like when I register glasses with a new person work fine

It is in there and works, but in some version registered_player_join is spammed every tick, so it may be just hard to capture. Already fixed in e50820f0334178957f1329dbaedc1cf4f76b4a78.


Throw error if IPSec key changed but key ID didn't

The key rotation for transparent encryption should always be performed with a new key ID as per the documentation. When reading the IPSec key, we should throw an error if we detect that the key changed while the key ID didn't.
The key is loaded in loadIPSecKeys() and we compare the key IDs at: https://github.com/cilium/cilium/blob/5b911b80127d7d79120252334b92f7fd6495831c/pkg/datapath/linux/ipsec/ipsec_linux.go#L548

[ ] Implement the error case.
[ ] Test it by doing a key rotation with an unchanged key ID.
[ ] Add a unit test in ipsec_linux_test.go.

@pchaigno, some questions:

Does it make sense to move it to Community cilium?
Are you actively working on this?


Drop the semicolon after bracketed block statements

The semicolon at the end of block statements, AKA for/if/while/match statements, is completely redundant and unnessecary.

Looks like this can't be removed due to the limitations of the macro parser, which don't allow block statements to be parsed directly after expression statements. Oh well.


OAK-10703: oak-doc: add branch history to roadmap

I'm curious, is the ASCII timeline view of the release history something you wrote by hand, or was a tool used to generate it?

:-) it's hand work.
A generic tool would be interesting, but it would need to be highly configurable (because not all changes have the same importance)


Update dependencies and items

Update BlishHUD dependency to 1.0.0 and static items to latest as of Sept 30, 2023

This should resolve #6


Issue with Using Pre-trained FoodLMM-Chat Model for Inference

Hi,
I've been trying to use the pre-trained FoodLMM-Chat model provided on Hugging Face (https://huggingface.co/Yueha0/FoodLMM-Chat) for inference, but I'm running into a couple of issues.
I tried using the Pipeline API and loading the model directly but I ran into errors. I also cloned the GitHub repository for FoodLMM and looked for a way to use the pre-trained weights for inference. However, the scripts seem to be more focused on training, and I'm not sure how to set them up for inference using the pre-trained weights available on Hugging Face.
Is there a recommended way to load and use the FoodLMM-Chat model for inference without retraining? Specifically, I'd like to understand how to properly load the pre-trained model and run inference using the available weights from Hugging Face.
Any guidance would be greatly appreciated!

Hi, you can refer to online_demo.py for conducting inferences with the pre-trained model.


Persist user agent and other headers across windows

This PR addresses #915.
The add_header, add_headers, and set_headers methods now set headers globally across all windows (existing windows as well as new ones). The only exception is temporary headers, which are local to the current window.

Other than the tests depending on the order of @session.windows it looks good to me.

Thinking about this a bit more -- I'm not sure this behavior is correct/desired.  I definitely expect the headers to be the same for a window opened from another window  (clicking a link with target _blank, etc) but I'm not convinced I would expect the same headers if I explicitly opened a new window.

@twalpole hard to say what is more convenient. We cannot set headers in real browser without extension, but once we set with a plugin like (ModHeader) it's set for all tabs and windows. I think it's the only way to make it work and not to overcomplicate code at the same time. Maybe it's a mistake to think that they belong to window but not the whole browser..

Ok - merging as is then


feat(avm-simulator): implement NOTEHASHEXISTS

Closes #4839.

Current dependencies on/for this PR:

#4882  👈
master

This stack of pull requests is managed by Graphite.

Benchmark results
Metrics with a significant change:

l2_block_public_tx_process_time_in_ms (32): 18,166 (+20%)
note_trial_decrypting_time_in_ms (64): 136 (+255%)

Detailed results
All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call  to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.
This benchmark source data is available in JSON format on S3 here.
Values are compared against data from master at commit 553c2c60 and shown if the difference exceeds 1%.
L2 block published to L1
Each column represents the number of txs on an L2 block published to L1.

Metric
8 txs
32 txs
64 txs

l1_rollup_calldata_size_in_bytes
5,700 (+10%)
18,884 (+3%)
36,452 (+1%)

l1_rollup_calldata_gas
66,096 (+3%)
238,844 (+1%)
469,724

l1_rollup_execution_gas
194,044 (+1%)
500,150
908,982

l2_block_processing_time_in_ms
1,204 (-1%)
4,687 (+3%)
9,015

note_successful_decrypting_time_in_ms
205 (+3%)
546 (+1%)
999

note_trial_decrypting_time_in_ms
12.6 (+45%)
66.5
:warning: 136 (+255%)

l2_block_building_time_in_ms
17,062 (+6%)
70,641 (+10%)
141,507 (+11%)

l2_block_rollup_simulation_time_in_ms
13,230 (+7%)
52,401 (+7%)
111,254 (+14%)

l2_block_public_tx_process_time_in_ms
3,803
:warning: 18,166 (+20%)
30,106

L2 chain processing
Each column represents the number of blocks on the L2 chain where each block has 16 txs.

Metric
5 blocks
10 blocks

node_history_sync_time_in_ms
14,895 (-3%)
26,959 (-4%)

note_history_successful_decrypting_time_in_ms
1,254
2,465 (+1%)

note_history_trial_decrypting_time_in_ms
92.3 (-3%)
157 (+11%)

node_database_size_in_bytes
18,784,336 (+1%)
35,500,112 (+1%)

pxe_database_size_in_bytes
29,923
59,478

Circuits stats
Stats on running time and I/O sizes collected for every circuit run across all benchmarks.

Circuit
circuit_simulation_time_in_ms
circuit_input_size_in_bytes
circuit_output_size_in_bytes

private-kernel-init
253 (+2%)
44,736
28,001

private-kernel-ordering
184 (+3%)
52,625
14,627

base-rollup
1,357 (+3%)
175,343
933

root-rollup
71.5 (+1%)
4,192
825

private-kernel-inner
319 (+1%)
73,715
28,001

public-kernel-app-logic
196 (+1%)
32,254
25,379

merge-rollup
5.75 (-1%)
2,712
933

Tree insertion stats
The duration to insert a fixed batch of leaves into each tree type.

Metric
1 leaves
2 leaves
8 leaves
16 leaves
32 leaves
64 leaves
128 leaves
512 leaves
1024 leaves
2048 leaves
4096 leaves

batch_insert_into_append_only_tree_16_depth_ms
10.2 (+2%)
10.9 (+7%)
17.4 (+2%)
16.1
22.5 (-1%)
35.3 (-18%)
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_16_depth_hash_count
16.9
17.5
23.0
31.6
47.0
79.0
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_16_depth_hash_ms
0.589 (+2%)
0.606 (+7%)
0.737 (+2%)
0.496
0.468 (-1%)
0.440 (-18%)
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_32_depth_ms
N/A
N/A
N/A
N/A
N/A
47.6 (+3%)
72.8
230 (-1%)
446
868 (-2%)
1,732 (-1%)

batch_insert_into_append_only_tree_32_depth_hash_count
N/A
N/A
N/A
N/A
N/A
96.0
159
543
1,055
2,079
4,127

batch_insert_into_append_only_tree_32_depth_hash_ms
N/A
N/A
N/A
N/A
N/A
0.487 (+3%)
0.448
0.420 (-1%)
0.416 (-1%)
0.413 (-2%)
0.414 (-1%)

batch_insert_into_indexed_tree_20_depth_ms
N/A
N/A
N/A
N/A
N/A
55.8 (+4%)
101
339 (-1%)
664
1,313 (-1%)
2,616 (-1%)

batch_insert_into_indexed_tree_20_depth_hash_count
N/A
N/A
N/A
N/A
N/A
104
197
691
1,363
2,707
5,395

batch_insert_into_indexed_tree_20_depth_hash_ms
N/A
N/A
N/A
N/A
N/A
0.496 (+3%)
0.480
0.462 (-1%)
0.456
0.455 (-1%)
0.454

batch_insert_into_indexed_tree_40_depth_ms
N/A
N/A
N/A
N/A
62.5 (+2%)
N/A
N/A
N/A
N/A
N/A
N/A

batch_insert_into_indexed_tree_40_depth_hash_count
N/A
N/A
N/A
N/A
109
N/A
N/A
N/A
N/A
N/A
N/A

batch_insert_into_indexed_tree_40_depth_hash_ms
N/A
N/A
N/A
N/A
0.544 (+1%)
N/A
N/A
N/A
N/A
N/A
N/A

Miscellaneous
Transaction sizes based on how many contracts are deployed in the tx.

Metric
0 deployed contracts

tx_size_in_bytes
19,179

Transaction processing duration by data writes.

Metric
0 new note hashes
1 new note hashes

tx_pxe_processing_time_ms
2,605
1,373 (+2%)

Metric
0 public data writes
1 public data writes

tx_sequencer_processing_time_ms
0.0342 (+16%)
480 (+1%)

Merge activity

Mar 1, 11:46 AM EST: @fcarreiro merged this pull request with Graphite.


Extend audit header with lastClassified?

Given that adding a classification to an entity should be possible for reference copies, and that doing so should not increment the version or change the updateTime, should we have a lastClassified time stamp?
This would be helpful to construct the correct history for a reference copy.

Also, does this change in classification behaviour MEA that the home repository classification methods (classifyEntity, declassifyEntity, etc) should also no longer update version or updateTime, for consistency?

What would lastClassified mean?  The audit header is used in entities, relationships and classifications.  What does it mean for each of these elements?
Classifications mean nothing to relationships
The reason we separated classifications into first class elements is for situations where the attached entity is read only to Egeria - eg an entity from IGC/Atlas.  Since Egeria can not update these, we enabled the classification to be saved by a different repository - eg graph repository. So this is not a value we can set in the entity.
The create / update time in the classification indicates when the classification was added to the entity or updated.  I am not
sure what a lastClassified property would add to the classification.

The classification methods no longer update the entity - we discussed this on the last functionality call and I have a tested change.


BudgetOptimizer does not seem to support time-varying intercept

Not quite sure if this will be fixed in the upcoming 0.7.0 release but noticed the re-written BudgetOptimizer does not work with time-varying intercept or there's a conflict with PyTensor.
I followed the mmm_budget_allocation_example.ipynb and in this cell was when I got the error.
response = mmm.allocate_budget_to_maximize_response(
    budget=total_budget,
    num_days=8,
    time_granularity="weekly",
    budget_bounds=budget_bounds
)

Here is the error I got using an MMM with tvp that did not come up with another MMM without time-varying intercept:
NotImplementedError                       Traceback (most recent call last)
[/usr/local/lib/python3.10/dist-packages/pytensor/link/basic.py](https://localhost:8080/#) in __set__(self, value)
    101                 # Use in-place filtering when/if possible
--> 102                 self.storage[0] = self.type.filter_inplace(
    103                     value, self.storage[0], **kwargs

10 frames
[/usr/local/lib/python3.10/dist-packages/pytensor/graph/type.py](https://localhost:8080/#) in filter_inplace(self, value, storage, strict, allow_downcast)
    127         """
--> 128         raise NotImplementedError()
    129

NotImplementedError:

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
[<ipython-input-49-329a8396104b>](https://localhost:8080/#) in <cell line: 1>()
----> 1 response = mmm.allocate_budget_to_maximize_response(
      2     budget=total_budget,
      3     num_days=8,
      4     time_granularity="weekly",
      5     budget_bounds=budget_bounds

[/usr/local/lib/python3.10/dist-packages/pymc_marketing/mmm/delayed_saturated_mmm.py](https://localhost:8080/#) in allocate_budget_to_maximize_response(self, budget, time_granularity, num_days, budget_bounds, custom_constraints, quantile)
   1977         )
   1978
-> 1979         return self.sample_posterior_predictive(
   1980             X_pred=synth_dataset,
   1981             extend_idata=False,

[/usr/local/lib/python3.10/dist-packages/pymc_marketing/mmm/delayed_saturated_mmm.py](https://localhost:8080/#) in sample_posterior_predictive(self, X_pred, extend_idata, combined, include_last_observations, original_scale, **sample_posterior_predictive_kwargs)
   1644
   1645         with self.model:  # sample with new input data
-> 1646             post_pred = pm.sample_posterior_predictive(
   1647                 self.idata, **sample_posterior_predictive_kwargs
   1648             )

[/usr/local/lib/python3.10/dist-packages/pymc/sampling/forward.py](https://localhost:8080/#) in sample_posterior_predictive(trace, model, var_names, sample_dims, random_seed, progressbar, progressbar_theme, return_inferencedata, extend_inferencedata, predictions, idata_kwargs, compile_kwargs)
    859                     param = _trace[idx % len_trace]
    860
--> 861                 values = sampler_fn(**param)
    862
    863                 for k, v in zip(vars_, values):

[/usr/local/lib/python3.10/dist-packages/pymc/util.py](https://localhost:8080/#) in wrapped(**kwargs)
    393     def wrapped(**kwargs):
    394         input_point = {k: v for k, v in kwargs.items() if k in ins}
--> 395         return core_function(**input_point)
    396
    397     return wrapped

[/usr/local/lib/python3.10/dist-packages/pytensor/compile/function/types.py](https://localhost:8080/#) in __call__(self, *args, **kwargs)
    895         if kwargs:  # for speed, skip the items for empty kwargs
    896             for k, arg in kwargs.items():
--> 897                 self[k] = arg
    898
    899         if (

[/usr/local/lib/python3.10/dist-packages/pytensor/compile/function/types.py](https://localhost:8080/#) in __setitem__(self, item, value)
    547
    548     def __setitem__(self, item, value):
--> 549         self.value[item] = value
    550
    551     def __copy__(self):

[/usr/local/lib/python3.10/dist-packages/pytensor/compile/function/types.py](https://localhost:8080/#) in __setitem__(self, item, value)
    503                     )
    504                 if isinstance(s, Container):
--> 505                     s.value = value
    506                     s.provided += 1
    507                 else:

[/usr/local/lib/python3.10/dist-packages/pytensor/link/basic.py](https://localhost:8080/#) in __set__(self, value)
    104                 )
    105             except NotImplementedError:
--> 106                 self.storage[0] = self.type.filter(value, **kwargs)
    107
    108         except Exception as e:

[/usr/local/lib/python3.10/dist-packages/pytensor/tensor/type.py](https://localhost:8080/#) in filter(self, data, strict, allow_downcast)
    240
    241         if self.ndim != data.ndim:
--> 242             raise TypeError(
    243                 f"Wrong number of dimensions: expected {self.ndim},"
    244                 f" got {data.ndim} with shape {data.shape}."

TypeError: ('Wrong number of dimensions: expected 0, got 1 with shape (118,).', 'Container name "intercept"')

Hi @AlfredoJF,
thank you for bringing this up. Are you having issues with the out of sample prediction as well? Would you be able to give that method as well.
CC: @cetagostini

It seems like a broadcasting issue right @wd60622 ?

Hi @wd60622, no issues with the Out-of-sample posterior only with the BudgetOptimizer
y_test_pred = mmm.sample_posterior_predictive(X_pred=X_test, extend_idata=False, include_last_observations=True)

Hi @AlfredoJF,
Thanks for checking that. If you have found the culprit, feel free to open a PR!
If not, we can do some investigation. Would you be able to post a small reproducible example (including the initialization of the class)?

Hi @wd60622,
Yes, I'll post a reproducible example and leave the investigation to the experts for now. :)

Yes, I'll post a reproducible example and leave the investigation to the experts for now. :)

Great, sounds good. We a reproducible example, we can dig into it a bit better.
We should be able to get to it soon

Hi @wd60622
I used the mmm and tvp example data to see if I could create a reproducible example for further investigation but I did not get the same error as in my model. I cannot share my dataset as it contains client's data but will try to create another dataset for this purpose.
Thanks for the patience!

I just tested the first workaround in #814, and it fixed this issue. So I assume #815 also fixes this one for the next release.

I just tested the first workaround in #814, and it fixed this issue. So I assume #815 also fixes this one for the next release.

Mmm interesting that that workaround helped out this one as well. Not obvious why that would be the case.
Are you using 0.7.0 then? or installed from main branch?

@wd60622 Yes, I'm using 0.7.0 and have overwritten the MMM class for now as I'm closer to the delivery date of my project. I'll upgrade to 0.8.0 once it's released in a future iteration of my model

Sounds good. Well glad that it sorted itself out.
If you run into any more issues, feel free to raise an issue.
I will leave this issue open for now


variant enum

for variant type like
 variant {
   a;
   b;
 };

use canister.xxx({"a": None}) returns AttributeError: 'NoneType' object has no attribute 'buildTypeTable'

Currently, variant is represented by:
None: [] (Empty list),
Some: [xxx] (list with one item)

Thus you need to use canister.xxx({"a": []})

Currently, variant is represented by:
None: [] (Empty list),
Some: [xxx] (list with one item)

Thus you need to use canister.xxx({"a": []})

Sry, I got it wrong. Option is used in this way.
I will look into this issue.

Thanks for the report. Try to fix it in #49.
You can switch to that branch and use canister.xxx({"a": None}) to test whether the result is correct.


Does this only support MySQL or can it be used for SQL Server?

I actually have no idea! I've never used SQL Server.


Preventing certain fields of a span being logged with BunyanFormattingLayer

Quick question, is it possible to configure BunyanFormattingLayer to prevent logging certain fields to the console?
My use case is as follows. I have a panic hook set up with color-eyre that gives me a fancy backtrace (with colors and stuff) whenever a panic occurs in my web app. I use tracing::error(error_report = %panic_report) to log the fancy backtrace generated by color-eyre to the BunyanFormattingLayer. The problem is, I want these logs to only appear in the context of an external observability solution, such as Jaeger, so not in the console.
So right now my panic logs in the console look like this: (without the newlines)
{
	"v": 0,
	"name": "<app name here>",
	"msg": "[REQUEST - EVENT] Lazy instance has previously been poisoned",
	"level": 50,
	"hostname": "<hostname here>",
	"pid": 34264,
	"time": "2022-07-17T11:32:18Z",
	"target": "panic_hook",
	"line": 20,
	"file": "<filename here>",
	"error_report": "<insert extremely long backtrace here>"
}

So the extremely long backtrace is polluting my logs.
My question is: is it possible to prevent the "error_report" field from appearing in the logs, while still having it available to external observability solutions?

Yes, this is possible - we just need to skip those fields when serializing the event/span.
If you'd like to work on a PR, I'm happy to review it.


Add MFEM_MPIRUN_CMD config var to set alternate MPI launcher [alternate_mpi_launcher]

Allows setting an alternate command for launching MPI jobs, for systems that use SLURM or other job schedulers instead of 'mpirun'.

@trb413 - I need to add CMAKE logic for this too...

I think MFEM_MPIRUN is a more consistent name for this variable, cf. MFEM_CXX.

Renamed this branch.  The '/' screws up some of the test scripts.


Chapter 12: Trust but Verify

Planning on discussing tool validation and needing to go beyond traditional commercial forensic tools.

Ooh I like that. I'm sure I'll have some ideas for this too 👍 but it's your chapter so you steer the ship! Let us know how we can support.

Are we thinking this will be generalized? Mobile? Computer?

Right now leaning towards generalized, with hitting on mobile and computer.

Ok cool. It'll only matter for placement in the order of chapters since we'll be having subsections.

Hey there! I'm tentatively setting 7/31/2022 as a milestone for publishing v1.0 of this book. We'll have the title decided in the next couple weeks which will be the first of multiple administrative tasks we'll complete in July. At this point, please let me know if you intend to have at least a working, editable version of your chapter by 7/31/2022.
If not, please know that's perfectly fine. It doesn't mean your chapter won't get published, it just won't get published in v1.0. It'll simply be added when it's ready to be published and I'll push out a new version of the book, (i.e., V1.3, v1.7, etc) with your new content. I hope we have about 10 ready to go by 7/31/2022 so we can push to publish v1.0 shortly thereafter, but I won't know that until I hear from you! So, please let me know!

Hey there, it's been a few months. Just wanted to check in and see how progress is going on your chapter? Whenever your chapter is ready to go, submit it to the repo, and once its reviewed it'll get published without delay! Cheers and hope you are well!