Refactor VariableValueResolver.CoerceVariableValues(Schema, OperationDefinitionNode, IReadOnlyDictionary)

I've selected VariableValueResolver.CoerceVariableValues(Schema, OperationDefinitionNode, IReadOnlyDictionary) for refactoring, which is a unit of 54 lines of code. Addressing this will make our codebase more maintainable and improve Better Code Hub's Write Short Units of Code guideline rating! 👍
Here's the gist of this guideline:

Definition 📖
Limit the length of code units to 15 lines of code.
Why❓
Small units are easier to analyse, test and reuse.
How 🔧
When writing new units, don't let them grow above 15 lines of code. When a unit grows beyond this, split it in smaller units of no longer than 15 lines.

You can find more info about this guideline in Building Maintainable Software. 📖

ℹ To know how many other refactoring candidates need addressing to get a guideline compliant, select some by clicking on the 🔲  next to them. The risk profile below the candidates signals (✅) when it's enough! 🏁

Good luck and happy coding! :shipit: :sparkles: :100:

Fixed with PR #98


Svg import broken

I am...
| -------------------------------------------------------------------------------------------------

Reporting a bug or issue

Expected behavior:

Observed behavior:
><EMAIL_ADDRESS>postinstall /Users/<redacted>/basic-svg
> npm run build && skpm-link

><EMAIL_ADDRESS>build /Users/<redacted>/basic-svg
> skpm-build

[1/2] 🖨  Copied src/manifest.json in 3ms
error Error while building ./my-command.js
./node_modules/node-gyp-build/index.js
Module not found: Error: Can't resolve 'fs' in '/Users/<redacted>/basic-svg/node_modules/node-gyp-build'
resolve 'fs' in '/Users/<redacted>/basic-svg/node_modules/node-gyp-build'
  Parsed request is a module
  using description file: /Users/<redacted>/basic-svg/node_modules/node-gyp-build/package.json (relative path: .)
    Field 'browser' doesn't contain a valid alias configuration
  after using description file: /Users/<redacted>/basic-svg/node_modules/node-gyp-build/package.json (relative path: .)
    resolve as module

How to reproduce:
curl https://codeload.github.com/airbnb/react-sketchapp/tar.gz/master | tar -xz --strip=2 react-sketchapp-master/examples/basic-svg
cd basic-svg

Sketch version:
63.1
Please attach ~screenshots, a zip file of your project, and~ or a link to your github project

https://github.com/airbnb/react-sketchapp/tree/master/examples/basic-svg

I believe this is caused by https://github.com/airbnb/react-sketchapp/blob/9f4b38cdfb7848b32ff9c1968b145dabd72739a9/src/components/index.ts#L8
Proposed fix:
import Svg from './Svg';
export { Svg };

Fixed in 78c317019391eaa6ea0505c09bdb2e7a2c1ccf76, but may be best to revert and apply in a new atomic PR.
I believe the import * as Svg is not actually needed, as this is already handled by applying static references at https://github.com/airbnb/react-sketchapp/blob/9f4b38cdfb7848b32ff9c1968b145dabd72739a9/src/components/Svg/Svg.tsx#L49 . On the other hand, I'm surprised that this was working though with commonjs, but not with the es2015 module, there must be some difference in the commonjs interop polyfills compared to ES.

Wait I’m confused, the error is completely unrelated to the svg import.
The import * as is on purpose (feature from a previous PR). And I’m not sure how this is going to fix the error

Wait I’m confused, the error is completely unrelated to the svg import.
The import * as is on purpose (feature from a previous PR). And I’m not sure how this is going to fix the error

Ah, sorry, that seems to caused by something else. I managed to get rid of it by aliasing react-sketchapp to lib/module, then was getting this:

And with doing this in the lib/module/components/index.js file:
export { Document } from './Document';
export { Page } from './Page';
export { Artboard } from './Artboard';
export { Image } from './Image';
export { RedBox } from './RedBox';
export { View } from './View';
export { Text } from './Text';
- import * as Svg from './Svg';
+ import Svg from './Svg';
export { Svg };

it renders fine
I can only find import * as Svg since the platform refactor. import Svg, { Circle, ... } from 'react-sketchapp/lib/components/Svg' should still work I think.

So the issue is that it's trying to bundle lib/module/platformBridges/macos.js but shouldn't (should bundle the sketch bridge instead)

Oh I think it's because of
"module": "lib/module/index.js",
"sketch": "lib/module/index.sketch.js",

skpm prioritize the module entry point instead of the sketch one

Ah, great catch, adding .sketch.js to the module entry seems to solve that issue.
Am still getting Error: Element type is invalid: expected a string (for built-in components) or a class/function (for composite components) but got: object. after that though.

I'll fix it

Great, thanks 🙂. I've undone the import Svg from patch in #516 , that was fixing it for me before, but isn't fixing it now 😕, anyway, that'll break destructuring from Svg I think.

Seems the import Svg from fix resolves the Element type is invalid error.
I just tested destructuring const { G, Path } = Svg; with it too, and that seems to still work.

Thanks a lot, ignore my last message. Works now with 3.2.5 🎉.
Looks like using export * from './components'; instead of export * from './index.common'; helped 🤔, not sure though.

@mathieudutour @macintoshhelper I think the latest version does't work completely.
My code does't work when I update react-sketchapp to 3.2.4 or 3.2.5.

The Svg default export was broken.

I need change my code this way to get the default export(Svg itself).

The Svg default export was broken.

Hi, I've run into this too (from running the basic-svg example). I'm using this code as a manual patch (inside node_modules):
export { Document } from './Document';
export { Page } from './Page';
export { Artboard } from './Artboard';
export { Image } from './Image';
export { RedBox } from './RedBox';
export { View } from './View';
export { Text } from './Text';
- import * as Svg from './Svg';
+ import Svg from './Svg';
export { Svg };

const { default: SvgContainer } = Svg; feels pretty unsafe to me, you should be able to use <Svg>


upgrade golang to 1.18 and upgrade go modules

Signed-off-by: haoqing0110<EMAIL_ADDRESS>

/assign @elgnay

/lgtm

/assign @yue9944882 plz help review this upgrade.

/assign @yue9944882
plz help review this upgrade.


[0.8.9] Crash in tree control

I was doing something else at the time this happened. I may have nudged the mouse or keyboard, but no idea what.
MeerK40t crash log. Version: 0.8.9000 on Windows: Python 3.8.10: AMD64 - wxPython: 4.2.1 msw (phoenix) wxWidgets <IP_ADDRESS>
Traceback (most recent call last):
  File "meerk40t\gui\wxmtree.py", line 1609, in on_drag_end_handler
  File "meerk40t\core\elements\elements.py", line 1552, in drag_and_drop
  File "meerk40t\core\node\groupnode.py", line 62, in drop
  File "meerk40t\core\node\node.py", line 1043, in append_child
  File "meerk40t\core\node\node.py", line 641, in notify_attached
  File "meerk40t\core\node\rootnode.py", line 55, in notify_attached
  File "meerk40t\gui\wxmtree.py", line 482, in node_attached
  File "meerk40t\gui\wxmtree.py", line 1066, in node_register
wx._core.wxAssertionError: C++ assertion ""param"" failed at ..\..\src\msw\treectrl.cpp(1113) in wxTreeCtrl::SetItemData(): failed to change tree items data

Looks like it was some sort of delete which I was able to undo.

This may relate to another set of crashes that I cannot ever track down. Like there's no repeatability or understanding how that unregistered node got in there or whatever.
I can't solve these bugs. It's annoying but I've seen similar crash logs many times and they've never been solved. If you ever figure out a method for consistently recreating this, it would be awesome.


RSpec/ExpectChange false positive when calling a method that returns different results each time it is called

Before:
it 'demonstrates the problem' do
  expect { false }.to change { changes_each_call.time }
end

def changes_each_call
  sleep(1)
  OpenStruct.new(time: Time.zone.now)
end

After:
it 'demonstrates the problem' do
  expect { false }.to change(changes_each_call, :time)
end

# changes_each_call unmodified

This results in changes_each_call only being called once when change is called. The previous behavior would construct two objects, one each time the block is evaluated.
This bit me when changes_each_call was the result of an ActiveRecord query which needs to be fetched from the database to see the change occur.

Fair enough.
expect { }.to change { rand.to_s }

is probably even more indicative.
I believe there's no way to differentiate between a variable and a method call:
def changes
  1
end

constant

change { changes.time }
change { constant.time }

parses to:
(begin
  (def :changes
    (args)
    (int 1))
  (send nil :constant)
  (block
    (send nil :change)
    (args)
    (send
      (send nil :changes) :time))
  (block
    (send nil :change)
    (args)
    (send
      (send nil :constant) :time)))

so (send nil :changes) vs (send nil :constant).

This bit me when changes_each_call was the result of an ActiveRecord query which needs to be fetched from the database to see the change occur.

Indeed. Not the only surprise with change I can think of, though [1, 2].
Would you suggest flipping the default enforced style to block for this cop?
To do so, we need some proof that the majority prefers in that way. https://github.com/pirj/real-world-rspec is a good source of well-known repos that use RSpec. Are there adjustments to the default style of RSpec/ExpectChange there?
I could only find it in:
canvas-lms/.rubocop.yml
gitlabhq/.rubocop_todo.yml

and it's Enabled: false in both.

Regardless of the default style for this cop, it should be tagged as SafeAutoCorrect: false since it can break things (that just happened to me). :wink:

This is hard to argue 🤷♂ 👍
Would you like to send a PR?

@pirj Sure!
I have started to do that and I have a question: should I delete all expect_correction statements from expect_change_spec.rb ? This feels like destroying other people's work. Is there a way to declare the cop auto-correctable only with the block style?

delete all expect_correction statements from expect_change_spec.rb

I can't remember working with SafeAutoCorrect: false cops, but I guess they still perform autocorrection. Thus, specs for this autocorrection should stay, too. I suggest looking at our parent project's such cops for inspiration.

declare the cop auto-correctable only with the block style?

I agree with your previous statement:

Regardless of the default style for this cop, it should be tagged as SafeAutoCorrect: false

change { rand.to_s } and change(rand, :to_s) will behave differently, and it doesn't depend on the direction of autocorrection - such correction is unsafe.

You are right, both directions are unsafe!
I am having trouble finding the right way to declare SafeAutoCorrect: false. I tried adding it to default.yml but then the rake task gives me

default.yml is out of sync

Should I change another file instead?
Thank you for your time! :slightly_smiling_face:

default.yml is out of sync

That's ok, you have to commit it, rake makes such a check, too.


Add "Big Picture Mode"

Add something similar to the Steam's "Big Picture Mode".

yes please, or a fullscreen button because i have over 40 games and the window is too small, it needs a fullscreen and a resize button.

So far I'm loving this app.....didn't care about xenia until now. Dude this would be amazing to use with a controller and use the guide button to bring up a pause menu to exit the game.

Update on this issue: For now I'll adapt the app so it can be maximized to Fullscreen because of lack of support for controlling the UI using the controller.

Update on this issue: For now I'll adapt the app so it can be maximized to Fullscreen because of lack of support for controlling the UI using the controller.

Look at my fork and tell me if you think is worth to merge it and go for it, thanks

It has potential, but since the formatting of the code (Xaml files mostly) is different, I can't just merge it.
I will have to work on the contribution guide so if someone wants to implement something they can follow it and we can be on the same page.
That being said, you gave me some ideas about how I might do it (from looking at the screenshots).

Just a sidenote: Don't be afraid to do a pull request if you have something that you would want to implement. I'm open to people contributing in the project. I'll either merge it if it's a good idea or if it has some formatting issues implement it myself and credit you.

 So FC of BPM will be a great solution. Right now I simply move the window but it's not ideal lol. My setup is on the couch, so it's a little inconvenient with my mini keyboard/touchpad. On my handheld also, the touch screen and full controller menu support will help.

 So FC of BPM will be a great solution. Right now I simply move the window but it's not ideal lol. My setup is on the couch, so it's a little inconvenient with my mini keyboard/touchpad. On my handheld also, the touch screen and full controller menu support will help.

I think this Fullscreen option I'm working on will help you with 175% display scale. I'll let you know when I'm done with it so you can properly test it on your own setup and give me feedback
For your handheld, controller support is for now scratched because of lack of support in .NET and C# for it. Worst case scenario is I make another app that will act like Big Picture Mode in something that supports controller input better or I create something from scratch. I also don't have any Windows devices that have touchscreen.

@SRC267 Check out the latest experimental and let me know if Fullscreen mode works better for you. (You only need to put the app once in Fullscreen and it will keep the current mode even if you close it and re open it again)

Works as expected in FCM, although it still cuts the top panel if I open it in , I suppose that's just the scaling that's causing it.
One issue I have in particular for this PC is I'm unable to select , but have no issues on my other systems. I've tried changing it in different window modes and scaling. It could be that it's unable to detect an English language pack, since I've installed W11 on this system without TPM, I selected international English upon Install. Not a big deal since the default language still is showing as English.

Yes it is expected to still be cutout in the windowed mode.
Not sure about User Language. People reported it working just fine in games like Gears of War so not sure.


NUTCH-2943 Implement core dependencies in build.gradle.kts

Right now the compile-core target does not work. We need to get that target working before we do any further work.
@AzureTriple @imanzanganeh @jbsimmon please focus only on getting compile-core working. Thank you

OK I fixed the resolve-defaults target by converting it to a Sync type.


Fix KeyError on change_ip

As response is initialised as an empty dict, line 233 was throwing a KeyError.
Alternatively, pystun initialised the object with None for each key/val
https://github.com/jtriley/pystun/blob/ab5eb34f143ccf916a2957d98f179944ef469dba/stun/__init__.py#L106-L108

Won't this still raise an error if the program tries sending a stun request to the change_address?
Regardless, I can merge this if it fixes the problem.

I'm not sure about the exact protocol flow, and what other errors might occur.
But, it fixes an issue (pynat doesn't work without it, for me).

If I remember correctly, there are new and "classic" versions of the STUN protocol--some servers don't implement the classic protocol and so don't always offer a CHANGE_ADDRESS attribute like PyNat expects. That makes newer STUN servers only really useful for like, discovering your external address, not discovering the NAT topology.
I'll merge it anyway and probably try and find a more robust way to signal that the CHANGE_ADDRESS was unavailable for use in PyNat's response later.


apps sc: Add S3-exporter for metrics about buckets

What this PR does / why we need it:
This adds an s3-exporter and service monitor so that we get metrics about our buckets. These metrics can be used to check if backups and retention jobs are working as they should, and keep an eye on the size.
Which issue this PR fixes: part of #373.
Public facing documentation PR (if applicable)

Special notes for reviewer:
Checklist:

[x] Added relevant notes to WIP-CHANGELOG.md
[x] Proper commit message prefix on all commits
[ ] Updated the public facing documentation
Is this changeset backwards compatible for existing clusters? Applying:

[x] is completely transparent, will not impact the workload in any way.
[ ] requires running a migration script.
[ ] will create noticeable cluster degradation.
E.g. logs or metrics are not being collected or Kubernetes API server
will not be responding while upgrading.
[ ] requires draining and/or replacing nodes.
[x] will change any APIs.
E.g. removes or changes any CK8S config options or Kubernetes APIs.
[ ] will break the cluster.
I.e. full cluster migration is required.

Pipeline config (if applicable)
If you change some config options (e.g. add/rename variable or change the default value) you may need to update the config used by the pipeline in pipeline/config.

Have you tested the exporter against different S3 providers?
I think I would be good to confirm wteather or not it uses path style  or dns access by default, so testing against safespring/citycloud (path access) and exoscale (dns access) would be nice 🙂

Have you tested the exporter against different S3 providers?
I think I would be good to confirm wteather or not it uses path style or dns access by default, so testing against safespring/citycloud (path access) and exoscale (dns access) would be nice

True, I have just tested with Exoscale yet. I'm also afraid there is no way to make this work with GCP... But then again there are already some other things that only work with S3 so I guess it is ok?
Will check Safespring soon :slightly_smiling_face:

True, I have just tested with Exoscale yet. I'm also afraid there is no way to make this work with GCP... But then again there are already some other things that only work with S3 so I guess it is ok?
Will check Safespring soon

Sounds good.
GCP is a completely different topic that I would say is out of scope.

I have now tested (and fixed) this on Safespring also and added the daily dashboard.
I'm not happy with the templating of the dashboard to add thresholds. It feels bad in many ways so I would suggest that we drop the last commit and go with a dashboard without thresholds for now. It will be less useful of course, but maintaining the templated json is very painful. If you have any suggestions for how to improve the situation I'm all ears!

Alright, after a discussion on slack I have now removed the templating from the dashboard. It was way too cumbersome to work with and made the whole dashboard much more complicated. Everything from thresholds to queries and "data density" had to be templated to make sure it would work in different situations.
So now there is no templating at all, no thresholds, limits or other "dynamic" features. The dashboard just shows the data and doesn't try to interpret it to tell you if the some value is too high or low. Some assumptions were still needed for data density, i.e. how much time is there between the data points. This is how it works:

It can take a long time to scrape metrics from a bucket if there are lots of objects in it. The situation doesn't change that fast either so it is not valuable for us to have data for ever 30 seconds or similar. For these reasons, the S3-exporter is configured to scrape the buckets with a 1h interval. This value can be changed.

Grafana automatically picks an interval between data points based on the time range and size of the screen. If there are missing data points for example because the sample rate is much lower than what Grafana chose, there will be "holes" in the graph. We can tell Grafana how to treat these null values (zero, "connected" or missing), but it can be quite confusing to look at and won't work with multiple time series (see example below).
Grafana cannot know about how dense or sparse the data is, so we would have to tell it how to work around it. We don't want to template the dashboard though, so it must be hard coded. I have chosen to hard code the dashboard to show only one data point per hour for the object storage metrics. This should be sparse enough to not cause trouble but still useful to look at.

Data density and phase
There are a number of things you can do to make Grafana work nicely with sparse data. Not everything will work in all situations though, so I'll try to describe with an example how and why I have done it.
What does the data look like?
Prometheus scrapes it from the S3-exporter. The metrics are per bucket, so we need to scrape for each bucket we want data from (very similar to blackbox exporter). Prometheus is configured with an interval and timeout to use, but does not scrape all buckets simultaneously. This means that even though we set the same interval for all of them, the metrics will most likely be staggered. With 1h interval, Prometheus may decide to start scraping bucket 1 at 12.00, bucket 2 at 12.10 and bucket 3 at 12.30. Bucket 1 would be scraped again at 13.00, bucket 2 at 13.10 and bucket 3 at 13.30. So the interval is correct but we cannot assume that the data points will line up.

What can we do to make Grafana understand the data?
The most obvious thing to do is probably adjusting the "Min interval":

A lower limit for the interval. Recommended to be set to write frequency, for example 1m if your data is written every minute. Default value can be set in data source settings for most data sources.

This will make sure that Grafana doesn't try to find data points more dense than this interval. Great!
Unfortunately though, it doesn't help when the data is staggered. Grafana would for example find the data for bucket 1 and maybe bucket 2 if it is close enough, but bucket 3 is completely out of phase so Grafana would never find it. This means the dashboards would be completely useless. If we calculate the total size of the buckets, there is no guarantee that all buckets are actually included! If we plot buckets separately, we may only see half of them. Compare this to the above.

To solve this, we need to write the queries so that they gather data over a certain time and then make that into a single data point. So instead of finding 1 data point and be happy about it, Grafana would query over 1h to find all the data points there, then take the average, max or min value of each series and preset as the single data point for the interval. This way the data from all buckets will be found, assuming that Prometheus scrapes at least once per hour.
Example query:
sum(max_over_time(s3_objects_size_sum_bytes[1h]))

Explanation:

s3_objects_size_sum_bytes = metric showing the total size of a bucket (so one value for each bucket)
max_over_time(<series>[1h]) = for each series, go through a 1h interval and find the maximum value. This gives us the maximum size of each bucket during the 1h around the point we are interested in.
sum(<series>) = aggregate over all the buckets to get the total size.


refactor: cleanup some DeepEqual usages

Please provide a description of this PR:

@howardjohn: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

unit-tests_istio
2ea2635a66cd57bc9c0aa1fa566a281813a8fc7c
link
unknown
/test unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

unit-tests_istio
2ea2635a66cd57bc9c0aa1fa566a281813a8fc7c
link
true
/test unit-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/retest
On Mon, Mar 27, 2023 at 7:10 PM Istio Automation @.***>
wrote:

@howardjohn https://github.com/howardjohn: The following test failed,
say /retest to rerun all failed tests or /retest-required to rerun all
mandatory failed tests:
Test name Commit Details Required Rerun command
unit-tests_istio 2ea2635
https://github.com/istio/istio/commit/2ea2635a66cd57bc9c0aa1fa566a281813a8fc7c
link
https://prow.istio.io/view/gs/istio-prow/pr-logs/pull/istio_istio/44129/unit-tests_istio/1640534553984503808
true /test unit-tests
Instructions for interacting with me using PR comments are available here
https://git.k8s.io/community/contributors/guide/pull-requests.md. If
you have questions or suggestions related to my behavior, please file an
issue against the kubernetes/test-infra
https://github.com/kubernetes/test-infra/issues/new?title=Prow issue:
repository. I understand the commands that are listed here
https://go.k8s.io/bot-commands.
—
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/44129#issuecomment-1486104570, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAEYGXIE3CYBAAP3UNWAFFTW6JCCZANCNFSM6AAAAAAWJQ543Y
.
You are receiving this because you were mentioned.Message ID:
@.***>


Add omega watches

Fixes #3
Changes

Create omega folder to main code and add 5 images to it
Add omega watch model data in script.js for omega images

Fixes #3
Changes

Create omega folder to main code and add 5 images to it
Add omega watch model data in script.js for omega images

Dear Contributor,
Thank you very much for your recent pull request and for the effort you have put into improving the project. I have reviewed the changes you proposed, and overall, the work looks quite promising. However, I have noticed an issue with the images you added in the omega folder. Specifically, one of the images is not displaying correctly due to a punctuation error in the code. This appears to be a minor issue, likely related to incorrect syntax in your script.js file, which is preventing the image from being rendered as expected.
To resolve this, I kindly ask you to review the code again, particularly focusing on the section where you reference the image files in script.js. It’s possible that there is either a missing or misplaced character, such as a comma or quotation mark, which could be causing this error. Once you have corrected it, I would appreciate it if you could test the changes locally to ensure all images display properly.
Afterward, please feel free to resubmit the pull request with the necessary corrections. I will be happy to review it again and proceed accordingly.
Thank you for your attention to this matter, and I look forward to your updated submission.
Best regards,
[Berat Gökdemir]

@brgkdm thank you for your patience and detailed instructions 🙏

@brgkdm thank you for your patience and detailed instructions 🙏

Added.


As an Admin, I want to confirm when I delete a widget or item in a widget.

As a Admin
I want I want to confirm when I delete a widget or item in a widget
so that I don't delete something inadvertently
Acceptance Criteria
Scenario 1:
Admin clicks a "delete" icon and sees a Modal:

h3: Delete
Content: Please confirm you want to delete (item name) from this (widget/survey/Engagement).
"Confirm" button: Deletes the item
"Cancel" button: item is not deleted

Scenario 2:
Confirm user interface to the visual design
https://xd.adobe.com/view/7360b654-4a25-4140-9670-a56f4adabe7a-2633/screen/e355ae52-998f-4cd0-89d7-54ef5edab5ad/specs/
Validation Rules?
(If yes, list here)
Screenshot
(Link above will always be the most up-to-date version.)

Saravan says "make this card more generic."

For the purpose of deleting widget items. I feel this doesn't play well with the 'Save & Close' button. If someone pressed delete on a widget item, saw the model, or pressed confirm but did not press 'save & close' no deletion will happen. Which feels weird to me

@marieanik @saravanpa-aot  could we please discuss this further for the above concern

Checked as a team and this was already implemented, more discussion might be needed in a separated card


Issues in GraphQL complexity analysis

Description:

Complexity analysis in graphql does not support query arguments that are not integers. For example, if we use a ENUM value as the argument of the query it will return the following error.

GraphQLQueryAnalysisHandler java.lang.ClassCastException: graphql.language.EnumValue cannot be cast to graphql.language.IntValue

Complexity analysis only supports queries with arguments that involve limits. For example, a simple query with id: 1000 for the argument will return the complexity limit exceeded error when rate limiting is enabled because this id value is considered as a limit.

eg:
query{
  human(id:1000){
    id
    name
  }
}

Affected Product Version:
3.2.0, 4.0.0
Suggested Labels:
Bug

A new issue  https://github.com/wso2/product-apim/issues/10945 is created to track further improvement so that GraphQL API creators can define preferred keywords (for aguments) to consider as multiplying factor values, via publisher Portal

Reopening to back-port the fix to APIM 3.2.0

Closing after back-porting


Reproducibility

I'm having issues with reproducibility. I run the following basic setup code twice, with fixed seed=0. The code saves the agent model after initialization and again after one step of training. Neither the initialization nor the post-training models are the same between runs. This is despite MinibatchRl.startup() setting the seed using rlpyt.utils.seed.set_seed. Nothing is running on GPU, so this setup should be deterministic, yes? Please let me know if you are aware of any underlying nondeterminism in torch that I'm not aware of, or if this is a real bug/unintended behavior. Models are compared both using diff on the saved files, and using torch.load and manually inspecting model weights (they're not even close).
import torch

from rlpyt.agents.qpg.sac_agent import SacAgent
from rlpyt.algos.qpg.sac import SAC
from rlpyt.envs.gym import make as gym_make
from rlpyt.runners.minibatch_rl import MinibatchRl
from rlpyt.samplers.serial.sampler import SerialSampler

def main():
    seed = 0
    env_name = "MountainCarContinuous-v0"
    batch_T = 4
    batch_B = 1

    agent = SacAgent()

    algo_kwargs = {
        "OptimCls": torch.optim.SGD,
        "optim_kwargs": {"nesterov": True, "momentum": 0.9, "weight_decay": 0},
        "learning_rate": 0.9,
        "min_steps_learn": 1,
        "batch_size": batch_T,
        "replay_ratio": 1,
    }
    algo = SAC(**algo_kwargs)

    sampler_kwargs = {
        "EnvCls": gym_make,
        "env_kwargs": {"id": env_name},
        "eval_env_kwargs": {"id": env_name},
        "batch_T": batch_T,
        "batch_B": batch_B,
        "max_decorrelation_steps": 0,
    }
    sampler = SerialSampler(**sampler_kwargs)

    runner = MinibatchRl(
        agent=agent,
        algo=algo,
        sampler=sampler,
        n_steps=batch_T,
        log_interval_steps=batch_T,
        seed=seed,
    )
    runner.startup()
    torch.save(agent.state_dict(), "init_dict_1") # I replace this with "init_dict_2" on the second run.

    runner.train()
    torch.save(agent.state_dict(), "state_dict_1")

if __name__ == "__main__":
    main()

I have confirmed that torch.manual_seed produced deterministic initializations on my system.

And similarly for numpy and random.

I can't find where the gym env seeds are set, if anywhere, so that might be one bug (https://harald.co/2019/07/30/reproducibility-issues-using-openai-gym/), but I'm not sure why that would cause initialization unreproducability.

Adding
        if seed is None:
            seed = make_seed()
        for env in envs:
            env.seed(seed)
            env.action_space.seed(seed)
            env.observation_space.seed(seed)

on line 45 of rlpyt.samplers.serial.sampler.py to set the gym environment seeds did not cause either to be deterministic. There is still definitely something in the initialization that's nondeterministic. (This fix also required adding a seed method to rlpyt.spaces.gym_wrapper.GymSpaceWrapper which just passed the call to the GymSpaceWrapper.seed method.)

The following does produce deterministic initializations:
import gym
import torch
from rlpyt.agents.qpg.sac_agent import SacAgent
from rlpyt.envs.gym import GymEnvWrapper

env_name = "MountainCarContinuous-v0"
env = GymWrapper(gym.make(env_name))
env.seed(0)
torch.manual_seed(0)
agent = SacAgent()
agent.initialize(env.spaces)
torch.save(agent.state_dict, "state_dict")

So whatever is causing the non-determinism isn't in the gym wrapper or SacAgent. The sampler initializes the agent, so I'll try that next.

Ok now I'm actually getting determinism for both. Removing the env fix gets determinism for only the initial dicts. I'll submit a PR for the gym env seeding.

And finished with 8f36bda2ddc57c5d828eea0d69569f325cb8dc88


Duplicate scheduling possible when scheduling while previous scheduling request is still being processed by Opencast

Describe the bug
The Stud.IP Opencast Plugin allows one-click scheduling. A lecturor can click an icon (a camera symbol) in a table of events. When they click the symbol in quick succession, two or more Episodes end up to be scheduled for the same time and recorder/space. This confuses our recorders and recordings abort.
To Reproduce
Steps to reproduce the behavior:

Go to stable.opencast.org
Log on as admin
Open your browser's JS console
Fill the following and press enter

for (i = 0; i < 10; i++) {
  jQuery.post('https://stable.opencast.org/recordings/capture/LS-860_050030', { 'workflowDefinitionId': 'fast' });
}

See result:

This is data from our systems:
MariaDB [studip]> select capture_agent, start, end, date_id from oc_scheduled_recordings where seminar_id =  '07dd9e1c81af9129998f4b893989d798' and date_id = '9470933d0d379863f56857d8f03df6dc';
+------------------+------------+------------+----------------------------------+
| capture_agent    | start      | end        | date_id                          |
+------------------+------------+------------+----------------------------------+
| ncast_ITZ_master |<PHONE_NUMBER> |<PHONE_NUMBER> | 9470933d0d379863f56857d8f03df6dc |
| ncast_ITZ_master |<PHONE_NUMBER> |<PHONE_NUMBER> | 9470933d0d379863f56857d8f03df6dc |
+------------------+------------+------------+----------------------------------+
2 rows in set (0.00 sec)

Expected behaviour: Only one event scheduled for one time/recorder. Consecutive events being rejected as conflicts.
Environment Information

OS: Ubuntu 20 LTS
Browser Firefox
Opencast version with minor release 11.4

Opencast conflict detection still does not work as expected. We could document the fact, though, that as long as the index is not updated, conflicting events can be scheduled and I'd be fine with that.


Security Disclosure: Exposing Secrets via Git Repos

This issue was created in response to a report to our security mailer.
Internal reference for this disclosure is NAUTOBOT-730.
In the belief of strong transparency, we record this here to publicly acknowledge our receipt of this disclosure, our investigation, our summary, and ultimate outcome.
The Report
Environment

Python version: N/A
Nautobot version: 1.2.11

I'm working with Nautobot v1.2.11, and found a way to expose secrets stored
as environment variables to a user in the web browser.  This is done
through error logs when attempting to sync a private Git repository with
invalid credentials, due to Nautobot masking certain secret types
(password, token, etc) but not others (username).  I can leverage this to
expose sensitive environment variables when set as the username instead of
token/password.  I have not tested this with other versions of Nautobot.
Steps to Reproduce

In Nautobot Secrets, create a new Secret with the following necessary
information.  Name is irrelevant:

git-user
Provider: Environment Variable
Variable: NAUTOBOT_DB_PASSWORD (or any other environment variable
you want to expose)

Create a second secret with any environment variable. This one is
irrelevant.

git-pass
Provider: Environment Variable
Variable:

Create a Secret Group.  Name is irrelevant.

Name: Group1
Secret 1:

Access Type: HTTP(S)
Secret Type: Username
Secret: nb-user

Secret 1:

Access Type: HTTP(S)
Secret Type: Token
Secret: nb-pass

Create a new Git repository.  Name is irrelevant.

Name: Repo1
Remote URL: https://github.com/networktocode-llc/website.git

NOTE: This can be any valid private Git repo that requires
authentication.  Another example repo that works with this
from a customer:

Branch: main
Username/Token:
Secrets Group: Group1
Provides:

Upon saving, the repo tries to sync the repo from GitHub (or Azure, etc),
then immediately throws an authentication failure.  The error logs replace
"token" and "password" secrets in the URL with asterisks, however it
displays the "username" secret in the URL in cleartext.
I can then go back to the "git-user" secret, change it to any other
environment variable, and rerun the failed Sync job for the repo, and that
secret will also be exposed.  Screenshot attached as an example.
Note: This does not work if the "git-pass" secret is set as a Secret
Type: Password instead of Secret Type: Token.  It only appears to work for
tokens.

The Analysis
The usage of secrets in Nautobot does come with an advisory note: https://nautobot.readthedocs.io/en/stable/core-functionality/secrets/#leakage-of-secret-values
Essentially anyone and anything that does have access to Nautobot could potentially access secrets in unsafe ways.
The root of this issue stems from GitPython logging the "username" portion of the accessing repo address in the error output unsanitized. It does strip the password portion of the url before emitting the log. There is a feature request in GitPython to change that behavior: https://github.com/gitpython-developers/GitPython/issues/1284
Our Summary
A user permitted to administer secrets and Git repository can modify the source of the secret providing the username as well as create or modify an existing repository to always throw an error, retrieving any secret value in plain text.
This chaining of misconfigurations was not well documented. In the case of some Git remote repositories allowing token-only authentication (GitHub being one of them), the secret that would ideally always be obfuscated could always be exposed since this commonly gets applied as a "username" in the URL structure.
Going Forward
Actions we have taken:

We updated our advisory message in the Secrets section to express the reason for using object level permissions to minimize user's access to certain sensitive secrets as well as the exploit potential in: https://github.com/nautobot/nautobot/pull/1751
We contributed a pull request to GitPython to close https://github.com/gitpython-developers/GitPython/issues/1284 which will strip the username portion of the Git url in error messages from that library in: https://github.com/gitpython-developers/GitPython/pull/1437

We are awaiting a new release of GitPython which includes this fix and will update Nautobot accordingly. At the time of this publishing it still has yet to be release.

We introduced Job Log filtering in Nautobot v1.3.4

This will immediately address the error messages from GitPython
This feature is user configurable to either restore existing functionality or add additional rules for stripping out secrets from log messages

This issue affects all versions of Nautobot between 1.2.0 and 1.3.3, however as mentioned above a future update of GitPython could be pinned in earlier Nautobot releases to address the root of the issue in GitPython's error message.

If you believe our analysis or response is insufficient, or have additional questions or concerns, please do not hesitate to follow up.

Closed via v1.3.4, #1715.


Masonry

Cascading grid layout library - http://masonry.desandro.com/

I think we'll pass on this one. Thanks for the suggestion.


Add option to change beatmaps difficulty settings

Would be cool to be able to change AR, OD etc. without editing the .osu file before parsing with rosu-pp.
Another good addition would be being able to also change the "speed rate" of the map. e.g. setting it to something like 1.2x instead of being limited to only DT/HT or 1.5x/0.75x respectively.

Attributes like AR and OD are public fields in the Beatmap struct so after parsing you can adjust their values.
Customizable clock rates sound alright. They require a little bit of restructuring of the user interface for difficulty calculation but I think that was overdue anyway.

Attributes like AR and OD are public fields in the Beatmap struct so after parsing you can adjust their values.

I probably should have stated that I'm using rosu-pp-js. Is that something that's also doable there? If so I wasn't able to figure out how. The readme only states the path field and changing the map attributes in the params doesn't work.

Customizable clock rates sound alright. They require a little bit of restructuring of the user interface for difficulty calculation but I think that was overdue anyway.

Main reason I'm bringing this up is for calculating pp on lazer scores using custom rates. So this would be really helpful

I probably should have stated that I'm using rosu-pp-js. Is that something that's also doable there? If so I wasn't able to figure out how. The readme only states the path field and changing the map attributes in the params doesn't work.

Oh yeah in there it's not currently possible.
I didn't really plan on extending rosu-pp-js functionality but simply accepting a few more fields for the params is really easy to do so I guess I can do that 👍

Oh yeah in there it's not currently possible. I didn't really plan on extending rosu-pp-js functionality but simply accepting a few more fields for the params is really easy to do so I guess I can do that :+1:

should also add it to the python bindings then :smile: :+1:

Can be closed as of version v0.5.1 :+1:


Document how to enable vSphere CSI driver in storage section

We currently have these files in the v2.5.6 branch:

{{}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/enable-out-of-tree-vsphere-provider-on-new-cluster/
{{}}/rancher/v2.x/en/cluster-provisioning/rke-clusters/node-pools/vsphere/migration/

We also need to add a link or the guide onto the storage section.

https://github.com/rancher/docs/commit/1f9db95bf3f73090709599041895bfd7768fcd1c


Conditional toggle between two alternative fields with checkbox or radios

I have used a few methods to display conditional fields with radios and checkbox from this branch. However, I cannot see that it is possible to display conditional fields "A or B" based on radios or checkbox like it works with selectfieldset:
http://ulion.github.io/jsonform/playground/?example=fields-selectfieldset
So basically, I would be looking to replace the select with either boolean radios or checkbox, to display either key A or key B. As far as I can understand, with radios and checkbox, it is only possible to toggle display of keys, not toggle between two different keys.
Perhaps there is some trick unknown to me? Thanks for this branch btw.

Then that is simply not supported currently. PR is welcome.
2015-08-18 22:33 GMT+08:00 Karl Ward<EMAIL_ADDRESS>
I have used a few methods to display conditional fields with radiosand
checkboxfrom this beanch. However, I cannot see that it is possible to
display conditional fields "A or B" based on radios or checkbox like it
works withselectfieldset`:
http://ulion.github.io/jsonform/playground/?example=fields-selectfieldset
So basically, I would be looking to replace the select with either
boolean radios or checkbox, to display either key A or key B. As far as I
can understand, with radios and checkbox, it is only possible to toggle
display of keys, not toggle between two different keys.
Perhaps there is some trick unknown to me? Thanks for this branch btw.
—
Reply to this email directly or view it on GitHub
https://github.com/ulion/jsonform/issues/24.

--
Ulion


deprecate?

Deprecate the npm package and archive this repo?

Is there a maintained alternative for this package?

As stated in the readme?

We published the successor module to this at https://github.com/fuse-friends/fuse-native


Reversal Index Sort Order

When there is more than one Reversal Index, the sort order for both indexes seems to be taken from the first Reversal that uploads.
I was testing sort orders and I reversed the sort order for the English reversal in FLEx, leaving the Spanish one correct, but after uploading, both Reversal Indexes were reversed. Refer to when the Entry Count for Reversal indexes was taken from the first Reversal Index that uploaded.
https://www.webonary.org/test-stutzman/browse-indonesian/

https://www.webonary.org/test-stutzman/browse/browse-english/

@vernastutzman This is how FLex is sending the reversal order. So it appears that FLex is sending the wrong order. Once this is fixed in FLex, and the data is reuploaded to Webonary, it should work correctly.


Tables unstyled

Tables have reverted to an unstylised state.

Tables remain to the left-hand side of the screen as opposed to the centre.
Highlighting no longer works.

This has been fixed in the latest update; enjoy!


Displaying sender address

When reading e2ee messages, the senders address is displayed.

:tada: This PR is included in version 2.35.1 :tada:
Your semantic-release bot :package::rocket:


Compile Litl library on ARM architecture

Hi @HugoGuiroux
I'm trying to compile Litl library on ARM architecture and replace #include <xmmintrin.h> header file with sse2neon
(sse2neon see https://github.com/DLTcollab/sse2neon/)
The error encountered in CLHT/include/atiomic_ops.h is as follows:
error: impossible constraint in ‘asm’ asm volatile("xchgb %0,%1"
How to solve the above problems to complete the compilation and operation of Litl on the ARM architecture.
Please let me know if you have a solution, thanks a lot.

Hi @HugoGuiroux I'm trying to compile Litl library on ARM architecture and replace #include <xmmintrin.h> header file with sse2neon (sse2neon see https://github.com/DLTcollab/sse2neon/) The error encountered in CLHT/include/atiomic_ops.h is as follows: error: impossible constraint in ‘asm’ asm volatile("xchgb %0,%1" How to solve the above problems to complete the compilation and operation of Litl on the ARM architecture. Please let me know if you have a solution, thanks a lot.

Thats x86_64 inline assembly. This library is not really portable cross arch. Your going to have to at the very least replace all the inline assembly with the corresponding aarch64 code and probably fix a litany definition of bugs due missing memory barriers.

Hi @HugoGuiroux I'm trying to compile Litl library on ARM architecture and replace #include <xmmintrin.h> header file with sse2neon (sse2neon see https://github.com/DLTcollab/sse2neon/) The error encountered in CLHT/include/atiomic_ops.h is as follows: error: impossible constraint in ‘asm’ asm volatile("xchgb %0,%1" How to solve the above problems to complete the compilation and operation of Litl on the ARM architecture. Please let me know if you have a solution, thanks a lot.

Thats x86_64 inline assembly. This library is not really portable cross arch. Your going to have to at the very least replace all the inline assembly with the corresponding aarch64 code and probably fix a litany of bugs due missing memory barriers.
Edit: If you do go about porting this to aarch64 can you ping back here with the fork?

I have solved the above problems, compiled and generated .sh files, but reported errors during use
'undefined symbol: CAS_U64'
Similar errors are as follows:
'undefined symbol: PREFETCHW'
'undefined symbol: getticks'
'undefined symbol: _mm_mfence'
'undefined symbol: FAI_U32'
I tried to modify the following 'platform-specific definitions' in line 205 in litl/obj/CLTH/ssmem/include/ssmem.h to define the corresponding functions according to the ARM architecture
But the error is still not resolved

The runtime errors are because the code is compiled down to a shared library and its expecting to find those symbols in the executable your running.
You need to fine where each of them is defined and re-define them with the corresponding aarch64 instructions.

The runtime errors are because the code is compiled down to a shared library and its expecting to find those symbols in the executable your running.
You need to find where each of them is defined and re-define them with the corresponding aarch64 instructions.

Thanks, the problem has been solved

Edit: If you do go about porting this to aarch64 can you ping back here with the fork?
Hi @goldsteinn,I've done porting code from x86 to ARM,But it includes modifications to ssmem and sspfd under CLHT. If I upload a new branch, the above two libraries will also be uploaded together

I see, any chance you can post the patches from master/HEAD or just push a fork without those two library and references to the versions?
If not cest la vie.

The runtime errors are because the code is compiled down to a shared library and its expecting to find those symbols in the executable your running.
You need to find where each of them is defined and re-define them with the corresponding aarch64 instructions.

Thanks, the problem has been solved

Would you please kindly share your _mm_mfence implement file?

The runtime errors are because the code is compiled down to a shared library and its expecting to find those symbols in the executable your running.
You need to find where each of them is defined and re-define them with the corresponding aarch64 instructions.

Thanks, the problem has been solved

Would you please kindly share your _mm_mfence implement file?

_mm_mfence is an intrinsic that causes the compiler to emit the mfence x86 instruction. You will need to be careful porting the code around there as the x86 memory model is significantly more forgiving than the arm memory model (to a degree that most people write buggy code that only really works on x86).

https://github.com/DLTcollab/sse2neon/issues/554#issuecomment-1328657516
problem sovled


Parameter smoothing

Control-rate inputs are sampled at block rate (1ms by default), which can lead to audible warbling-stepping according to what smoothing is happening in the hardware.
It could be good to include a few subpatcher examples of different ways to smooth these (onepole, ramplimit, lowshelf, etc).
Additionally, adding the ability to use [in cv1] instead of [param cv1] etc. might be a way to automatically provide smoothed input.

oopsy.ctrl.smooth2 and oopsy.ctrl.smooth3 implement 2-pole/3-pole smoothing which cuts pretty much all the noise on CV/pots on Patch. 3-pole is snappier.


Add more Sphinx documentation

Description
Please explain the changes you've made
I generated more documentation using Sphinx-ApiDoc. It automatically generates documentation for each modules and their sub-modules. I extended the doc command in tox.ini. This may not be necessary because you can always generate the .rst files and commit them. I also added steps to generate documentation in the README.md.

Issue reference
We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.
Please reference the issue this PR will close: #41
I also noticed that there are upcoming tasks such as

Create convenient layer wrapping gRPC and HTTP clients for Dapr
Flask extensions for Dapr State/Pubsub/Bindings

Could I try to tackle them?
Checklist
Please make sure you've completed the relevant tasks for this PR, out of the following list:

[X] Code compiles correctly
[ ] Created/updated tests
[X] Extended the documentation

Thanks for your contribution. we will take this into next milestone.

@chinzhiweiblank Thanks for your contribution. please fix the merge-conflict.

Create convenient layer wrapping gRPC and HTTP clients for Dapr
Flask extensions for Dapr State/Pubsub/Bindings

@chinzhiweiblank  Thanks for your contribution. We need to have the design doc before working on it. I will file two issues to discuss the design.


Add Support for Donate from Event/Project in Contribution Flow

User story
It might be useful for collective admins to be able to use the contribution flow to transfer money from a Project/Event to it's parent collective.
Best solution for this problem
Add the ability for collective admins to choose the events or projects in the Contribute as section in the contribution flow.

Documentation
We also need to align our documentation with these changes; specifically the section https://opencollective.slack.com/archives/C0RMV6F8C/p1626725166268200
Relevant Slack Discussion: https://opencollective.slack.com/archives/C0RMV6F8C/p1626725166268200

Thinking further I am a bit dubious about adding the fund transfer support for events and projects in the contribution flow. Makes me think the contribution flow is more towards external users. if we do add it we will have to probably remove the platform tip (as it doesn't make sense to add platform tips from event to parent collective transfer). Also maybe remove the final success page; since it doesn't seem to make sense to have the fireworks for internal contributions. For these reasons my thinking is only keep these internal transfers in the settings section as suggested in; https://github.com/opencollective/opencollective/issues/4468
Let me know what you think. 😄
cc: @Betree @opencollective/core

I don't this it really makes sense as a contribution, because money in an Event or Project already belongs to the Collective. However, we do need a better way to move money between Events, Projects, and the main budget, in both directions.

I don't this it really makes sense as a contribution,

@alanna : You meant I don't think... right?
Agreed, I think this needs design in order to proceed and/or further discussion. Adding this support by just adding the event in the payee screen might not be the best experience. So probably let us wait until we can come up with a better way to handle this. 🤔
@Betree : Let me know if you have any thoughts on this one. 😄

we do need a better way to move money between Events, Projects, and the main budget, in both directions.

was, I believe, the original intent for this.

@suds can we change this to 'improve the process for moving money between events/projects and parent collectives. In my mind I think we should do something through the contribution flow that prioritises the available contributors, maybe being a little more intentional in the language (mock up tbd)

@SudharakaP can we change this to 'improve the process for moving money between events/projects and parent collectives. In my mind I think we should do something through the contribution flow that prioritises the available contributors, maybe being a little more intentional in the language (mock up tbd)

@BenJam : Sure, I've re-titled this one. Yes my thoughts as well. 👍🏽

Just to note maybe part of what's needed here (transfer from event/project -> parent collective) is done at, https://github.com/opencollective/opencollective/issues/4468 (demo at, https://github.com/opencollective/opencollective-frontend/pull/6670#issue-695453855). My thinking is that what's probably we need to focus here is the forward transfer (parent collective -> event/project) probably. Anyways, feel free to let me know your thoughts. 🤔


Adding POSTGRES_HOST_AUTH_METHOD to docker-compose.yml

We have Pongo running in our CI environment, and recently started getting weird build failures out of nowhere with the following error:
./spec/helpers.lua:115: [PostgreSQL error] failed to retrieve server_version_num: host or service not provided, or not known
On closer inspection, we realised the digest for the Postgres Docker Image was different in the failing build.
They introduced a breaking change on Friday for security measures. You can see the discussion thread in the docker-library repo here: https://github.com/docker-library/postgres/issues/681
Anyway, I followed their recommendation and added the POSTGRES_HOST_AUTH_METHOD env variable to the docker-compose.yml. This might be useful for you

thx for the heads-up and the fix! @C13490888


Idea: should we automatically run all experiments every 15 days and publish it ?

Maybe for people who have limited computational power, it would be great if the list of experiments and frozenexperiments was rerun every 15 days ? so that when people add an algorithm, they automatically get an assessment.
(cf. discussion with Dimo, and discussion with Carola + Diederick + Hao)
Proposal:
(A) every 15 days Nevergrad creates all plots and the CSV files in a webpage
(B) when the user runs an experiment, a CSV file is created (this is already implemented) and Nevergrad
prints
(1) the path to this CSV file    (this is already the case in nevergrad I think, so not much to do except cosmetics)
(2) the link to the webpage of CSV files (exported as in (A))
(not necessary for IOH if IOH automatically finds CSV files)
(3) a link to IOH profiler page
(C) naming convention: the naming of the file should include as a prefix an identifier of the experiment, so that the list of files will show all CSV files which are comparable
In terms of coding:

on the Nevergrad side:

automatic rerun every 15 days or so
check that we display the path to the CSV file in a cool manner
the name should include a specification of the experiment (it's already the case,
maybe we also specify if it's in frozenexperiments.py)
display how to view results (it's already the case for the png files, we just also provide the link to IOH)

on the IOH side:

ensure that people can upload a bunch of CSV (or a concatenated CSV ?) and plot them all together
should we also ensure that IOH identifies, in the repository, existing CSV files with the same prefix and adds them automatically ?

@brockho

Now implemented in the dashboard though not really 15 days :-)
Dashboard: https://dl.fbaipublicfiles.com/nevergrad/allxps/list.html


Customization of icons for tiles

[ ] Create a new configuration file
[ ] Add tags like so [WALL_ICON:35] // '#' by default
[ ] The tag name should match a list in the graphics class
[ ] The value will be the ASCII index (i.e. 65 for 'a')
[ ] Change SpriteComponent to use the Graphics enum value, instead of a char
[ ] Graphics class should only allow enum value to be drawn (with co-ordinates)
[ ] Enum value will then use the correct character based on the value in the config file
[ ] Defaults will be hard-coded, but overriden by the config file

Or this could be customized in the PreFab files

Or, leave it as noted above, but use number mapping. So essentially map 35->22.
That way you can override the sprite in the PreFab files by only changing a single, unrelated file.


Rename "Useful Links" to something more useful

Feedback from beta tester

Renamed to "uO Info" for University of Ottawa app.


Logging Not Registering In Flask 0.12

I'm having some trouble getting logging to work with Raven. I noticed today that the handler gets set up properly if I run Flask 0.11.1, but not on 0.12. I made some example code below which is able to reproduce the issue. I've been running this on Python 3.5 with just Flask and Raven installed, Raven version 5.32.0.
Symptoms

Sentry events not reported for the below code, upon hitting the / route, using Flask 0.12, while events show for the same conditions using Flask 0.11.0

Expected Outcome

Sentry events should be reported across Flask versions

import logging

from flask import Flask
from raven.contrib.flask import Sentry

sentry = Sentry()

def create_app():
    app = Flask(__name__)

    app.config['DEBUG'] = True

    sentry.init_app(
        app,
        dsn='https://<redacted>',
        logging=True,
        level=logging.WARNING
    )

    def internal_server_error(exception):
        app.logger.exception(exception)
        return 'Internal Server Error'

    app.errorhandler(Exception)(internal_server_error)

    return app

app = create_app()

@app.route('/')
def test_route():
    raise Exception('TEST')

if __name__ == '__main__':
    app.run()

@mitsuhiko did you change something in Flask 0.12 around the logging bits?

This code above should never have worked. The correct way to register server error handler has always been @app.errorhandler(500) and that continues to work. It also always logs.
If you catch down an Exception it means the error is "handled" and the default error logging is disabled. I suppose this is why an attempt is made here to log with app.logger.exception. However since the exception is an object and not a string i suppose something might break.
I will have a look at this but the correct way is to use app.errorhandler(500) and from a quick look this works.

It appears swapping out to using app.errorhandler(500) does in fact work as expected with Flask 0.12. In Flask 0.11.1, the way I posted above was totally functional with Sentry's logging module, hence the surprise when it stopped working under 0.12. Nevertheless, closing as this is resolved.
Thanks!

@mitsuhiko - apparently I was too quick to close this. You are definitely correct regarding the error handler, however the hook into the Flask application logger still appears to be broken. I have updated my sample application and pasted below. Once again, on Flask v0.11.1, upon hitting /log I get an error in Sentry, however when I updated Flask to v0.12, I no longer saw an error appear in Sentry upon hitting that route.
import logging

from flask import Flask, jsonify
from raven.contrib.flask import Sentry

sentry = Sentry()

def create_app():
    app = Flask(__name__)

    app.config['DEBUG'] = True

    sentry.init_app(
        app,
        dsn='https://<redacted>',
        logging=True,
        level=logging.DEBUG
    )

    @app.errorhandler(500)
    def internal_server_error(exception):
        return 'Internal Server Error'

    return app

app = create_app()

@app.route('/log')
def test_log():
    app.logger.error('TEST ERROR')
    return jsonify({'message': 'ok'})

@app.route('/')
def test_route():
    raise Exception('TEST')

if __name__ == '__main__':
    app.run()

Is there a specific reason you are running this in debug mode?

Nope, just my default for development. Turning off DEBUG and restarting the app does not alter the behavior.

I wonder if we currently require blinker to be installed. I cannot reproduce your behavior here. I will try a bit later with latest release versions though.

Interesting that you cannot reproduce. In case it helps, I'm using a bare virtualenv, with Python 3.5 on OSX 10.12.2, and have only explicitly installed Flask, and raven[flask] v5.32.0 in the virtualenv.
Below is a readout of all the packages/dependencies in the virtualenv:
$ pip3 freeze
blinker==1.4
click==6.7
contextlib2==0.5.4
Flask==0.12
itsdangerous==0.24
Jinja2==2.9.5
MarkupSafe==0.23
raven==5.32.0
Werkzeug==0.11.15
wheel==0.24.0

Hey, I'm experiencing the same issue but using python 2.7.13:
appdirs==1.4.0
blinker==1.4
click==6.7
contextlib2==0.5.4
Flask==0.12
itsdangerous==0.24
Jinja2==2.9.5
MarkupSafe==0.23
packaging==16.8
pyparsing==2.1.10
raven==5.32.0
six==1.10.0
Werkzeug==0.11.15

Tried setting app.logger.propagate = True and it worked as expected. Got it from https://github.com/pallets/flask/commit/e00e2c22aadf9e5b9dc353c31f1b70dad488b47a

I had the same issue in Raven 6.0.0 and Flask 0.12 (logging to app.logger doesn't propagate).
@jm66 's fix worked for me, however.


Could not find com.github.nomi9995:VideoCompressor:715bcc16e9.

I cant build the app after npm install react-native-compressor@rnlessthan65. I have done a gradelw clean, removed node_modules and reinstalled.
Current behavior
When running app with npx react-native run-android I get a build error of:
Could not resolve all task dependencies for configuration ':app:debugRuntimeClasspath'.

Could not find com.github.nomi9995:VideoCompressor:715bcc16e9.
Searched in the following locations: .........

Expected behavior
App Builds
Platform

[X] Android
[] iOS

React Native Version
0.60.4
React Native Compressor Version
Reproducible Steps And Demo

@lukeTheWizard can you add this line into build.gradle > allprojects > repositories and test again.
maven { url 'https://jitpack.io' }

@nomi9995 that fixes the compilation issues. However Im now getting a runtime error:
error: bundling failed: SyntaxError: /REDACTED/node_modules/react-native-compressor/src/Audio/index.tsx: Unexpected token, expected ")" (62:18)
60 |         });
61 |       }

62 |     } catch (error: any) {
|                   ^
63 |       throw error.message;
64 |     }
65 |   },

@lukeTheWizard are you trying to compress audio?

No just a regular image. The error pops up from just importing. Could this be a node version or something? My project isn't typescript either

@lukeTheWizard which version of node, you are using?


Whitespace Control with for loop and nested if statement

I am not able to control the vertical spacing/whitespace when I have a template that has an if statement inside a for loop that ends up skipping some of the items in the list. I have the Pebble Engine configured to disable New Line Trimming since I see no way to get the vertical spacing that I want with New Line Trimming enabled.
I have made a JUnit test to demonstrate the issue. The example template as well as expected template output and actual template output is shown below. I will submit a pull requests with the test code. One test is failing due to this issue. That test is marked with @Ignore so that adding the tests does not break the build.
Template:
text before for loop followed by blank line
{% for item in items %}
{% if item.itemType equals "ITEM_TYPE1" -%}
Item 1
{% elseif item.itemType equals "ITEM_TYPE2" -%}
Item 2
{% endif -%}
{% endfor %}
text after for loop preceded by blank line

Expected Template Output:
text before for loop followed by blank line

Item 1

Item 2

text after for loop preceded by blank line

Actual Template Output:
text before for loop followed by blank line

Item 1

Item 2

text after for loop preceded by blank line

This is a big problem for me since I am using Pebble for code generation. This results in generated code such as the following, which is a real problem for generated code. I will be glad to work on fixing this issue in the Pebble code if you confirm that this is an issue. If there is another way to get the desired output in a Pebble template, please let me know.
public class SchoolModel implements Serializable {

    private static final long serialVersionUID = 1L;

    private Long id;

    public String identifier;

    @NotNull
    private String abbreviation;

    private LocalTime goHomeNotesDailyCutoffTime;

    private LocalDate goHomeNotesStartDate;

    private LocalDate goHomeNotesStopDate;

    @NotNull
    private String name;

    public String getIdentifier() {
        return this.identifier;
    }

    public void setIdentifier(String identifier) {
        this.identifier = identifier;
    }

    public Long getId() {
        return id;
    }

    public void setId(Long id) {
        this.id = id;
    }

    public String getAbbreviation() {
        return this.abbreviation;
    }

    public void setAbbreviation(String abbreviation) {
        this.abbreviation = abbreviation;
    }

    public LocalTime getGoHomeNotesDailyCutoffTime() {
        return this.goHomeNotesDailyCutoffTime;
    }

If you use this template, it's gonna work. I've modified the {%- endif -%} and {% endfor -%}
text before for loop followed by blank line
{% for item in items %}
{% if item.itemType equals "ITEM_TYPE1" -%}
Item 1
{% elseif item.itemType equals "ITEM_TYPE2" -%}
Item 2
{%- endif -%}
{% endfor -%}
text after for loop preceded by blank line

Cool! Yea, that works for the test case (I have not tested it with my real scenario yet). Can you explain why? I have many cases that I do this or something similar. I need to be able to think clearly about the whitespace behavior.

It seems that I have to specify the whitespace control differently depending on if I have one, two, or three "elseif" statements in order to get the same spacing in the output.
Template with one elseif statements:
text before for loop followed by blank line
{% for item in items %}
{% if item.itemType equals "ITEM_TYPE1" -%}
Item 1
{% elseif item.itemType equals "ITEM_TYPE2" -%}
Item 2
{%- endif -%}
{% endfor -%}
text after for loop preceded by blank line

Template with two elseif statements:
text before for loop followed by blank line
{% for item in items %}
{% if item.itemType equals "ITEM_TYPE1" -%}
Item 1
{% elseif item.itemType equals "ITEM_TYPE2" -%}
Item 2
{% elseif item.itemType equals "ITEM_TYPE3" -%}
Item 3
{%- endif -%}
{% endfor %}
text after for loop preceded by blank line

Template with 3 elseif statements:
text before for loop followed by blank line
{% for item in items %}
{% if item.itemType equals "ITEM_TYPE1" -%}
Item 1
{% elseif item.itemType equals "ITEM_TYPE2" -%}
Item 2
{% elseif item.itemType equals "ITEM_TYPE3" -%}
Item 3
{% elseif item.itemType equals "ITEM_TYPE4" -%}
Item 4
{% endif -%}
{% endfor %}
text after for loop preceded by blank line

I have over 100 templates with various nested template structures. I need to be able to have good control over the spacing and whitespace in the output. I will work on addressing this in the Pebble code once I am sure that this is an issue and not just a misunderstanding on my part.

Seems like a bug

I will be glad to work on this. I started looking at the code. I see that the whitespace control is done in the LexerImpl. I will try to make unit tests that isolate the issue at a lower level than the current template tests. Maybe I can make unit tests for the LexerImpl. Not sure yet.
I submitted a pull requests to add the tests that I made so far (which are template integration tests rather than true unit tests). No matter if this issue is addressed or not, I think these are useful tests. Will you review and accept the pull request or let me know if you have any issues with it?

This appears to be a design issue in that the Lexer processes the Whitespace Control Character (i.e. the dash character) and the remaining new line characters (i.e. the \n) are passed to the Parser as part of a Text Token. Therefore, when the Whitespace Control Character is used on an if statement, the output will be affected only if the condition in the if statement is true. For an if statement in a for loop, the new line characters in the output will vary depending on which iterations result in the body of the if statement being executed or not.
I think that the responsibilities between the Lexer and Parser need to be modified. My first thought would be for the Whitespace Control Character to be treated as a separate token (Operator Token maybe?) and for the new line characters to be included as separate TEXT tokens or perhaps a new type of token. In other words, all of the information would be passed to the Parser as Tokens in the Token Stream and so it would be the Parser's responsibility to remove the new line tokens or not based on the Whitespace Control Characters.
This is important to me. So, I will work on this and I will add unit tests for the Lexer and the Parser separately and possibly other tests at various levels. I would like to know if you have any concerns about me working on it. I will be open to feedback when I make pull requests. Do you want me to make pull requests for small changes? Would that be easier to review?
The following Lexer unit tests that I have written shows how the whitespace control character on an if statement affects the Text Token, which will be passed to the Parser.
  /*
   *
		{% if item equals "whatever" %}
		some text
		{% endif %}
   */
  @Test
  public void testIfStatement() {
	ClasspathLoader loader = new ClasspathLoader();
	Reader templateReader = loader.getReader("templates/lexer-tests/IfStatement.peb");

	TokenStream tokenStream = this.lexer.tokenize(templateReader, TEMPLATE_NAME);

	assertThat(tokenStream.getTokens().size()).isEqualTo(11);
	int i = 0;

	assertThat(tokenStream.peek(i).getType()).isEqualTo(Type.EXECUTE_START);
	assertThat(tokenStream.peek(i++).getValue()).isNull();
	assertThat(tokenStream.peek(i).getType()).isEqualTo(Type.NAME);
	assertThat(tokenStream.peek(i++).getValue()).isEqualTo("if");
	assertThat(tokenStream.peek(i).getType()).isEqualTo(Type.NAME);
	assertThat(tokenStream.peek(i++).getValue()).isEqualTo("item");
	assertThat(tokenStream.peek(i).getType()).isEqualTo(Type.OPERATOR);
	assertThat(tokenStream.peek(i++).getValue()).isEqualTo("equals");
	assertThat(tokenStream.peek(i).getType()).isEqualTo(Type.STRING);
	assertThat(tokenStream.peek(i++).getValue()).isEqualTo("whatever");
	assertThat(tokenStream.peek(i).getType()).isEqualTo(Type.EXECUTE_END);
	assertThat(tokenStream.peek(i++).getValue()).isEqualTo("%}");

	assertThat(tokenStream.peek(i).getType()).isEqualTo(Type.TEXT);
	assertThat(tokenStream.peek(i++).getValue()).isEqualTo("\nsome text\n");

	assertThat(tokenStream.peek(i).getType()).isEqualTo(Type.EXECUTE_START);
	assertThat(tokenStream.peek(i++).getValue()).isNull();
	assertThat(tokenStream.peek(i).getType()).isEqualTo(Type.NAME);
	assertThat(tokenStream.peek(i++).getValue()).isEqualTo("endif");
	assertThat(tokenStream.peek(i).getType()).isEqualTo(Type.EXECUTE_END);
	assertThat(tokenStream.peek(i++).getValue()).isEqualTo("%}");

	assertThat(tokenStream.peek(i).getType()).isEqualTo(Type.EOF);
  }

  /*
   *
		{% if item equals "whatever" %}
		some text
		{% endif %}
   */
  @Test
  public void testIfStatementWithWhitespaceControl() {
	ClasspathLoader loader = new ClasspathLoader();
	Reader templateReader = loader.getReader("templates/lexer-tests/IfStatementWithWhitespaceControl.peb");

	TokenStream tokenStream = this.lexer.tokenize(templateReader, TEMPLATE_NAME);

	assertThat(tokenStream.getTokens().size()).isEqualTo(11);
	int i = 0;

	assertThat(tokenStream.peek(i).getType()).isEqualTo(Type.EXECUTE_START);
	assertThat(tokenStream.peek(i++).getValue()).isNull();
	assertThat(tokenStream.peek(i).getType()).isEqualTo(Type.NAME);
	assertThat(tokenStream.peek(i++).getValue()).isEqualTo("if");
	assertThat(tokenStream.peek(i).getType()).isEqualTo(Type.NAME);
	assertThat(tokenStream.peek(i++).getValue()).isEqualTo("item");
	assertThat(tokenStream.peek(i).getType()).isEqualTo(Type.OPERATOR);
	assertThat(tokenStream.peek(i++).getValue()).isEqualTo("equals");
	assertThat(tokenStream.peek(i).getType()).isEqualTo(Type.STRING);
	assertThat(tokenStream.peek(i++).getValue()).isEqualTo("whatever");
	assertThat(tokenStream.peek(i).getType()).isEqualTo(Type.EXECUTE_END);
	assertThat(tokenStream.peek(i++).getValue()).isEqualTo("%}");

	assertThat(tokenStream.peek(i).getType()).isEqualTo(Type.TEXT);
	assertThat(tokenStream.peek(i++).getValue()).isEqualTo("some text");

	assertThat(tokenStream.peek(i).getType()).isEqualTo(Type.EXECUTE_START);
	assertThat(tokenStream.peek(i++).getValue()).isNull();
	assertThat(tokenStream.peek(i).getType()).isEqualTo(Type.NAME);
	assertThat(tokenStream.peek(i++).getValue()).isEqualTo("endif");
	assertThat(tokenStream.peek(i).getType()).isEqualTo(Type.EXECUTE_END);
	assertThat(tokenStream.peek(i++).getValue()).isEqualTo("%}");

	assertThat(tokenStream.peek(i).getType()).isEqualTo(Type.EOF);
  }

@nward1234 Go on this one. I'll gladly accept your PR for this

Great! I submitted a pull request for a few "template/functional" tests previously and that pull request is still pending. Would you be willing to approve that pull request to start with? (Or, let me know if you have any issues with it).
https://github.com/PebbleTemplates/pebble/pull/477
I will then submit another pull requests for these Lexer Impl tests and then continue working on this change probably with more tests at various levels before I get into changing the processing.

I made a comment yesterday on your PR

I checked the pull request comments again just now. The last comment that I see from you was when the build failed 15 days ago (which is now passing).

Sorry forgot to submit the review. You should see it now

Pull request is merged, why not close the issue?

Pull request is merged, why not close the issue?


Use of deprecated function

During a PR build, something came up.
On 7.2, PHP throws warnings because of the use of the deprecated count() function in this codebase.
See: TravisCI build
Please patch this code so we don't use count() any longer.

Fixed in PR#12


fit PSF with least squares

rather than using GaussianMixtures.jl to perform EM

It seems like the run-time performance of GaussianMixtures is not actually that bad, but the compile performance is a killer. For example, after loading a RawPSFComponents and running
@time psf_gmm, scale = fit_psf_gaussians(raw_psf)
@time psf_gmm, scale = fit_psf_gaussians(raw_psf)

I see
 17.973990 seconds (35.54 M allocations: 2.139 GB, 2.78% gc time)
  0.201622 seconds (1.69 M allocations: 71.089 MB, 9.68% gc time)

I would guess that this could be improved in GaussianMixtures. (It is already using precompilation though.)

Yep, this issue is more about fitting the PSF well rather than speed. It's probably easiest for @rgiordan to work on this since he wrote the PSF code.

Ah, I see. Well, hopefully this will also give us a bit more control over performance and we can have the best of both worlds.

Getting rid of GaussianMixtures will also help us migrate to Julia 0.5. I'm not sure how to work though those errors otherwise. @rgiordan -- Are you / will you work on this?

Yes, I will work on this after the parameter bounds fix.
On Tue, Mar 1, 2016 at 2:04 PM, Jeffrey Regier<EMAIL_ADDRESS>wrote:

Getting rid of GaussianMixtures will also help us migrate to Julia 0.5.
I'm not sure how to work though those errors otherwise. @rgiordan
https://github.com/rgiordan -- Are you / will you work on this?
—
Reply to this email directly or view it on GitHub
https://github.com/jeff-regier/Celeste.jl/issues/120#issuecomment-190926551
.

Any objections to taking Celeste down to two PSF components (per Dustin's having done so for DECaLS)?

Up to you. I'm a little concerned about the fit not working well with 3
components, since we know Dustin did it that way, and that's the way that
got good results before. But a 50% speed up would be nice, and since
neither 2 or 3 are great we might as well take the speed up
On Wed, Mar 2, 2016, 8:23 PM Ryan<EMAIL_ADDRESS>wrote:

Any objections to taking Celeste down to two PSF components (per Dustin's
having done so for DECaLS)?
—
Reply to this email directly or view it on GitHub
https://github.com/jeff-regier/Celeste.jl/issues/120#issuecomment-191525449
.

Cool.  When I make this change I'll set it up so the number of PSF components is not hard-coded, so it will be easier to go back and forth if needed.

Another reason not to worry is that two components fit with LS are still a better fit (in terms of LS) than three fit with EM.

@jeff-regier , you use GaussianMixtures in gen_priors.jl (line 66).  Shall we just comment that out for now to get rid of the package quickly?

Sorry, I take that back -- I think we can leave it in and simply remove GaussianMixtures from REQUIRE since gen_priors.jl is in bin.  It's bad form, but not too bad.

Yes, let's do that. We'll figure out what to do with gen_priors.jl another
time
On Wed, Mar 9, 2016, 6:11 PM Ryan<EMAIL_ADDRESS>wrote:

Sorry, I take that back -- I think we can leave it in and simply remove
GaussianMixtures from REQUIRE since gen_priors.jl is in bin. It's bad
form, but not too bad.
—
Reply to this email directly or view it on GitHub
https://github.com/jeff-regier/Celeste.jl/issues/120#issuecomment-194619996
.

This is probably still too slow -- around a second to a second and a half per source on my computer.  About half the time is spent initializing  BvnComponents, which I currently need to do for every pixel since it's immutable.  @jeff-regier , what do you think about making BvnComponent mutable?  What are the downsides?

I make things immutable by default but mutating it is going to make it go
noticeably faster, we should do that.
Why do you initialize BvnComponents every pixel? Do you mean that you
initialize it for every iteration of the optimization?

  Yes, I can pull that out of the loop.  That will probably just fix this problem.  Sorry

It's down to about 0.4 seconds per source.  Maybe re-scaling will make it faster, but otherwise this might have to do for now.

This is fast enough now, I think.


Display n smaller/larger bins on quality metric plots

22887ca4b18572173d127405e30136063d9adf7b adds the n larger bin to the genotype quality and depth graphs. The site quality metrics graphs still do not show the n smaller/larger bins.

FS, ReadPosRankSum, SOR, ClippingRankSum, BaseQRankSum, DP, InbreedingCoeff, MQRankSum, and VQSLOD all have n smaller/larger values.

It was decided that the number of sites in the n smaller/larger bins for site quality metrics are so low  compared to the other bins that it's not worth adding them.


"gzip -cn: gzip: command not found" error when installing any git-based package

On latest vcpkg HEAD (6633d421b7f53ba6faadb8ee85ef806718dfd64b):
PS C:\Users\edechamps\Documents\vcpkg> .\vcpkg install fdlibm
The following packages will be built and installed:
    fdlibm[core]:x86-windows
Starting package 1/1: fdlibm:x86-windows
Building package fdlibm[core]:x86-windows...
-- Fetching https://android.googlesource.com/platform/external/fdlibm...
CMake Error at scripts/cmake/vcpkg_execute_required_process.cmake:56 (message):
    Command failed: C:/Users/edechamps/Documents/vcpkg/downloads/tools/git-2.19.1-windows/cmd/git.exe;archive;FETCH_HEAD;-o;C:/Users/edechamps/Documents/vcpkg/downloads/temp/fdlibm-59f7335e4dd8275a7dc2f8aeb4fd00758fde37ac.tar.gz
    Working Directory: C:/Users/edechamps/Documents/vcpkg/downloads/git-tmp
    See logs for more information:
      C:\Users\edechamps\Documents\vcpkg\buildtrees\fdlibm\git-archive-err.log

Call Stack (most recent call first):
  scripts/cmake/vcpkg_from_git.cmake:104 (vcpkg_execute_required_process)
  ports/fdlibm/portfile.cmake:3 (vcpkg_from_git)
  scripts/ports.cmake:71 (include)

Error: Building package fdlibm:x86-windows failed with: BUILD_FAILED
Please ensure you're using the latest portfiles with `.\vcpkg update`, then
submit an issue at https://github.com/Microsoft/vcpkg/issues including:
  Package: fdlibm:x86-windows
  Vcpkg version: 2018.10.20-nohash

Additionally, attach any relevant sections from the log files above.

C:\Users\edechamps\Documents\vcpkg\buildtrees\fdlibm\git-archive-err.log:
gzip -cn: gzip: command not found

The same issue occurs on any package that uses vcpkg_from_git().
Looks like gzip is missing. Maybe vcpkg should install it automatically?

For some reason, the git package that vcpkg downloads does not include usr\bin\gzip.exe. This is surprising, because the official portage package on git-scm.com does include it.
Workaround: download the official 32-bit portable git from git-scm.com and copy usr\bin\gzip.exe into vcpkg\downloads\tools\git-2.19.1-windows\usr\bin. Works for me.


Some more docs and testing

Refactoring:

[x] Refactored MANIFEST and setup.py to include only required code from sdist. (excludes tests .pyc files)
[x] Refactored the unittest classes.
[x] Started applying some type annotations
[x] __slots__ where it was easy to do. (turns class instances into tuples versus dicts, mostly massive memory savings)

Tests:

[x] Wrote tests for unittest classes.
[x] Wrote tests for tortoise.backends.base.db_url.
[x] Wrote tests for QuerySet methods.

Docs:

[x] Expanded docs for Exceptions
[x] Expanded docs for QuerySet
[x] Split up Models and Fields in documentation.
[x] Split out init() documentation

I feel that this PR is getting too large, as it is changing a lot of things already.

@Zeliboba5 I feel that this PR is getting too large, as it is changing a lot of things already. Please review.

👍


constructing an object of class type 'Delegate' with a metatype value must use a 'required' initializer

class FallbackDelegateBox<Delegate: NSObject>: AnyFallbackDelegateBox {
    var storage: DelegateStorage

    enum DelegateStorage {
        case type(_ type: Delegate.Type)
        case instance(_ delegate: Delegate)
    }

    override var delegate: NSObject? {
        switch storage {
        case let .type(type):
            let delegate = type.init() // error: constructing an object of class type 'Delegate' with a metatype value must use a 'required' initializer
            ...
        }
        #endif
    }
}

Tracked upstream via https://github.com/apple/swift/issues/71874


Fix PTZ control

Hass was undefined when making service call. Using the objects hass property fixes this

Thanks


Types

How do we polyfill them?

I saw a little chatter about this in Discord, but I didn't totally follow what needed to be worked out. As long as the consumer is on the latest @types/ember__object, using @computed('dep') should 'just work' with TypeScript.
Can confirm I was pleasantly surprised when I discovered that myself this morning 🙂

Awesome! I guess this is fixed then 😄


Assertion failed at src/task.c:951:task_system_get_thread_scratch(): "(tid < MAX_THREADS)". tid=64

Hello,
I set the SplinterDB in the following way:
#include "splinterdb/default_data_config.h"
#include "splinterdb/splinterdb.h"
/* Application declares the limit of key-sizes it intends to use */
#define USER_MAX_KEY_SIZE ((int)100)
#define DB_FILE_SIZE_MB 1024 // Size of SplinterDB device; Fixed when created
#define CACHE_SIZE_MB 64 // Size of cache; can be changed across boots
SplinterDB::SplinterDB(const char* dbfilename, const std::string& config_file_path)
{
// get SplinterDB config
default_data_config_init(USER_MAX_KEY_SIZE, &splinterDataCfg_);
memset(&splinterdbCfg_, 0, sizeof(splinterdbCfg_));
splinterdbCfg_.filename = dbfilename;
splinterdbCfg_.disk_size = (DB_FILE_SIZE_MB * 1024 * 1024);
splinterdbCfg_.cache_size = (CACHE_SIZE_MB * 1024 * 1024);
splinterdbCfg_.data_cfg = &splinterDataCfg_;
int createStatus = splinterdb_create(&splinterdbCfg_, &db_);
if (createStatus != 0) {
cerr << "Failed to create underlying database, status = " << createStatus << endl;
throw std::runtime_error("Failed to open underlying database");
} else {
cerr << "Successfully created underlying database at path: " << splinterdbCfg_.filename << endl;
}
}
However, the error message appears when inserting KV pairs with YCSB-C after inserting around 2000 KV pairs.
"Assertion failed at src/task.c:951:task_system_get_thread_scratch(): "(tid < MAX_THREADS)". tid=64"
How can I configure the thread number in SplinterDB or other solutions to fix this bug?

Hello,
The problem still occurs; when I write KV pairs less than 1 million (e.g., 0.1 M), it does not show the problem, but when the KV pair number is over 1 M, the problem kills the application.

Thank you so much for your help.

Hi Yanjing,
Thanks for the report.
The error indicates that there are more than 64 threads.  Splinter
currently supports only up to 64 threads, including any background
threads.  Did you start more than 64 threads in your experiments?
Best,
Rob
On Sun, Aug 11, 2024 at 5:33 AM Yanjing Ren @.***>
wrote:

Hello,
The problem still occurs; when I write KV pairs less than 1 million (e.g.,
0.1 M), it does not show the problem, but when the KV pair number is over 1
M, the problem kills the application.
image.png (view on web)
https://github.com/user-attachments/assets/461c3317-8f01-440b-98be-c872ddd8a583
Thank you so much for your help.
—
Reply to this email directly, view it on GitHub
https://github.com/vmware/splinterdb/issues/629#issuecomment-2282745487,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA45FRL2IMEDJIEYCQJQF4DZQ5KZRAVCNFSM6AAAAABK4FNWW2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOBSG42DKNBYG4
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Thanks for your help; I integrated SplinterDB with YCSB-C, and here's the initialization code:
#ifdef __cplusplus
typedef bool _Bool;
#endif
extern "C" {
#include "splinterdb/default_data_config.h"
#include "splinterdb/splinterdb.h"
}
/* Application declares the limit of key-sizes it intends to use */
#define USER_MAX_KEY_SIZE ((int)100)
#define DB_FILE_SIZE_MB 1024 // Size of SplinterDB device; Fixed when created
#define CACHE_SIZE_MB 64 // Size of cache; can be changed across boots

SplinterDB::SplinterDB(const char* dbfilename, const std::string& config_file_path)
{
    // get SplinterDB config
    default_data_config_init(USER_MAX_KEY_SIZE, &splinterDataCfg_);
    memset(&splinterdbCfg_, 0, sizeof(splinterdbCfg_));
    splinterdbCfg_.filename = dbfilename;
    splinterdbCfg_.disk_size = (DB_FILE_SIZE_MB * 1024 * 1024);
    splinterdbCfg_.cache_size = (CACHE_SIZE_MB * 1024 * 1024);
    splinterdbCfg_.data_cfg = &splinterDataCfg_;
    int createStatus = splinterdb_create(&splinterdbCfg_, &splinterdb_);
    if (createStatus != 0) {
        cerr << "Failed to create underlying database, status = " << createStatus << endl;
        throw std::runtime_error("Failed to open underlying database");
    } else {
        cerr << "Successfully created underlying database at path: " << splinterdbCfg_.filename << endl;
    }
}

This is learned from the example (examples/splinterdb_intro_example.c), and I only set the disk and cache size. After that, I learned from the large insert stress test (tests/unit/large_inserts_stress_test.c) and set the num_memtable_bg_threads and num_normal_bg_threads to one, but it does not help.
When I run the YCSB-C, it only runs with one thread, and the total thread number should not exceed 64.
Thanks.

I suspect the problem is that you haven't registered the thread with
splinterdb.  In YCSB-C, each thread runs DelegateClient, which begins by
calling db->Init().  I'd recommend calling splinterdb_register_thread
from your SplinterDB class Init method.
BUT!
Even better, use the version of YCSB-C that we have already integrated with
SplinterDB:
https://github.com/rtjohnso/YCSB-C
It supports lots of SplinterDB settings from the command line.  It also has
several optimizations and other feature enhancements.
Best,
Rob
On Sun, Aug 11, 2024 at 5:34 PM Yanjing Ren @.***>
wrote:

Thanks for your help; I integrated SplinterDB with YCSB-C, and here's the
initialization code:
#ifdef cplusplustypedef bool Bool;
#endifextern "C" {
#include "splinterdb/default_data_config.h"
#include "splinterdb/splinterdb.h"
}/* Application declares the limit of key-sizes it intends to use /
#define USER_MAX_KEY_SIZE ((int)100)
#define DB_FILE_SIZE_MB 1024 // Size of SplinterDB device; Fixed when created
#define CACHE_SIZE_MB 64 // Size of cache; can be changed across boots
SplinterDB::SplinterDB(const char dbfilename, const std::string& config_file_path)
{
// get SplinterDB config
default_data_config_init(USER_MAX_KEY_SIZE, &splinterDataCfg);
memset(&splinterdbCfg, 0, sizeof(splinterdbCfg));
splinterdbCfg_.filename = dbfilename;
splinterdbCfg_.disk_size = (DB_FILE_SIZE_MB * 1024 * 1024);
splinterdbCfg_.cache_size = (CACHE_SIZE_MB * 1024 * 1024);
splinterdbCfg_.data_cfg = &splinterDataCfg_;
int createStatus = splinterdb_create(&splinterdbCfg_, &splinterdb_);
if (createStatus != 0) {
cerr << "Failed to create underlying database, status = " << createStatus << endl;
throw std::runtime_error("Failed to open underlying database");
} else {
cerr << "Successfully created underlying database at path: " << splinterdbCfg_.filename << endl;
}
}
This is learned from the example (examples/splinterdb_intro_example.c),
and I only set the disk and cache size. After that, I learned from the
large insert stress test (tests/unit/large_inserts_stress_test.c) and set
the num_memtable_bg_threads and num_normal_bg_threads to one, but it does
not help.
When I run the YCSB-C, it only runs with one thread, and the total thread
number should not exceed 64.
Thanks.
—
Reply to this email directly, view it on GitHub
https://github.com/vmware/splinterdb/issues/629#issuecomment-2282952376,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA45FRM4PCEFEPYUHCZSE5LZQ77KHAVCNFSM6AAAAABK4FNWW2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEOBSHE2TEMZXGY
.
You are receiving this because you commented.Message ID:
@.***>

Thanks a lot for your help!
Yes. The issue comes from the thread registration. Following your recommendation, I switched to the SplinterDB integrated YCSB-C (https://github.com/rtjohnso/YCSB-C); it works well and performs better than the original one.
Best,
Yanjing


fixunnecessary cast will be added on children in CaseExpr sometimes

Proposed changes
Issue Number: close #9599
Problem Summary:
unnecessary cast will be added on children in CaseExpr because use symbolized equal to compare to Expr's type.
it will lead to expression compare mistake and then lead to expression substitute failed when use ExprSubstitutionMap
Checklist(Required)

Does it affect the original behavior: (Yes)
Has unit tests been added: (Yes)
Has document been added or modified: (No Need)
Does it need to update dependencies: (No)
Are there any changes that cannot be rolled back: (No)

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

better add case in regression test

done


Extend SigningCredentialsResolver in AccessTokenClient to retreive from keyvault

Description
We need accesstoken to connect to different platform components. To generate access token we need a certificate. The certificate must be stored in the keyvault.
We use the accesstoken package to generate token. Currently the signingcredentialresolver does not support keyvault access to get certificate information. So we need to extend this to accept keyvault and this package can be reused by many and can be a centralized solution.
Tasks

[ ] Certificate to generate access token
[ ] Extend SigningCredentialResolver to retreive certificate from keyvault

Acceptance Criteria

[ ] Certificate is created for accesstoken generation
[ ] SigningCredentialResolver gets the certificate from keyvault

This issue is resolved by adding extension in AccessManagement.UI instead of AccessToken as we don't have the time to wait or prioritize that. We will revisit this case again when our backlog is less

@acn-dgopa Does this mean that this issue is no longer a "Needs" for the initial go live?
since you already have a PR in review: https://github.com/Altinn/altinn-accesstoken/pull/15

Did not have the time to prioritize that PR comments. As we have a working solution in place, i re prioritized this and decided to address this after our production release


Not a bug: How do you recover from game server problems?

All,
Every once in a while, the factorio game will hit a problem and be unable to connect to the multiplayer servers, or similar. I've found that this is unrecoverable. It may have been a transient DNS issue, whatever.
Are any of you doing any sort of log watching to spot these issues and restart the container if they occur?
Here's some examples. They often come together but not always.
62215.251 Error ServerMultiplayerManager.cpp:710: Matching server connection failed: Error when creating server game: Download failed (status code 520): failed to parse JSON response: Unexpected character (C) at ?:1
62242.696 Error HttpSharedState.cpp:136: CURL failed: code:6, 6; Could not resolve host: multiplayer.factorio.com

31918.802 Error HttpSharedState.cpp:136: CURL failed: code:6, 6; Could not resolve host: multiplayer.factorio.com
31929.352 Error ServerMultiplayerManager.cpp:710: Matching server connection failed: Error when creating server game: Download failed (status code 404).

I have yet to watch logs for events, but it's something I've thought about doing. For example, respond to errors such as above. Or to grab and push chat output to a Discord channel. Or to log other events (player joins/leaves) to a file for forensics after a grief.
Thoughts?

I would suggest that you monitor your docker logs somehow. I don't know if there is any software for that out there. Maybe some bash script could periodically check the logs for such error and restart the server. Maybe you also want to submit a bug report to the factorio devs that the server cleanly crashes or why the crash occurs at all.
PS: 520 gets returned by Cloudflare if the server did not respond with any, an empty or some other weird answer.

Thanks for the details on the 520 error.
I could have a process just doing "docker logs -f  and grepping for a list of things. Seems crude but could be worth it.
I could get complicated and run something to put all the logs into such as splunk or prometheus or something too. I run this on dedicated bare metal hardware so extra processes won't cost me much of anything except the time to put them in place and debug them (grin).

One more clarification: The servers are up, and just keep responding with the "matching server connection failed" or in some cases "Could not resolve host: multiplayer.factorio.com". The dedicated server should have an option to "exit on failure" or something like that. Then docker would just restart it.

Would be a solution, too.
I need to setup such a system for me, too.

Well, there's the use of common open source logging packages to capture the logs (ELK/EFK for one), which is a fairly complex externality and would be an additional cost if running in the public cloud. In my case, a simple process doing "docker logs -f" might be better. If I come up with something I'll share it here. I'm closing this for now. Thanks!


Allow operator to specify job order via ops file

Use Case
We would like to deploy an HA Proxy instance group at the start of our deployment as other jobs depend on it. However, ops-files only allow instance groups to be appended to the end of the deployment. Could a feature be added to prepend to an array?
A slightly different use case would be specifying a given index at which to insert the instance group. For example, the bits-service ops file currently requires a two stage deploy as the bit service VM must be deployed after the WebDav VM but before the CC VM.
Workaround
Rel-Int currently has this hack that replaces the smoke-test job with the HA Proxy job to make it appear earlier in the list.
Our current process for deploying the bits-service is to deploy the bits-service in one deploy, then configure CC to use the bits-service in a second deploy.

related: https://github.com/cppforlife/go-patch/issues/11

v2.0.42 has :before/:after syntax. see syntax details in http://bosh.io/docs/cli-ops-files