Error: 500 Server Error during cluster join

Hello when bootstrapping maas-anvil my final node fails to add:
maas-anvil cluster join     --role database --role region --role agent --role haproxy     --token TOKEN
An unexpected error has occurred. Please run 'maas-anvil inspect' to generate an inspection report.
Error: 500 Server Error: Internal Server Error for url: http+unix://%2Fvar%2Fsnap%2Fmaas-anvil%2Fcommon%2Fstate%2Fcontrol.socket/cluster/control

Inspection report can be found here: https://drive.google.com/file/d/1RJF5NtLhNNdM9pUeV24X9Qsdk6BG3z8g/view?usp=sharing
Currently investigating why this might have happened and will update this issue once/if I find something
Peter

Interestingly, trying to remove the node, and start again - I appear to have some inconsistent state:
maas-anvil cluster add --name k8s-hm-2.demos.vlan32.maas
Node already a member of the MAAS cluster
ubuntu@k8s-hm-3:~$ maas-anvil cluster remove --name k8s-hm-2.demos.vlan32.maas
Error: Node does not exist in the sunbeam cluster

move information:
maas-anvil cluster list
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━┓
┃ Node                       ┃ Status ┃ Region ┃ Agent ┃ Database ┃ HAProxy ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━┩
│ k8s-hm-1.demos.vlan32.maas │   up   │   x    │   x   │    x     │    x    │
│ k8s-hm-2.demos.vlan32.maas │   up   │        │       │          │         │
│ k8s-hm-3.demos.vlan32.maas │   up   │   x    │   x   │    x     │    x    │
└────────────────────────────┴────────┴────────┴───────┴──────────┴─────────┘
ubuntu@k8s-hm-3:~$ maas-anvil cluster remove --name k8s-hm-2.demos.vlan32.maas
Error: Node does not exist in the sunbeam cluster

I could not reproduce this issue on my environment - I suspect the issue might have been time related, as such I will close this issue until I have a reason to re-open it.


BigQuery Fails to Load Tables Created from Drive

Describe the bug
When reading from Google BigQuery, tables that are created from Google Drive will fail to read with the following error:

This occurs even after adding the service account to the requisite sheet

To Reproduce
Steps to reproduce the behavior:

Load table into GBQ from Drive
Create service account and grant access
Attempt to query table from mage

Referenced here, the issue is that, when using the oauth method, additional scopes must be passed to the service_account.Credentials object.
The dbt article uses the following scopes:
["https://www.googleapis.com/auth/userinfo.email",
            "https://www.googleapis.com/auth/cloud-platform",
            "https://www.googleapis.com/auth/sqlservice.login",
            "https://www.googleapis.com/auth/drive"]

When I supply these to service_account.Credentials.from_service_account_info() and service_account.Credentials.from_service_account_file(), the issue is resolved and the table loads


Optimize meshopt_computeMeshletBounds

Optimize meshopt_computeMeshletBounds by passing in pointer to meshlet instead of by value (636 bytes)

No issue, just popped up in static analysis.

Since this came up again in a linked issue I'm having second thoughts and leaning to merging this. The performance impact is minimal but it seems like it's better to pass meshlet by pointer for now. Eventually I kinda want to redesign the meshlet API to not have a structure with hardcoded limits but this complicates the meshlet builder - for now pointer is fine I guess. Let me try to see if I can reopen & merge this...


GridspecLayout isn't rendered

Environment data

VS Code version: 1.75.1
Jupyter Extension version (available under the Extensions sidebar): v2023.1.2010391206
Python Extension version (available under the Extensions sidebar): v2023.2.0
OS (Windows | Mac | Linux distro) and version: Ubuntu 20.04
Python and/or Anaconda version: 3.8.10 (system)
Type of virtual environment used (N/A | venv | virtualenv | conda | ...): docker
Jupyter server running: Local | Remote | local

Expected behaviour
I'm using ipywidgets.GridspecLayout to arrange multiple widgets in my notebook.
Actual behaviour
VS Code doesn't render the grid at all. (Displaying individual widgets works fine, so I assume the problem is really with the grid display itself.)
Steps to reproduce:
[NOTE: Self-contained, minimal reproducing code samples are extremely helpful and will expedite addressing your issue]

First cell:import numpy
from matplotlib import pyplot as plt
%matplotlib widget
import ipywidgets as widgets
slider = widgets.IntSlider()
slider

Second cell:grid[0, 0], grid[0, 1] = widgets.Label('Input'), widgets.Label('Output')
grid

The first cell above shows a slider upon execution. The second doesn't display anything. (I can change the "presentation" to "text/plain", in which case I get a stringified version of the object. Changing back to "application/..." shows nothing again.

The second cell content is actually
grid = widgets.GridspecLayout(1, 2)
grid[0, 0], grid[0, 1] = widgets.Label('Input'), widgets.Label('Output')
grid

of course.

@stefanseefeld thanks for filing this issue, however this displays just fine at my end
Please could you try installing the latest pre-release version of the Jupyter extension
Open the extensions page, search for Jupyter and click the button Switch to Pre-Release Version

Thanks for the tip. I switched to the "pre-release" version, but now my cells stall as soon as I try displaying any widgets. (And by "stall" I mean the little "executing" spinner is spinning and nothing else is happening.) Is there anything else I need to do when switching to the pre-release version of this extensions ?

Aaah, sorry my bad, please install vsode insiders and the latest pre-release version of the Jupyter extension and the latest version of the Notebook Renderer extension.

Can you be a bit more specific ? I'm searching for "insiders" which yields a number of results. I did install IntelliCode Insiders" but that didn't change the faulty behaviour.

Vscode insiders is an official build of vscode, it contains the latest changes as of the day, it’s something that’s released on a daily basis
Here is the link https://code.visualstudio.com/insiders/
you can install and use this along with the regular stable version of vscode

Ah ! I'm slowly getting there...
So now I can indeed see my widget grid, but I do get a lot of javascript errors when trying to display my matplotlib figure:
at ph.loadClass (https://vscode-remote+attached-002dcontainer-002b7b22636f6e7461696e65724e616d65223a222f746e6176227d.vscode-resource.vscode-cdn.net/home/user/.vscode-server-insiders/extensions/ms-toolsai.jupyter-renderers-1.0.15/out/node_modules/%40vscode/jupyter-ipywidgets8/dist/ipywidgets.js:2:4099813)
    at ph.loadClass (https://vscode-remote+attached-002dcontainer-002b7b22636f6e7461696e65724e616d65223a222f746e6176227d.vscode-resource.vscode-cdn.net/home/user/.vscode-server-insiders/extensions/ms-toolsai.jupyter-renderers-1.0.15/out/node_modules/%40vscode/jupyter-ipywidgets8/dist/ipywidgets.js:2:4403287)
    at ph.loadModelClass (https://vscode-remote+attached-002dcontainer-002b7b22636f6e7461696e65724e616d65223a222f746e6176227d.vscode-resource.vscode-cdn.net/home/user/.vscode-server-insiders/extensions/ms-toolsai.jupyter-renderers-1.0.15/out/node_modules/%40vscode/jupyter-ipywidgets8/dist/ipywidgets.js:2:4097773)
    at ph._make_model (https://vscode-remote+attached-002dcontainer-002b7b22636f6e7461696e65724e616d65223a222f746e6176227d.vscode-resource.vscode-cdn.net/home/user/.vscode-server-insiders/extensions/ms-toolsai.jupyter-renderers-1.0.15/out/node_modules/%40vscode/jupyter-ipywidgets8/dist/ipywidgets.js:2:4094616)
    at ph.new_model (https://vscode-remote+attached-002dcontainer-002b7b22636f6e7461696e65724e616d65223a222f746e6176227d.vscode-resource.vscode-cdn.net/home/user/.vscode-server-insiders/extensions/ms-toolsai.jupyter-renderers-1.0.15/out/node_modules/%40vscode/jupyter-ipywidgets8/dist/ipywidgets.js:2:4092246)
    at ph.handle_comm_open (https://vscode-remote+attached-002dcontainer-002b7b22636f6e7461696e65724e616d65223a222f746e6176227d.vscode-resource.vscode-cdn.net/home/user/.vscode-server-insiders/extensions/ms-toolsai.jupyter-renderers-1.0.15/out/node_modules/%40vscode/jupyter-ipywidgets8/dist/ipywidgets.js:2:4091039)
    at https://vscode-remote+attached-002dcontainer-002b7b22636f6e7461696e65724e616d65223a222f746e6176227d.vscode-resource.vscode-cdn.net/home/user/.vscode-server-insiders/extensions/ms-toolsai.jupyter-renderers-1.0.15/out/node_modules/%40vscode/jupyter-ipywidgets8/dist/ipywidgets.js:2:4402511
    at g._handleCommOpen (https://vscode-remote+attached-002dcontainer-002b7b22636f6e7461696e65724e616d65223a222f746e6176227d.vscode-resource.vscode-cdn.net/home/user/.vscode-server-insiders/extensions/ms-toolsai.jupyter-2023.2.1000421627/out/webviews/webview-side/ipywidgetsKernel/ipywidgetsKernel.js:2:36493)
    at async g._handleMessage (https://vscode-remote+attached-002dcontainer-002b7b22636f6e7461696e65724e616d65223a222f746e6176227d.vscode-resource.vscode-cdn.net/home/user/.vscode-server-insiders/extensions/ms-toolsai.jupyter-2023.2.1000421627/out/webviews/webview-side/ipywidgetsKernel/ipywidgetsKernel.js:2:38387)

Is there anything else I need to install / update / fine-tune ?

Please could you enable logging as follows:

Open settings & go into Jupyter->Logging
Change the value to verbose
Reload VS Code,
Attempt to repro the issue & then copy the output from the Jupyter output panel.

I tried this on my work laptop today and everything worked smoothly there. I still haven't quite figured out what the issue is on my personal laptop, but this is much lower priority now. Any idea when this jupyter extension pre-release will turn into a formal release ?

Any idea when this jupyter extension pre-release will turn into a formal release ?

approx 3 weeks
let’s close this issue for now then
If you run into other issues or this same one, please feel free to Ping here or create a new issue


Refactor the refiner.

@jonsterling Currently I am not working on this. I would like to hear your comments.

[x] Rewrite the goal-appending process in ComKit.
[ ] Partition the refiners into more modules. Possibly one to for composition, one for structural rules, one for base types, and another one for functions, pairs and identifications etc.
[ ] Each base type should provides an EqVal, if appropriate, and that Auto should not dispatch on value terms. Leave the value determination to each EqVal rule. I also doubt the usefulness of exposing EqTT and EqFF when EqVal is available.
[ ] Use the flipWrapper hack to skip all Equality.Symmetry.
[ ] Fix the wrong HCom and Coe tactics which wrongly assume the structural equality rule is invertible.
[ ] Fix the wrong FCom rule which does not check whether the type is a (weak) base types.

Dream:

[ ] Investigate why Telescope.Internal.ConsView.out is taking so much time.

Re ConsView.out, the implementation of the telescopes is very crappy and non-optimized. I bet we can improve it quite a bit if necessary...

Each base type should provides a EqVal, if appropriate, and that Auto should not dispatch on value terms. Leave the value determination to each EqVal rule. I also doubt the usefulness of exposing EqTT and EqFF when EqVal is available.

I disagree.  We should have one PRL rule for tt = tt and one for ff = ff; it is the job of tactics to do more clever dispatch.

I disagree.  We should have one PRL rule for tt = tt and one for ff = ff; it is the job of tactics to do more clever dispatch.

Are you against each type having an EqVal then? They can provide all the primitive rules and some useful tactics so that Auto is more modular.

I see your points. Are you against each type having an EqVal then?

Yes, I think I'm opposed to that, at least for now. My preference is for the rules of the PRL to be highly modular, and not at all to embed any decision procedures inside these rules.

My preference is to write a tactic to capture any of the things that you are currently proposing to embed in the rules. If it is currently difficult or not possible to write a tactic that you need, then I can fix that.

Hmm I feel I never used the terminology correctly, and we should probably clarify that: by "rules" are you talking about everything currently implemented in the refiner except for AutoEq and Auto? Or, is AutoEq in various types actually a rule? I think I am confused because I thought AutoEq is not a primitive rule and yet we have it for almost every type, and then concluded that it is okay to put more stuff in. If you also said "we should move AutoEq out of those modules" then I would have understood it better. I also like a complete separation of "rules" and "tactics", which is different from what I thought but also makes perfect sense.
BTW, to avoid confusion, I will use the following terminology:

rule: any rule (scheme) that can be directly traced back to some rule (scheme) in the proof theory on paper.
tactic: any other admissible theorems based on the rules.

While we are trying to reach some agreement, I suppose you are fine with me fixing the wrong Auto rule which assumes too much invertibility and that FCom.Eq which fails to check whether it is in a base type.

@jonsterling There is still something I want to do, like having a SYNTH rule for strict bools, but I am not sure how to do that... I hope the breaking down makes it obvious what's lacking now.

@favonia You're doing a great job!
Re: SYNTH for strict bools, let's chat about what you have in mind. I think it might be a little tricky to have type synthesis for the if/s(e1; e2; e3) form, because this does not have an annotation of the motive, unlike the usual if(a.A; e1; e2; e3) form for the weak booleans.


Simplify the driver registration removing the factory

Use a type to register a class and add an initializer in the Driver protocol to create the driver

@marcc-orange fixed :)


Authenticate using linkedin app if already installed and running

Hello,
Is there any way to use your module without typing the linkedin password ? if the LinkedIn app is already installed.
Here's an app that implements it seemlesly: www.shapr.co
Thanks

It should be possible to get cookies from external connexion with WebBrowser into expo, but for now I'm using webview.
Do you think WebBrowser implementation (ie. in expo) can give you what you're looking for?

@botmane @xcarpentier a MUST capability. No one actually remembers their LinkedIn credentials which mean this package can't be used...


When getting a server misbehave error, there is no way to retry the request

Getting the error:
Error while pulling image: Get https://index.docker.io/v1/repositories/library/cassandra/images: dial tcp: lookup index.docker.io on <IP_ADDRESS>:53: server misbehaving
but there is no way to retry the downloading phase. It just stuck that way as seen in the screenshot.

If you have in your preferences to 'shutdown Linux VM on closing Kitematic' checked, then you should be able to close kitematic (which shuts down the VM) and should also kill the above pending download.

yup, I have it checked. the thing is, I need to retry on this again and again until the IP is not misbehaving. IMO there should be a way to handle it, a refresh button maybe?

Do you meant the ability to retry the download over and over when it fails?

yes.


Connectivity change listener

Can you add connectivity change listener to detect where device is online or offline?

Hi, thanks for the suggestion. I will consider it for one of the upcoming releases.

Thank you. I wait for it.


Hot loading fails to reload

After a bit of mucking, I finally got RHL working to the extent that it doesn't blow up, however the hot loader doesn't reload the view in-app. On the first source change, I see the following error:
[HMR] Update check failed: Error: Manifest request to http://localhost:8082/08705b544a5affd23d86.hot-update.json timed out.
    at request.onreadystatechange (http://<IP_ADDRESS>:8080/index.ios.bundle:41405:23)
    at XMLHttpRequestBase.$XMLHttpRequestBase_setReadyState (http://<IP_ADDRESS>:8080/index.ios.bundle:11371:7)
    at XMLHttpRequestBase.callback (http://<IP_ADDRESS>:8080/index.ios.bundle:11360:10)
    at XMLHttpRequest.<anonymous> (http://<IP_ADDRESS>:8080/index.ios.bundle:11127:14)
    at MessageQueueMixin._invokeCallback (http://<IP_ADDRESS>:8080/index.ios.bundle:7582:10)
    at http://<IP_ADDRESS>:8080/index.ios.bundle:7645:34
    at Array.forEach (native)
    at http://<IP_ADDRESS>:8080/index.ios.bundle:7637:15
    at ReactDefaultBatchingStrategyTransaction.Mixin.perform (http://<IP_ADDRESS>:8080/index.ios.bundle:6151:20)
    at Object.ReactDefaultBatchingStrategy.batchedUpdates (http://<IP_ADDRESS>:8080/index.ios.bundle:13995:19)
$HOME/app/node_modules/webpack-dev-server/client/index.js?

Which version of react-native are you on? I noticed they recently pulled in a web socket polyfill into core which might be related. Though your stack trace seems to suggest something happening in the XHR polyfill.

@mjohnston I'm on 0.4.4, so I'm using the latest websocket code... curious...

@auser If you had an existing project prior to 0.4.x you may need link against the new RCTWebSocket library. I did a clean npm install and build of the example project and it works for me.
See: https://github.com/mjohnston/react-native-webpack-server/issues/30

@auser Were you able to get hot loading working? Closing for now


Please go back to using partials

I would like to go back to the stylesheet being built using Sass partials instead of everything existing within _base.scss or, at the very least, put mixins in a partial of their own. My stylesheet uses the new Sass module system, so one of my partials would use @use instead of @import. For example:
@use '~codyhouse-framework/main/assets/css/base/breakpoints';

.some-class {
	@include breakpoints.breakpoint(md) {
		//
	}
}

With the change introduced in v2.7, these partials no longer exist, forcing me to use @import globally again, which will eventually be removed altogether, per the Sass blog post.

Agreed, though for different reasons. I include only parts of the framework within my projects. The partials make the framework more flexible allowing users to implement the entire project or to pick and choose.
Love the framework -- thanks for all of your hard work!

Hi there!
@peiche when you import the _base.scss file in your general _style.scss file, it includes also the breakpoints. Can you please give me an example of when it's convenient to import just the mixins? I'm trying to understand this use case.
@seansean can you please give me an example of which base partials would you import separately? I'm asking because most of the partials were intertwined (for example, you couldn't import the utility classes without importing the breakpoints and the spacing).

@sebastiano-guerriero that's how @import works, but not @use. Per the blog post:

@use only makes names available in the current stylesheet, as opposed to globally.

So while I do import the entire _base.scss in one partial, I now need to re-import the entire thing if I want to use the breakpoints mixin, for example, in a separate partial. Either that, or I need to switch back to using @import instead of @use, which I prefer not to do because the support for it will eventually be removed altogether.

@sebastiano-guerriero  I use my own color and typography system so I exclude CodyHouse's counterpart. Here's a recent project base stylesheet:
/** Tools */
@import 'tools/all';

@import 'tokens/all';

/** CodyHouse Framework */
@import 'codyhouse-framework/main/assets/css/base/reset';
@import 'codyhouse-framework/main/assets/css/base/mixins';
@import 'codyhouse-framework/main/assets/css/base/breakpoints';
@import 'codyhouse-framework/main/assets/css/base/spacing';
@import 'codyhouse-framework/main/assets/css/base/grid-layout';
@import 'codyhouse-framework/main/assets/css/base/forms';

@import 'tokens/breakpoints';
@import 'tokens/spacing';

/** CodyHouse Overrides/extends */
@import 'custom-style/all';

/** Generic */
@import 'generic/all';

/** Elements */
@import 'elements/all';

/** Vendor */
@import 'vendor/all';

/** Objects */
@import 'objects/all';

/** Components */
@import 'components/all';

/** Modules */
@import 'modules/all';

/** CodyHouse utilities */
@import 'codyhouse-framework/main/assets/css/base/visibility';
@import 'codyhouse-framework/main/assets/css/base/accessibility';
@import 'codyhouse-framework/main/assets/css/base/util';

/** Utilities */
@import 'utilities/all';

@peiche wouldn't it be more convenient to use:
@use '~codyhouse-framework/main/assets/css/base' as *;

.some-class {
	@include breakpoint(md) {
		//
	}
}

with no need to import single partials (e.g., in case you need the breakpoints + mixins + some-other-partial)? The @use shouldn't import the code multiple times even if you use it in multiple partials.

@seansean we're working on a solution that would allow you to specify the base partials to include:
$cody-base-partials: 'accessibility', 'breakpoints', 'colors', 'forms', 'grid-layout', 'icons', 'mixins', 'reset','spacing', 'visibility', 'z-index';
@import 'codyhouse-framework/main/assets/css/base';

@import 'custom-style';
@import 'some-other-style';

$cody-base-partials: 'util';
@import 'codyhouse-framework/main/assets/css/base';

What do you think?

@sebastiano-guerriero That should work for my use case. Thank you for considering my request. I appreciate your team and your framework!

This change is the worst. It doesn't "simplify" anything. I really don't want to give up using this framework, especially since I prefer it to anything else out there. My only other option is to revert and never upgrade past v2.6.2. This proposed change would just be a "fix" that would be unnecessary if you didn't break anything in the first place.

I understand your point. This change may not simplify your workflow and, like any other change, may not be welcomed by everyone. When we say "simplify", we refer to those users who got in touch because the difference between "base" and "custom-style" folders (with sort of duplicate file names) wasn't clear. It makes me think that many potential new users could be discouraged from using CodyFrame if we don't simplify this aspect.
At this point, we're monitoring the reactions to the change. We have had a couple of complaints so far. It appears the new SASS modules shouldn't be an issue. If you feel the new version is creating a problem in your workflow, let us know, and we'll def look into it.

No, importing the entire library with @use is actually an issue. My use case is in building a custom stylesheet which first imports the entire framework... no issue there. But then inside a specific class selector, I import some local partials which also now are importing the framework, solely for the use of mixins, like breakpoint, reset, etc. This ends up importing the entire framework again, inside this class. Not only does this complicate some overrides, it also massively bloats the size of the generated CSS file. I would still ask that you separate functions and mixins into separate partials.

Can you please send me the source code (hello@codyhouse.co)? I'm not 100% clear on how you're importing local partials inside a class selector. From what I understood on the SASS documentation about modules, @use and @forward, unlike @import, shouldn't compile the same code multiple times, but just once (unless you're compiling in different .css files).

I decided to just copy the mixins I need to import from the framework into my own partials. This issue can be closed.


Restart files can't

Since RectGridIO unconditionally reads sst and sss fields, and the written restart files do not contain these fields, trying to read a restart file written by the model fails.
Fix this. By removing the restart file dependency on SST and SSS. These should be provided by OceanState implementations instead.

closed by PR #139. Work remains following issue #154


Gale Shapley Algorithm

Aim
Add code for Gale Shapley Algorithm
Details
A greedy based algorithm with proper problem statement commented and tested output in comments.
Programming language

[x] C++

pls assign this to me @Albus23


use github path for files

the current path won't work unless you clone the repo

Yes, we always assume that the project is cloned.

Like @lordofthejars said we assume that the project is always cloned locally


Status line in query control mode

Maybe this?
[ 100-110 of 10,434,543 ]

This issue will be addressed with #65.


[Suggestion, new commandline option] Consider adding --shrink to allow users to arbitrarily shrink .pdf files (and perhaps --shrink=NUMBER_VALUE to decide how much to optimise, such as --shrink=3 or something like that)

Hello Jay and everyoneelse,
Today I received some .pdf files (company bills and what not) which I intend to store for many years to
come. Because this is slowly increasing in total size (e. g. averaging 1MB per .pdf file, give or take), I
was thinking of optimising the .pdf file for size - that is, to shrink it when possible. A little bit like JPEG
compression when it can be considered acceptable (I usually store between the range of 80-90% of
the original quality, excluding family fotos which I tend to save in best quality). So far so good.
There are many online-compression websites for .pdf files, which work well, but I was thinking that
perhaps qpdf could be used here instead. This would be helpful in the sense that I could do so
via the commandline as-is. (It is possible to do so via a remote webpage too, of course, but it requires
some more work to do via scripts, as well as a working internet connection which is not always the
case, thus commandline seemed appealing.)
I was reading through qpdf --help but not finding an option. If no way exists to shrink or optimise .pdf
files via qpdf then this is fine, don't worry and ignore the suggestion. If, however had, there is a way
to optimise .pdf files via qpdf, could perhaps an alias be added? Something like "--shrink" or
"--compress". Ideally a simple to use variant.
There are a few options mentioned, such as "compress" via --help, and perhaps other options for
zlib, but ideally I would like to just have a "please compress it as much as possible" variant without
having to understand what it does exactly. For simple .pdf files, such as office bills, it's usually just
a single .pdf file and as long as one can read the text it's really fine - any result given to me as the
end user here, if the file size is smaller, seems worthwhile to have.
At any rate this is just a suggestion; please feel free to close or disregard at any moment in time.
Thanks for reading.

Coincidentally I wrote code to do this for work but it uses the GraphicsMagick library. I have an old image scaling algorithm I implemented back about 20 years ago at the same time I originally wrote qpdf and which I also also obtained the copyright for -- it's from the same overall system that I extracted qpdf from. I have been considering adding some of that functionality. I think working that in could be a possibility. I have no doubt that I could obtain permission to release my current tool as an open source project as well though it would not be part of qpdf itself because of the extra dependencies. I'd have to peel it apart from some of the application-specific logic. That code is basically an enhanced version of the image optimization code that's already in qpdf. In addition to trying to recompress images as jpeg if possible, it also downsamples images so that they have bit depth <= 8 and size that fits within 1500x2000 pixels. In my day job, we use this for medical records. Frequently the records contain high resolution images, but for our purposes, we really only care about text.
This request gives me a little extra impetus to dust off my old image scaling algorithms and fold them into qpdf. I've also considered adding native support for /CCITTFaxDecode, at least in the /K < 0 case (basically CCITT group 4/ITU.6) which I think wouldn't be more than a day's work.
So let's call this is a maybe. Thanks as always for your suggestions. I really appreciate that you often take the trouble to write up and suggest your ideas. It helps me better understand how people want to use the library.

See my example in reducing the size of PDF by re-compressing some JPEG images with a lower quality at https://github.com/qpdf/qpdf/issues/712.
I used a Python module from ocrmypdf, which is based on pikepdf.


Support All Statistics and Enable Backpressure in Parallel Parquet Writer

This PR is Draft since it cannot compile without arrow-rs changes: https://github.com/apache/arrow-rs/pull/4850
Which issue does this PR close?
Closes #7591
Closes #7589
Related to https://github.com/apache/arrow-rs/issues/1718
Rationale for this change
Parallel parquet serialization process implemented in #7562 did not support all parquet metadata (indexes/bloom filters) and had no backpressure on serialization tasks. This PR aims to address these two deficiencies.
What changes are included in this PR?

Parallel parquet serialization tasks now use ArrowRowGroupWriter directly rather than ArrowWriter
Upstream arrow-rs changes filed to make ArrowRowGroupWriter public and Send so it can be used across an .await
Parquet serialization tasks can be throttled via a bounded channel mechanism.

Benchmarking Results
The results show the parallel parquet process in this PR is ~10% faster than the previous in addition to supporting statistics/bloom filters. Channel_limit=N means that the maximum number of parallel parquet serialization tasks running at one time is set to N. Surprisingly setting this number low can actually increase peak memory usage, which is a surprising result. Perhaps RecordBatches are accumulating in memory as some streams are being consumed but not others, more than offsetting the benefit of limiting the amount of serialized parquet data in memory.
See #7562 for benchmarking script used.
Test 1, All 16 Columns, ~3.6GB Parquet File (release build)
Execution Time (s)

Parallelism
Main (single_file_output=false)
Main (single_file_output=true)
This PR (single_file_output=true, channel_limit=100)
This PR (single_file_output=true, channel_limit=4)

1
22.48
22.53
21.04
22.17

4
12.24
14.4
12.49
12.73

8
10.79
12.37
10.7
11.03

16
10.52
12.67
10.78
10.85

32
10.91
12.07
10.31
10.25

64
10.21
12.97
12.34
11.62

Peak Memory Usage (MB)

Parallelism
Main (single_file_output=false)
Main (single_file_output=true)
This PR (single_file_output=true, channel_limit=100)
This PR (single_file_output=true, channel_limit=4)

1
1753.3
1758
1757.1
1760

4
2445.4
7104
5690.7
5684.2

8
3387
7623.1
6642
7804

16
5047.6
8262.6
8151
10437.7

32
7683.6
7672.6
9358
11657.5

64
10961.1
10370.2
10388
12898

Test 2, Subset of 3 Columns, ~895MB Parquet File (release build)
Execution Time (s)

Parallelism
Main (single_file_output=false)
Main (single_file_output=true)
This PR (single_file_output=true, channel_limit=100)
This PR (single_file_output=true, channel_limit=4)

1
3.57
3.15
3.38
3.39

4
1.78
2.37
2.53
2.3

8
1.45
2.07
2.58
2.23

16
1.54
2.09
2.06
1.71

32
1.7
2.1
2.08
1.65

64
1.89
2.72
2.76
2.07

Peak Memory Consumption (MB)

Parallelism
Main (single_file_output=false)
Main (single_file_output=true)
This PR (single_file_output=true, channel_limit=100)
This PR (single_file_output=true, channel_limit=4)

1
450.6
451.6
448.9
449.4

4
584.5
1659.1
1257.0
1284.4

8
759.4
1939.7
1225.7
1339.2

16
1045.8
2051.2
1359.8
1438.5

32
1564.6
1899.7
1445.5
1545.3

64
2318.8
1726.1
1732.0
1735.3

Are these changes tested?
Yes, by existing tests and adhoc benchmarking
Are there any user-facing changes?
No
Next Steps
So far, parquet serialization is only being parallelized in terms of RowGroups. This means we are limited in terms of parallelization based on the number of RowGroups we want in our file, which can be as low as 1 in general. Parquet files generally have a large number of columns and we could parallelize at the column level in addition to speed up more.
We could also break free of the Parallelism=RowGroupNumber limit if it were possible to concatenate (ArrowColumnChunk, ColumnCloseResult) tuples together before writing them into a RowGroup. This might not be possible efficiently, since ArrowColumnChunk's are already compressed. Aggregating column min/max statistics would be trivial, but distinct values + bloom filters would not be trivial.

@alamb Thank you for the review! I have significantly reworked this PR in a new PR #7655, relying primarily on column wise parallelization rather than row group wise. The new approach is much more complex in the code, but the performance advantage is huge. 20% faster and 90% lower memory overhead vs. this PR.


DM-34488: Initial version

The performance table does not render very well -- it's far too wide. Perhaps I should try some other format. Suggestions welcome.

I just saw your message about the table and then went to check, I had only seen the first couple of columns... can you explain how you measure latency in the text? I think we should still try to measure latency for different throughputs though..

I rewrote the performance section, including breaking the data into two tables in order to make all the data visible without scrolling. Please have another look.

@dmills-lsst offered some text for updating  ts_sal and packages that rely on it. I have incorporated it into the document.

@mareuter excellent suggestions. I am still working on some of them, but wanted to say that I have added a Conclusions section (which should have been there all along) that summarizes why we prefer Kafka. I have also made most of your small suggested changes. I'll let you know when I finish the rest of the rewriting.

Regarding "I see that schemas are talked about in the technical appendix..." schemas are also talked about in the main part of the document. I rewrote that section to highlight the fact that if we use Kafka we don't need to restart the whole system, even if schemas change in an incompatible way.

Regarding "How do the CSCs react if they startup before Kafka is ready": I suspect they fail to start. Once the CSC has started up (and thus registered all topics) they should handle all brokers disappearing by failing to write or read, which would presumably send them to FAULT, where they could be re-enabled when a broker returns.


a function for making arcaea_database.db able to be downloaded

I see that I can upload the arcaea_databse.db file through the admin web page on any network, if I do some additional port-fowarding settings on my router.
However, It seems that there aren't any functions for downloading the .db file, it would be helpful if both downloading & uploading is possible. It would let people able to fix the .db files instantly if any errors happen / or just let people like me add some accounts directly to the .db or edit some data when they aren't able to access the server directly.
As this is more like an additional feature, nevermind this report if this would be a bit complex or troublesome.

Well, the function of uploading database is used to do manual updating. It should not be used to change data in it, and it may cause some safety issues.
If you want to change something at the file level, the more recommended way is using RDP or SSH.
Additionally, I have indeed considered the solution of directly operating the database on the front-end page, although this may be a bit unsafe. But, this doesn't seem like a simple thing; at least I won't be able to achieve it in the near future.

As long as the data in the .db file is edited properly, it won't make any issues. However, I understand your opinions, as you said I could just use SSH instead. Thanks for replying.


Update Documentation

Steps to reproduce

Read the Documentation of ShadPopover Section
You can see here that it has builder property within ShadPopover

Expected results
The builder is deprecated it is now popover
Actual results
The documentation still displaying the builder
Platform
Windows
Code sample
Code sample
[Paste your code here]

Screenshots or Video

Screenshots / Video demonstration
[Upload media here]

Logs
Logs
[Paste your logs here]

Flutter Doctor output
Doctor output
[Paste your output here]

fixed, thanks 👍


How to deserialise native type arrays

Hi, I'm new to Gloss and I struggle deserialising native object arrays.
For example I want to deserialise this object
{
   name: "Duffycola",
   values: [1,2,3,4,5]
}

I don't see how I would correctly extend Int to conform to the Decodable protocol, because the individual object "1" has no "key" and no enclosing brackets in the json. I think the problem is the fact that Decoder.decode() requires a key. Would it be possible to add a Decoder.decode method which simply casts the given string to Int?
extension Int : Decodable
{
	public init?(json: JSON) {
		guard let value : Int = "" <~~ json
		else { return nil }
		self.init(value)
	}
}

Second, the documentation gives as an example how to deserialise a collection of objects, meaning the array starts at the root. In this example, however, the array is a subpath. I thought of doing something like this, but I'm not sure:
guard let durations: [Int] = Int.modelsFromJSONArray(("values" <~~ json)!)

You don't have to extend Int to conform to Decodable. It will just work. Below is the code for model for that json.
struct Dummy: Decodable {

    var name: String
    var values: [Int]?

    init?(json: JSON) {
        guard let id: String = "id" <~~ json else { return nil }

        self.id = id
        self.values = "values" <~~ json
    }

}

thanks @RahulKatariya !


irclogs.laravel.io does not work

Server not found
Firefox can't find the server at irclogs.laravel.io

I recently transferred the DNS for Laravel.IO and I think this record didn't came along.
Anyhow, I don't have any intention anymore to do this through Laravel.IO. I also have no idea who hosts the IRC logs. You'll have to ask around on IRC to see who hosts the logs to see if they can set up a domain for it.

okay. Thanks for fast reply :+1:

@driesvints  can you please do this
irclogs.laravel.io
@taylorotwell : the DNS entry should point to <IP_ADDRESS>, could you restore it?

I'll try to setup the DNS record later.
Taylor doesn't have anything to do with this btw.

thanks.

@rajlaksh when I go to that IP I get to see the following:

Are you sure this is the correct IP address?

Yes, IP is correct.
That IP i got from Glose.

Ok. Should be back up now: http://irclogs.laravel.io/

Thanks. Please keep this DNS safe :)


Theif Cat Basic Implementation (Needs Art)

Summary
Thief cat now can see the player in their vision detection "box" and runs to them. They do not run away but honestly, idk if they should. They are pretty weak to begin with and this may be annoying chasing them down for the "kill all baddies" rooms.
Only thing left to do is all of the art, which I am waiting on @Elginive for.
Closes #229
Testing
Add him to a map and brawl.

I will work on more baddy stuff this Friday, if I get the cucukin and bat finished off, I will move on to this fella.


Varvara - File Device typo?

The table describing the fields of the file device describes the length as:
addr(length)*
However, the length field contains a value, not an address, so it should probably be
length*

Thank you, fixed in #734af7e2a..f2025ce9b


Support 12 Series

Can you support 12 Series ？
12mini / 12 / 12pro / 12 pro max

Yes we are working on it: https://github.com/devicekit/DeviceKit/pull/262 😊


fix: missing dependency

also added .gitignore file just in case

@MannuVilasara good job

Thank you @enty8080 ✌🏻


Update CHANGELOG with changes for v1.3.3

Motivation
I noticed that some changes are listed twice or listed at the wrong release in CHANGELOG.md and fixed those
Changes

update CHANGELOG to list changes at the correct versions and deduplicate entries.

Test Plan

n/a

TODO

[x] Explain motivation or link existing issue(s)
[x] Test changes and document test plan
[x] Update documentation as needed
[x] Update changelog as needed

bors merge


fix: Fix the order of the files when running the tests

Proposed changes
Fix order of files when running test, now tests will run according to the order of
files in directory.
See more in this question
Types of changes
What types of changes does your code introduce?

[x] Bugfix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist

[x] I have read the CONTRIBUTING doc
[x] Lint and unit tests pass locally with my changes
[ ] I have added tests that prove my fix is effective or that my feature works.
[ ] I have added necessary documentation (if appropriate)

Coverage remained the same at 81.128% when pulling<PHONE_NUMBER>ed90eff01723555ef8e9ce5a899449 on patrickalima98:develop into 4a461e2144c4ea8bf9b7a5477916cb81c029dc63 on adonisjs:develop.


Tabs disabled docs

The disabled tabs example seems to be broken on the documentation site.
￼

commit dc04d00f485e3c4d51942e094483049d06e12a5a (HEAD -> master)
Author: Rob Levin<EMAIL_ADDRESS>Date:   Sat Dec 11 06:53:39 2021 -0600
Fixes bug with the Tabs docs page where disabled tab buttons were not correct IDs so were not in fact disabled.


board.fs - init not found

While uploading the board.fs into BluePill (128k flash, latest mecrisp flashed in) I get
` ok.
!s always.fs
1> always.fs 3: Finished. Reset ☺Mecrisp-Stellaris RA 2.3.5 for STM32F103 by Mat
thias Koch
1> always.fs 13: ( always end: ) 00005060  ok.
1> always.fs 14: Redefine eraseflash.  ok.
!s board.fs
1> board.fs 3: eraseflash
Finished. Reset ☺Mecrisp-Stellaris RA 2.3.5 for STM32F103 by Matthias Koch
1> board.fs 6: ( board start: ) 00005400  ok.
7> hal.fs 14: Redefine bit.  ok.
1> board.fs 27: init not found.
Send failed.`

This may work?
: init ( -- )  \ board initialisation
\ init  \ uses new uart init convention
['] ct-irq irq-fault !  \ show call trace in unhandled exceptions
jtag-deinit  \ disable JTAG, we only need SWD
72MHz
1000 systick-hz
hello ." ok." cr
;

Yes, you could remove init, but the reason is really that Embello assumes you're using these cores. The source for these is in the spezial branch of Mecrisp-Stelaris on GitHub. There are a few small differences (the init change is needed to support USB console I/O on F103).

ok
1> board.fs 26: Redefine init.  ok.
1> board.fs 36: ( board end, size: ) 00007BB4 10164  ok.
Mecrisp-Stellaris RA 2.3.5 for STM32F103 by Matthias Koch
128 KB  3707460D ram/flash: 19356 99328 free ok.
CPU 72000000 Hz, systick 1000 Hz ok.
compiletoflash  ok.
!s 8queens.txt
compiletoram  ok.
10 queens 724 solutions, 34815 nodes ok.
14 queens 365596 solutions, 26992957 nodes ok.
10 testQ 724 solutions, 34815 nodes elapsed 68 ms ok.
14 testQ 365596 solutions, 26992957 nodes elapsed 50892 ms ok.
120MHz  ok.
1000 systick-hz  ok.
10 testQ 724 solutions, 34815 nodes elapsed 42 ms ok.
14 testQ 365596 solutions, 26992957 nodes elapsed 30530 ms ok.
128MHz  ok.
1000 systick-hz  ok.
10 testQ 724 solutions, 34815 nodes elapsed 39 ms ok.
14 testQ 365596 solutions, 26992957 nodes elapsed 28621 ms ok.


rgw/admin: add suport for link/unlink bucket

This change implements link/unlink bucket APIs of
rgw/admin.
Related to #645
Signed-off-by: wanggangfeng<EMAIL_ADDRESS>Checklist

[x] Added tests for features and functional changes
[x] Public functions and types are documented
[x] Standard formatting is applied to Go code
[x] Is this a new API? Is this new API marked PREVIEW?

Thank you! Will look at it soon. 🙂

I have some early feedback already. First, the linter check failed, have a look. You need to add comments to all the newly added exported methods and types. Please also add the  PREVIEW and run make api-update as described here. Thanks!

Thank you for your guidance!
I have pushed two commits for above question.

@ansiwen

I've re-run make api-update, and resolved review comments,re-run test, The version docs should be now ok, PTAL.

PTAL @leseb when you have some time,

@ansiwen All checks have passed, please merge.

@mergifyio squash

close #645


Upgrading to 11.0.0-beta.0 from 8.2.6 causes another linter plugin to start failing

We have a linter plugin (written/published internally) that checks for unnecessary blank lines in our source files. For the purposes of this issue, it's relevant that this plugin prohibits blank lines before the import statements at the top of a file, except when the first line is a comment. In that case, a blank line between the comment and the first import statement is acceptable.
This plugin is passing, but if I upgrade to the latest beta version of babel-eslint, it starts failing, saying that blank lines are not allowed above import blocks. I haven't dug into this, but I'm guessing that somehow the (first line) comments are disappearing. Does babel-eslint somehow modify the files being linted?

Ok closed as intended change


Added logic to allow developers to utilize full request API.

These changes were made so that users can place arbitrary config values into the request field when configuring a Connection in order to manipulate configuration values for the request library.
Included is a very basic unit test that tests out the rawRequest function to ensure these values are passed to request correctly.

Could add a simple documentation of this new feature to readme.MD? Otherwise looks good to me, :+1:.

@panuhorsmalahti let me know what you think about the documentation.

Looks good, :+1:


onTap of gesture detector not recognized

Thank you very much for the creation of this package. Really enjoying it so far!
I am however facing one issue:
I want to use the widget like you did in the example. But the onTap callback of the enclosing gesture detector is not recognized.
I need it to be working though, because if have to await an asynchronous call before show the tooltip. What am I doing wrong here?
My code:
GestureDetector(
      onTap: () async {
        print("test");          <-------- This code is not being run
        await justTheController.showTooltip();
      },
      child: SuperTooltip(
        showBarrier: true,
        controller: justTheController,
        content: const Text(
          "Lorem ipsum dolor sit amet, consetetur sadipscing elitr,",
          softWrap: true,
          style: TextStyle(
            color: Colors.white,
          ),
        ),
        child: Container(
          width: 40.0,
          height: 40.0,
          decoration: const BoxDecoration(
            shape: BoxShape.circle,
            color: Colors.blue,
          ),
          child: Icon(
            Icons.add,
            color: Colors.white,
          ),
        ),
      ),
    )

I have not found any other way to update the state of the tooltip once it has been shown.

Hi @DavidHajum thanks for submitting an issue.
That will not work.
You can add the onShow callback function
Like this.
 SuperTooltip(
       onShow: () => print('When tooltip is showen') // Call back when tooltip is shown
        showBarrier: true,
        controller: justTheController,
        content: const Text(
          "Lorem ipsum dolor sit amet, consetetur sadipscing elitr,",
          softWrap: true,
          style: TextStyle(
            color: Colors.white,
          ),
        ),

hope this will you.

Closing this

Hi! It would be great to know why this not works. I mean, using a wrapping GestureDecorator is recommend by your example.


Ready glossary page for vanilla Docsy

Set front matter for the glossary page so that we can use Docsy-native custom styling, if we want to.
The way this works is to set body_class front matter. This change does not change the way the current site displays.
Current glossary page | Preview glossary page
/area web-development
Helps with issue https://github.com/kubernetes/website/issues/41171

Its good to have this change to align with upstream Docsy plus preview shared in PR (https://deploy-preview-45937--kubernetes-io-main-staging.netlify.app/docs/reference/glossary/?fundamental=true) looks same as current page. So, its a win-win.
/lgtm


[eslint] Enforce react/jsx-indent

Very nice :+1: :+1: Thank you @oliviertassinari
Feel free to merge ^_^

@alitaheri Thanks. The next rules I want to enforce is prefer-const.

The next rules I want to enforce is prefer-const.

That's gonna take a long time. But it's a really good rule. :+1: :+1: :+1:

Yeah, maybe no-console first. I hope there is a codmod for let/const.

@oliviertassinari Should update the eslint-plugin-react dependency to 3.14.0 for this? It looks like they just added support for this in their last version. When I initially got the latest from master, this rule wasn't working for me until I updated.

Yes, It was added very recently. You need to run npm update. But 3.14.0 is inside the scope of our package.json.


ValueError: invalid literal for int() with base 16: ''

Description
Error while running mythril on truffle project through the docker container. A few contracts are analysed correctly, and then I see the following issue:
Traceback (most recent call last):
  File "/usr/local/bin/myth", line 4, in <module>
    __import__('pkg_resources').run_script('mythril==0.18.11', 'myth')
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 658, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 1438, in run_script
    exec(code, namespace, namespace)
  File "/usr/local/lib/python3.6/dist-packages/mythril-0.18.11-py3.6.egg/EGG-INFO/scripts/myth", line 9, in <module>
    mythril.interfaces.cli.main()
  File "/usr/local/lib/python3.6/dist-packages/mythril-0.18.11-py3.6.egg/mythril/interfaces/cli.py", line 147, in main
    mythril.analyze_truffle_project(args)
  File "/usr/local/lib/python3.6/dist-packages/mythril-0.18.11-py3.6.egg/mythril/mythril.py", line 183, in analyze_truffle_project
    return analyze_truffle_project(self.sigs, *args, **kwargs)  # just passthru by passing signatures for now
  File "/usr/local/lib/python3.6/dist-packages/mythril-0.18.11-py3.6.egg/mythril/support/truffle.py", line 51, in analyze_truffle_project
    sym = SymExecWrapper(ethcontract, address, args.strategy, max_depth=args.max_depth)
  File "/usr/local/lib/python3.6/dist-packages/mythril-0.18.11-py3.6.egg/mythril/analysis/symbolic.py", line 30, in __init__
    self.laser.sym_exec(address)
  File "/usr/local/lib/python3.6/dist-packages/mythril-0.18.11-py3.6.egg/mythril/laser/ethereum/svm.py", line 61, in sym_exec
    transaction.run(self.open_states, self)
  File "/usr/local/lib/python3.6/dist-packages/mythril-0.18.11-py3.6.egg/mythril/laser/ethereum/transaction.py", line 58, in run
    evm.exec()
  File "/usr/local/lib/python3.6/dist-packages/mythril-0.18.11-py3.6.egg/mythril/laser/ethereum/svm.py", line 71, in exec
    new_states, op_code = self.execute_state(global_state)
  File "/usr/local/lib/python3.6/dist-packages/mythril-0.18.11-py3.6.egg/mythril/laser/ethereum/svm.py", line 97, in execute_state
    new_global_states = Instruction(op_code, self.dynamic_loader).evaluate(global_state)
  File "/usr/local/lib/python3.6/dist-packages/mythril-0.18.11-py3.6.egg/mythril/laser/ethereum/instructions.py", line 69, in evaluate
    return instruction_mutator(global_state)
  File "/usr/local/lib/python3.6/dist-packages/mythril-0.18.11-py3.6.egg/mythril/laser/ethereum/instructions.py", line 34, in wrapper
    new_global_states = func(self, global_state_copy)
  File "/usr/local/lib/python3.6/dist-packages/mythril-0.18.11-py3.6.egg/mythril/laser/ethereum/instructions.py", line 552, in codecopy_
    int(bytecode[2*(concrete_code_offset + i): 2*(concrete_code_offset + i + 1)], 16)
ValueError: invalid literal for int() with base 16: ''

How to Reproduce
The repository I am trying to run this on, is a public repository: https://github.com/DigixGlobal/dao-contracts
$ docker run -v $(pwd):/tmp -w "/tmp/" mythril/myth --truffle

Environment

OS

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 16.04.5 LTS
Release:	16.04
Codename:	xenial

Docker version:

$ docker --version
Docker version 18.06.1-ce, build e68fc7a

Python version:

$ python --version
Python 3.5.6
$ python3 --version
Python 3.5.6

Mythril version:

$ docker run mythril/myth --version
Mythril version v0.18.11

Solidity compiler and version: solc --version

$ solc --version
solc, the solidity compiler commandline interface
Version: 0.4.24+commit.e67f0147.Linux.g++

hi @roynalnaruto , thanks for pointing it out, this issue was fixed in master but we are yet to have a release(hopefully in a few days), till then can you use the master branch? as it is quite stable.

Hi @norhh, thanks for the quick response.
so I can create the docker image from the master branch?
I am trying to run the docker_build_and_deploy.sh script, this is the output:
./docker_build_and_deploy.sh: 2: set: Illegal option -o pipefail

How can build the Dockerfile (what options) to run a container from master branch?

@roynalnaruto do you mount folder with contracts to the Docker container, as mentioned here?

@norhh I also tried to install the python package by python setup.py install
My python version is 3.5.6
On running myth --version I see this error:
Traceback (most recent call last):
  File "/home/rohit/.local/bin/myth", line 6, in <module>
    import mythril.interfaces.cli
  File "/home/rohit/.local/lib/python3.5/site-packages/mythril/interfaces/cli.py", line 16, in <module>
    from mythril.mythril import Mythril
  File "/home/rohit/.local/lib/python3.5/site-packages/mythril/mythril.py", line 13, in <module>
    from ethereum import utils
  File "/home/rohit/.local/lib/python3.5/site-packages/ethereum/__init__.py", line 11, in <module>
    from . import slogging  # noqa
  File "/home/rohit/.local/lib/python3.5/site-packages/ethereum/slogging.py", line 6, in <module>
    from ethereum.utils import bcolors, is_numeric
  File "/home/rohit/.local/lib/python3.5/site-packages/ethereum/utils.py", line 11, in <module>
    import rlp
  File "/home/rohit/.local/lib/python3.5/site-packages/rlp/__init__.py", line 1, in <module>
    from . import sedes  # noqa: F401
  File "/home/rohit/.local/lib/python3.5/site-packages/rlp/sedes/__init__.py", line 4, in <module>
    from .big_endian_int import BigEndianInt, big_endian_int  # noqa: F401
  File "/home/rohit/.local/lib/python3.5/site-packages/rlp/sedes/big_endian_int.py", line 1, in <module>
    from eth_utils import (
  File "/home/rohit/.local/lib/python3.5/site-packages/eth_utils/__init__.py", line 24, in <module>
    from .applicators import (  # noqa: F401
  File "/home/rohit/.local/lib/python3.5/site-packages/eth_utils/applicators.py", line 43, in <module>
    def combine_argument_formatters(*formatters: List[Callable[..., Any]]) -> Formatters:
  File "/usr/lib/python3.5/typing.py", line 1025, in __getitem__
    tvars = _type_vars(params)
  File "/usr/lib/python3.5/typing.py", line 284, in _type_vars
    _get_type_vars(types, tvars)
  File "/usr/lib/python3.5/typing.py", line 279, in _get_type_vars
    t._get_type_vars(tvars)
  File "/usr/lib/python3.5/typing.py", line 786, in _get_type_vars
    _get_type_vars(self.__args__, tvars)
  File "/usr/lib/python3.5/typing.py", line 277, in _get_type_vars
    for t in types:
TypeError: 'ellipsis' object is not iterable

This error was I think fixed after Python 3.5.2, so it should not occur for me. Is there some dependency issues? Have you guys faced an issue like this?
I think building and using the Docker image would be far more comfortable, but that too has an issue for me (the comment above)

@norhh python3.5 --version also shows 3.5.6

@roynalnaruto it is weird that Docker does not work for you.
Can you try to compile your project to bytecode and then run analysis on the bytecode, like
$ docker run mythril/myth -xc aabc
If this will work, then I'd say the problem is with your Docker setup and permissions to access your host filesystem. Otherwise, something wrong with Docker build or the latest Mythril version itself.

@birdofpreyru it wasn't exactly the docker problem, there was a bug with the previous version and so when he tried to build docker on master branch he is having some issues to build.

hey @roynalnaruto , can you use the latest version of mythril 0.18.12, it doesn't have this bug

@norhh you mean, try to build docker image from the 0.18.12 release available?

hey @roynalnaruto , can you use the latest version of mythril 0.18.12, it doesn't have this bug

@norhh can you please provide me with the necessary steps to proceed in doing so? I downloaded the 0.18.12 tar.gz file from the releases page. I am on an ubuntu 16.04 machine.

@roynalnaruto , can you try doing pip install mythril==0.18.12 .It might be good to do it in python's virtual environment.

@norhh I am in python's virtualenv, and I installed mythril==0.18.12.
When I run myth --version in the virtualenv, I see this:
AttributeError: /home/rohit/Downloads/z3-4.7.1-x64-ubuntu-16.04/bin/libz3.so: undefined symbol: Z3_mk_lambda

Any other command to setup and install the z3 libraries?

@norhh Ok, I think from a virtual environment it can work. I am trying to analyse a truffle project, which has a directory structure, with the contract .sol files inside those directories. It also imports contracts from node_modules directory. Mythril output says the file is outside of allowed directories. How do I explicitly set all of those directories as allowed?
Traceback (most recent call last):
  File "/usr/local/bin/myth", line 11, in <module>
    load_entry_point('mythril==0.18.12', 'console_scripts', 'myth')()
  File "/usr/local/lib/python3.5/site-packages/mythril-0.18.12-py3.5.egg/mythril/interfaces/cli.py", line 154, in main
    mythril.analyze_truffle_project(args)
  File "/usr/local/lib/python3.5/site-packages/mythril-0.18.12-py3.5.egg/mythril/mythril.py", line 180, in analyze_truffle_project
    return analyze_truffle_project(self.sigs, *args, **kwargs)  # just passthru by passing signatures for now
  File "/usr/local/lib/python3.5/site-packages/mythril-0.18.12-py3.5.egg/mythril/support/truffle.py", line 43, in analyze_truffle_project
    sigs.import_from_solidity_source(contractdata['sourcePath'])
  File "/usr/local/lib/python3.5/site-packages/mythril-0.18.12-py3.5.egg/mythril/support/signatures.py", line 181, in import_from_solidity_source
    self.signatures.update(SignatureDb.get_sigs_from_file(file_path))
  File "/usr/local/lib/python3.5/site-packages/mythril-0.18.12-py3.5.egg/mythril/support/signatures.py", line 217, in get_sigs_from_file
    raise CompilerError("Solc experienced a fatal error (code %d).\n\n%s" % (ret, stderr.decode('UTF-8')))
mythril.exceptions.CompilerError: Solc experienced a fatal error (code 1).

/home/rohit/Develop/dao-contracts/contracts/storage/DaoUpgradeStorage.sol:3:1: Error: Source<EMAIL_ADDRESS>not found: File outside of allowed directories.
import<EMAIL_ADDRESS>^--------------------------------------------------------------^
/home/rohit/Develop/dao-contracts/contracts/storage/DaoUpgradeStorage.sol:4:1: Error: Source "/home/rohit/Develop/dao-contracts/contracts/common/DaoConstants.sol" not found: File outside of allowed directories.
import "../common/DaoConstants.sol";
^----------------------------------^

@norhh so I have tried using --allow-paths . to compile the contracts/storage/DaoUpgradeStorage.sol contract using this:
solc --allow-paths . --bin -o compiledout contracts/storage/DaoUpgradeStorage.sol

and it compiles successfully.
But when I use --solc-args "--allow-paths ." along with the mythril command, it still doesn't work and throws the same issue that the file is not in allowed directories.
myth --solc-args "--allow-paths ." --truffle

How am I supposed to use the solc-args option?

@roynalnaruto , currently --truffle option uses truffle's build source, so currently it doesn't have a support for the --solc-args option

@norhh So how do I allow those paths from node_modules? Also, this means Mythril will not be able to analyse a truffle project with contracts inside a directory structure as the dao-contracts repository?

i think just the --truffle option might do that, I am not totally sure, But the truffle module is currently outdated and we are planning to remove that as we are integrating mythril platform with truffle in the near future.

@norhh the --truffle option throws the error that some paths are not allowed (this error is thrown by solc). As I have mentioned in this comment.

I am on version 0.18.12 and I am facing this issue as well with --truffle.

Yeah the ‘—truffle’ option is broken

On version 0.18.11, "--truffle" option works well with both Docker and Mythril installed with pip on Mac.

@ecouffes

On version 0.18.11, "--truffle" option works well with both Docker and Mythril installed by pip on Mac.

Yea, it works on my Mac too, but not on Ubuntu.
@norhh any updates on this issue?


New Cors Settings

Is there info on how the new release changes things with cors. I'm wondering if I need to change my settings.
I'm making a low-rent mint tool here: https://www.xcp.io/mint
And I'm trying to have the client call /compose and getting...
Access to XMLHttpRequest at 'https://api.counterparty.info/v2/addresses/19QWXpMXeLkoEKEJv2xo9rn8wkPCyxACSX/compose/fairmint' from origin 'https://www.xcp.io' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

Nope that wasnt it.

im geting cors error on some routes like v2

@JavierCervilla we need more information to debug this. What exact requests? Can you replicate it with cURL / HTTPie? As far as we know, all of the headers are being sent.

Okay when I made an OPTIONS request I didn't get the headers:
adam@Adams-MacBook-Pro ~ > http OPTIONS https://api.counterparty.io:4000/v2
HTTP/1.1 200 OK
Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Content-Length: 0
content-type: text/html; charset=utf-8
date: Wed, 30 Oct 2024 20:23:38 GMT
server: gunicorn
via: 1.1 google

and I'm now manually injecting them at the LB-level:
http OPTIONS https://api.counterparty.io:4000/v2
HTTP/1.1 200 OK
Alt-Svc: clear
Content-Length: 0
access-control-allow-headers: *
access-control-allow-methods: *
access-control-allow-origin: *
content-type: text/html; charset=utf-8
date: Wed, 30 Oct 2024 20:34:14 GMT
server: gunicorn
via: 1.1 google

I see this behavior is tested here! so something is wonky.
https://github.com/CounterpartyXCP/counterparty-core/commit/9cbfd69832c0ac5093a1e6ff87ba4dfa311a7c1d

verified the headers are missing on localhost too, so they're not getting stripped


Namespace XHP elements for PSR-4 autoloading in Composer

Is it possible to use XHP-elements together with PSR-4 autoloading? If not 'native', maybe a workaround? Through aliasing or some trick?
Problem is that PSR-4 autoloading (http://www.php-fig.org/psr/psr-4/) requires a namespace to map on. As I cannot  see a way to combine custom namespaces with the XHP syntax I cannot seem to get autoloading to work with Composer, using PSR-4 compliant autoloading (https://getcomposer.org/doc/01-basic-usage.md#autoloading and custom https://getcomposer.org/doc/04-schema.md#autoload).
Any thoughts?

This is basically a wontfix - however, Composer can now autoload this, via a classmap.


Latex not rendered correctly

Should be a single line not multiple

Here's what it should look like (single line latex output)

This needs to be done in the renderer extension repo
Update nteract and related libraries

Moving to parking lot as no one has reported this issue, hence not a priority

Moving to parking lot as no one has reported this issue, hence not a priority

Still an issue with the native notebooks builtin renderer.


Feat: Add arrows on image detail for Basilisk

Thank you for your contribution to the KodaDot NFT gallery.
👇 _ Let's make a quick check before the contribution.
PR type

[ ] Bugfix
[x] Feature
[ ] Refactoring

What's new?

[x] PR closes #3660
[ ] Requires deployment <rubick/magick_issue>
[ ] <brief_description_of_what_I've_added>

Before submitting Pull Request, please make sure:

[x] My contribution builds clean without any errors or warnings
[x] I've merged recent default branch -- main and I've no conflicts
[x] I've tried to respect high code quality standards
[x] I've didn't break any original functionality
[x] I've posted a screenshot of demonstrated change in this PR

Optional

[ ] I've tested it at </rmrk/collection/26902bc2f7c20c546a-1FVG7>
[x] I've tested PR on mobile and everything seems works
[ ] I found edge cases
[ ] I've written some unit tests 🧪

Had issue bounty label?

[x] Fill up your KSM address: Payout

Community participation

[x] Are you at KodaDot Discord?

Screenshot

[x] My fix has changed something on UI; a screenshot is best to understand changes for others.

desktop:

mobile:

nice fire, we had same recently

pay 100

😍 Perfect, I’ve sent the payout
💵 $100 @ 61.92 USD/KSM ~ 1.615 $KSM
🧗 Caiv9TbPz68q5dC8EcHu5xKYPRnremimGzqmEejDFNpWWLG
🔗 0xc9c46e13197e8e962ef42eafca2e49040ec374b76436d63a075047634686ba78
🪅 Let’s grab another issue and get rewarded!
🪄 github.com/kodadot/nft-gallery/issues


[SDK Automation] Breaking change should be re-evaluated on each run

Context
Currently, the SDK breaking change will be evaluated on each run of the pipeline. But the breaking change labels will be adjusted accordingly only when suppression file exist. When suppression file doesn't exist, if the generated SDK in the latest pipeline run removes the potential breaking change, the related BreakingChange-Sdk-{Language} won't be deleted.
Expected
Breaking change labels need to be adjusted even if the suppression file does not eixst.

This issue has been fixed.


DRAIN "into" description is odd

Drain 2d6 into SPD (20 Active Points); OAF (-1), OAF (-1), No STR Bonus (-1/2).  Is this a 5e TRANSFER leftover?  The word into should not be here.  Looks to be associated with (the yet to be release) 3.0.53 as this does not seem to be an issue with 3.0.52.

Ya. Probably my stuff. Looking at things closer there are few oddities in the descriptions of a few adjustment powers I'll have a look at.


fix: using getQueryInclude inside findOptions hook

The info object provided by the findOptions hook was not specific to the current field so the result of getQueryInclude was inaccurate (only working at the root level).
This was fixed in https://github.com/accesimpot/graphql-lookahead/pull/104, so this PR simply upgrade graphql-lookahead.

:tada: This PR is included in version graphql-gene-v1.3.1-beta.1 :tada:
The release is available on:

npm package (@beta dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version @graphql-gene/plugin-sequelize-v1.2.2-beta.1 :tada:
The release is available on:

npm package (@beta dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Middleware state

Based on the conversations over in #339, I am opening this issue so that we can keep track and discuss the current issue of middleware not being able to maintain it's own state independently of the bot.
For the sake of clarity, the problem right now is that middleware is not able to store state in a de-coupled way from the bot itself (i.e. it would need to know about the structure of the POCO being used for bot state) and therefore this makes it impossible to build middleware that can be generic and used across multiple bots where the middleware requires some state management. Ideally the bot state needs to move to some similar form of collection with get / set (maybe still using POCOs - e.g. middleware would implement it's own POCO for state management) so that various pieces of the puzzle (middleware components and the bot) can store their own state object - probably with a named key for the cases where a piece of middleware is used multiple times on a bot and require a separate state object for each.
@billba over to you to provide some additional info based on your discussions I think? 👍

This has always been possible, in that you can create your own storage key formula. For instance the following creates state scoped to a conversation, but specific to a namespace. (Obviously this is JS so please apply the necessary mental transformation for C#)
interface MyMiddlewareState {
  foo: string;
}

const MyMiddleware {
  static namespace = "github.com/billba/MyMWState";

  constructor(storage) {
    this.myState = new BotState<MyState>(storage, MyMiddleware.namespace, context =>
      `channel/${context.channelId}/conversation/${context.conversation.id}/${MyMiddleware.namespace}`
    );
  }

  async onProcessRequest(context, next) {
    const state = await this.myState.read(context);
    // maybe do stuff with state.foo
    await next();
    // maybe do stuff with state.foo
    await this.myState.write(context);
  }
}

Yesterday we decided to make this a little more turnkey. Specifically:

All storage keys get a namespace suffix, e.g. channel/{channelId}/conversation/{conversationId}/{namespace}
new ConversationState() gets a namespace argument (still to be decided: optional or required)

Please let me know if this solves the problem as you see it.

@billba Ah, that wasn't my understanding of how state worked, at least in C# right now.  I thought that recent (within the last few weeks) changes were made to state that meant that essentially you could only have a single state object e.g. 'MyBotState' per bot.  If that is not the case then great, but we need to confirm.  Certainly when I last looked I could not see a way of achieving the above from an isolated perspective within middleware and that the only option was to add additional middleware specific properties to the 'MyBotState' object.  This is effectively my issue - if it is possible to have multiple state objects and add / create these within middleware then fantastic.
@cleemullins @drub0y Is the above example from @billba analogous with the C# SDK right now?

It looks identical. Here's the relevant code for .net

Related JS conversation: https://github.com/Microsoft/botbuilder-js/issues/129

@billba I totally see that I can add additional state objects, where I could make one specifically for my middleware. However, wouldn't that mean that I also needed to register my state middleware (given that BotState is itself a piece of middleware) prior to registering the middleware that used it? Otherwise at what point in the pipeline would the state be persisted?

I made this same mistake myself before @Stevenic cleared it up for me.
BotState can act as middleware, but it's purely optional. You always have the ability to persist it yourself, and I did so above in the last line:
await this.myState.write(context);

I think this is the right pattern when middleware creates its own private state.

@billba Wondering if this is where the divergence is between C# and JS.  Right now BotState is an abstract class in C# meaning like something like this is not possible from within middleware.  You can sub-class BotState, but the read / write methods are protected which mean that you cannot implement them on the sub-class.
this.myState = new BotState<MyState>(storage, MyMiddleware.namespace, context =>
      `channel/${context.channelId}/conversation/${context.conversation.id}/${MyMiddleware.namespace}`
    );

It looks like the way ConversationState and UserState are implemented right now means they have to be registered and used as middleware in order to persist their state.
Again - I have been thinking about this for sometime now and I might just being going round in meaningless circles in my head, but I have tried to implement it in the way you suggested above and I just don't see how it can be done right now on the C# side.

Those sound like bugs in the C# implementation.

What may be causing confusion is that there are some extension methods that hide this functionality.

Honestly I think it's the fact that you have to configure a middleware for each TState you intend to store  (and at each level you intend to store it) vs a single piece of middleware that enables state as a feature and then all the other state management APIs "light up" and let you store as many pieces of state of as many types as you want.
I think the changes we have discussed are going to greatly improve the experience here. The foundational work around services is in PR review now. Once that's in I'll be able to spike on the PoC for the changes to state management and we'll have some "real" code (vs whiteboard only) that demonstrates the full changes that are being proposed and start to have a real discussion around them... even if that leads to everyone scoffing at me in disgust. 😆

You can sub-class BotState, but the read / write methods are protected which mean that you cannot implement them on the sub-class.

I don't think in the current API you would aim to subclass BotState, you would just new up a new subclass that targets which state you want to store and where... like new ConversationState<MyState>.

BotState being abstract in C# sounds like a bug. Same with read and write being protected.

This stuff is (hopefully) going to change pretty substantially, so we might want to hold off improving what's there and make sure the new stuff looks right.

@drub0y so just to confirm the case now, am I right in suggesting that in order to use the state functionality now in C# you do need to register some middleware and this cannot be done totally isolated within the middleware that requires state?

I strongly feel we should fix what's there. They are both small fixes, and together they unblock @garypretty's scenario. Moreover we should implement the namespace feature as agreed upon (also a small change). Then if @drub0y's cunning plan comes to fruition, all the better.

@drub0y Do you think I am right in saying that we can just make the BotState class non-abstract and make the read / write methods public vs protected to accomplish this?  If that's the case I would love to get this in as it will unblock a bunch of scenarios I want to progress in terms of some useful middleware.
I could also add an optional string param which could be used as a namespace as @billba is suggesting.

Yes. In my mind this is two PR's, one to bring C# BotState into alignment JS, another to add namespace support to [Converstion|User]State.

@drub0y Maybe you can see if the above changes are really straightforward to make. i.e. allow you to do the following in C#.  I have just had a look and once I make the read / write public and remove the abstract from BotState I am mostly there, but then the Read method returns StoreItems and not StateT.
interface MyMiddlewareState {
  foo: string;
}

const MyMiddleware {
  static namespace = "github.com/billba/MyMWState";

  constructor(storage) {
    this.myState = new BotState<MyState>(storage, MyMiddleware.namespace, context =>
      `channel/${context.channelId}/conversation/${context.conversation.id}/${MyMiddleware.namespace}`
    );
  }

  async onProcessRequest(context, next) {
    const state = await this.myState.read(context);
    // maybe do stuff with state.foo
    await next();
    // maybe do stuff with state.foo
    await this.myState.write(context);
  }
}

Also - @billba - Thanks for working through this. Glad we finally figured out that right now what is possible in JS is not possible in C#. Getting this small change in will really help! :)

That looks wrong to me. Here is the corresponding read in JS: https://github.com/Microsoft/botbuilder-js/blob/master/libraries/botbuilder-core-extensions/src/botState.ts#L53

@billba Agreed. If nobody else gets a chance I'll take a look later (heading out for a few hours now) and see if I can at least bring the read / write into alignment.

@garypretty Sorry, I have been working with a partner and haven't had a chance to keep up with everything going on. 😊
Yeah, so, I mean... your analysis of the current state is fair and everything you point out is absolutely true with the design choices that have been made up to this point. That said, I'm pretty sure thereis a way to work with it the way it is now. It's not great, but, yeah... you have to .Use(...) each instance of a new ConversationState<T> or UserState<T> for each T you want to store in each particular state "bucket". Yes it does mean the app has to register N middleware for all the Ts have of state you might want and yes you have to live with the auto-saving, but it SHOULD work because each one of those will be persisted into its own namespace based on the T.
As for changes, I personally have a hard time justifying spending any time on rev'ing the current design in any way because I know it's about to radically change to solve all these issues and enable all sorts of scenarios that we can't even do right now with the existing design.

All totally makes sense @drub0y - Thanks for confirming my current understanding. If the changes to state are imminent then I understand not making further changes now. Thanks for working through though. All ultimately helps towards the final understanding /solution.

@garypretty Just to be totally clear, there is no booked plan to make the changes @drub0y is discussing, but I greatly admire his confidence. I still think the right course is to bring C# into alignment with JS right now because it's hard to know how long it will take @drub0y to sell his plan.

@billba That being the case, if it is potentially going to take some time, then I would agree that an interim solution of bringing things into line would be great.  I am happy to do the work if we can agree between us what needs doing. This will allow me to crack on a build some great generic middleware options for developers using v4 which can be packaged up and placed on NuGet.
@cleemullins @tomlm @Stevenic Can you guys weigh in on this please so that we can figure out what the best way forward is here?

but it SHOULD work because each one of those will be persisted into its own namespace based on the T.

Soooo, I just put this idea to the test and actually doesn't work because, while the state is stored under the services collection according to it's different namespace which includes the T:
https://github.com/Microsoft/botbuilder-dotnet/blob/18252f42a1bdcdb026acfe60d3e385e56fb7e811/libraries/Microsoft.Bot.Builder.Core.Extensions/BotState.cs#L94
the so called keySelector implementation for UserState<T> and ConversationState<T> actually always returns a singular string for all instances regardless of T:
https://github.com/Microsoft/botbuilder-dotnet/blob/18252f42a1bdcdb026acfe60d3e385e56fb7e811/libraries/Microsoft.Bot.Builder.Core.Extensions/BotState.cs#L98
So in order to even use the idea I was proposing you'd need to at least bug fix that.

Right that's why I specifically suggested using BotState.

@drub0y @billba Ok, this is only an initial stab at a couple of small alterations to allow BotState to be used as middleware via ConversationState / UserState or on it's own.  Take a look at https://github.com/garypretty/botbuilder-dotnet/blob/ae698e2087861d6e20edcbdb7cba41cfe727265c/libraries/Microsoft.Bot.Builder.Core.Extensions/BotState.cs

Nice. That sample code looks right to me.

It was low enough hanging fruit to fix without completely breaking the current model, so seems reasonable to me! I say PR it with concise explanation. 👍

Closing this so we can keep conversation in the PR / DCR raised by @drub0y

I want to reopen this issue since the State Management DCR PR (#376) has been put on hold.

This has been resolvbed with the merging of #345


handle self signed certificates for http client

The user can specify whether TLS is enabled in the config, and provide a custom ca certificate, which will be used to validate the requests:
https://github.com/opendatahub-io/caikit-nlp-client/blob/20791c6333afbf490c8e31fb9fdf18cefb7d6102/src/caikit_nlp_client/http_client.py#L11-L18
https://github.com/opendatahub-io/caikit-nlp-client/blob/20791c6333afbf490c8e31fb9fdf18cefb7d6102/src/caikit_nlp_client/http_client.py#L47-L59


Navigation without triggering route matching

Great library. Thanks for your work!
Is there a way to set the url hash without triggering the route handlers?
I've tried path(newPath), but that triggers them just like navigate(newPath).
I can imagine doing it with preventDefault() in a pretty hackish way - I'd need to have an ignoreMatching flag of some sort, but turning this on and off safely seems rather fragile. (eg, I'd need to clear the flag whether or not a match was found, and worry about event orders.)
Is there a more efficient way to do it? Some other libraries just have a silently option for navigate().

+1 on this question as well. thanks

There currently isn't a way to do this. But you can use preventDefault() as a workaround. The window always fires the hashchange or popstate events, but you can specify whether or not you want the router to continue. @thunderkid @droplab

Do you mean something like this as a workaround?
    this.router.on('hashchange', (event) => this.onPopstate(event, this.router.path()));

then
    onPopstate(event, path: string) {
        if (path == pathThatIJustSet)
            event.preventDefault();
 }

when I try this I get that event is undefined. Thanks.

@thunderkid maybe he meant this: https://github.com/baseprime/grapnel/issues/46#issuecomment-129550677


fix bug in instance evaluation

There is a bug in your instance evaluation when filtering out small instances. The current version leaves a gap in the label maps and will consequently give wrong results.

@ykirchhoff Yes, a condition statement is lacking when filtering out all small instances. Thanks for your reminder. One function can be used here:
from skimage.morphology import remove_small_objects
remove small objects lower than 11 pixels
if post_processing == True:
pred = remove_small_objects(np.bool_(pred), min_size = threshold, connectivity = 2)
gt = remove_small_objects(np.bool_(gt), min_size = threshold, connectivity = 2)
if np.sum(pred) == 0 or np.sum(gt) == 0:
(dice, jc, pr, se, F1, hd, asd)  = (1,1,1,1,1,0,0) if np.sum(pred) == 0 and np.sum(gt) == 0 else (0,0,0,0,0,np.inf,np.inf)
print("Either the prediction or label does not contain any object.")


Feature Request: Prevent customization from showing up in the tabs.

Would it be possible to introduce an option to remove the customizations for links do not show up in the tabs names displayed above the contents of the file? In stacked view, they are rather distracting.

There should already be such an option in the latest release?

There should already be such an option in the latest release? Or maybe i misunderstand

Just checked you are correct. Thanks a lot.


Implement geocoding

We need two way geocoding.

When markers are set on the map an adress close by needs to be searched and displayed in the search box
When an adress is entered into the serach box an according coordiante needs to be resolved

The original gh-maps-client already did something on this topic probably @karussell kann give some insight on this.

When markers are set on the map an adress close by needs to be searched and displayed in the search box

It might be a nice feature. You can use "reverse" geocoding, e.g. see this example request:
https://graphhopper.com/api/1/geocode?reverse=true&point=49.932707,11.588051&key=<yourkey>
Still personally I'm not a big fan of this user experience. You can go to https://www.openstreetmap.org/directions where this is done. Paste e.g. 49.167339,9.624023 in the start location and if you tab or click into the destination input it will replace the numbers with a text. Now the problem on osm.org is that the operation is not always an identity, so if you have:
text = reverse_geocode(coordinates)
new_coordinates = geocode(text)

then new_coordinates is not always equal to coordinates. Which makes problems if you use the "text" to share the URL and you expect the identical route, but you'll get the route with new_coordinates instead.
I think google maps is also doing this but they use text only as some UX representation and internally (as well as in the URL) uses the coordinates and so a shared URL will always look the same (as it uses the coordinates). But it might be tricky to get it right :)

When an adress is entered into the serach box an according coordiante needs to be resolved

Here the "normal" geocoding can be used:
https://graphhopper.com/api/1/geocode?q=dresden&key=<yourkey>
Please note that our API supports different geocoding providers and the best provider is IMO provider=nominatim as it covers many use cases and countries. Please note that it does not support auto completion, but when this is needed we could switch and use provider=default? (This is the way we do it for GraphHopper Maps)

Implemented foreward geocoding in cbcb8d3e977d6c8e5debf1d62b51362700b865d8

:+1:
I'm not sure if reverse geocoding is required or should be high priority for now as there is a certain complexity involved and the current GH maps does not have it. The problem is that even if we reverse geocode lat+lon and use the prettier location name for the search boxes, we still need to store the lat+lon pairs in the UI state (and URL) to avoid imprecision due to lat+lon->name->different lat+lon.

Closing for now

Ups -> I cannot close it :D
@Janekdererste Would you mind to move the repo into the graphhopper org?

Closing this evn though @karussell could now :-)


How to fetch old metrics data from stackdriver

Hi,
I have connected Grafana with Prometheus and fetching the metrics data using stackdriver_exporter. I am able to see only the current data which is fetched, but not able to see the old data. How can I fetch the old metrics from stackdriver?

This is not supported.


Publish 8.4-20230905054818+0000

Publish 8.4-20230905054818+0000

@bot-gradle test and merge

Your PR is queued. See the queue page for details.

I've triggered a build for you. Click here to see all failures if there's any.


add a table with dataset sizes

having a table with dataset and some information about size/ time to download would be useful
https://docs.google.com/document/d/1KCAB-OTHphcCh-4oITIL8r7ih-HuslMKX1Rls_P03CY/edit could serve as complementary information

I will add information here as I download things.
Starting with CC3M, I intend to download it then produce some clip embeddings (using https://github.com/rom1504/clip-retrieval/) / list of clip filtered files
Once it's clear enough, will PR to readme

I downloaded cc3m and cc12m (improving their script a bit in the process)

cc3m took 20h and resulted in 100GB of resized images, 5.6M of them, in size on dimension 320 the other larger
cc12m took 10h and resulted in 300GB of resized images, 10.7M of them, in size 256

cc3m can obviously take way less time if using the improved script of cc12m
I confirmed in the process that handling million of files is painful and will make it possible to download directly as collection of tars (== webdataset format)

@rom1504 the doc is not available now.
i want to download the data, can you please help me.
I just find download_open_images.txt file in the repo. how to download using text file ?


Update windows code signing task

Old keys expired. New keys are being issued now, but industry standards changed to require a FIPs certified HSM (YubiKey). As such, need to rework signing tasks to use a YubiKey with osslsigncode software.
Also, I'm using a different CA and will need to update the timestamp server to use ssl.com instead of comodo.

Notes, in case I need to revisit this issue later on down the road:
pkcs11-tool --module /usr/lib/x86_64-linux-gnu/libykcs11.so -L

https://github.com/Yubico/yubico-piv-tool/issues/21
https://github.com/mtrojnar/osslsigncode/issues/306


Made some minor changes to Sdk end to end tabular custom training notebook

Made some minor changes to Sdk end to end tabular custom training notebook
If you are opening a PR for Community Notebooks under the notebooks/community folder:

[x] This notebook has been added to the CODEOWNERS file under the # Community Notebooks section, pointing to the author or the author's team.
[x] Passes all the required formatting and linting checks. You can locally test with these instructions.

changed as per review comments

ingestion test not ran:
Changed files did not match file filter

/gcbrun

/gcbrun

@ivanmkc test been running since June 27

hi,
This PRis been closed ,Please find the new pr link below
https://github.com/GoogleCloudPlatform/vertex-ai-samples/pull/1292


Gulp Pug on Linux

I've recently moved my environment from OSX to Linux (Manjaro).
My pug files properly compiles under OSX but won't on Linux. Running my task gives no error:
23:02:40] Using gulpfile ~/project/gulpfile.js
[23:02:40] Starting 'pug'...
[23:02:40] Finished 'pug' after 9.77 ms

However running npm gives me this warning message:
npm WARN optional SKIPPING OPTIONAL DEPENDENCY<EMAIL_ADDRESS>(node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for<EMAIL_ADDRESS>wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

I've searched online and they've mentioned this was a non-issue.
Does anyone know what's going on here?

That has nothing to do with gulp-pug (it's the file watcher for OSX).

Ok, any idea why my pug files aren't compiling on linux?


Consider adding logging for TryRefreshAsync method

Add logging for failures during IConfigurationRefresher.TryRefreshAsync to allow users to fix errors during the refresh operation. Check the discussion in https://github.com/Azure/AppConfiguration-DotnetProvider/pull/119 for more details. Also allow users of the middleware to register an ILogger to retrieve failure logs.

@avanigupta @jimmyca15
While it's useful to add logs when anything goes wrong during the refresh, it's also helpful whenever a configuration update has happened. Users need a positive signal to confirm a successful configuration update. Is this something we can add?

Sure, we could add an informational log every time IConfiguration is updated by a refresh operation. To keep it simple, we could start with a single log entry when anything is updated:
IConfiguration updated from Azure App Configuration.
Alternatively, we could provide more details by indicating which type of key-value changed. For example:
Key Vault secret(s) updated from Azure Key Vault.
Feature Flag(s) updated from Azure App Configuration.
Key-Value(s) updated from Azure App Configuration.
Or if LoadAll() was triggered:
IConfiguration updated from Azure App Configuration.

Sounds good, @avanigupta. What's the difference between the first and the third?

I meant that we have two options:
Option 1: A single log for any kind of changes:
- IConfiguration updated from Azure App Configuration.
Option 2: Multiple logs indicating which type of key-value changed - this will further be divided into two categories and we will pick the log depending on the condition:
A. LoadAll() was triggered so the entire configuration was updated:
- IConfiguration updated from Azure App Configuration.
OR
B. LoadAll() was not triggered but something specific was updated (could be 1, 2 or all 3 types were updated during the same refresh operation):
- Key Vault secret(s) updated from Azure Key Vault.
- Feature Flag(s) updated from Azure App Configuration.
- Key-Value(s) updated from Azure App Configuration.

That sounds like a debugging scenario to me. What's the motivation for this?

This is not for debugging. It creates the full circle for change and health monitoring. For example, in our service, we like to see logs whenever a change happened - either it's a cert rotation or it's a resource cleanup. In the config provider, we are making changes on behalf of the users, if we don't log those changes, users will have a hard time adding their own.
@avanigupta how about 2.A and 2.B? I think it's important to see the specific logs for secret rotation.

Echo here, may I know the progress of the feature? We are considering adding logs by ourselves to notify configuration changes.

Error/warning logs are already supported in the latest release of the provider library.
This PR is adding informational logs: #354


Fix typo

great - thank you :) It's my most common typo :D and my spellchecker seems to have enough of me and stopped alerting me ...


Newsletter signup: Add e2e tests

It's a pretty important part of the site, but AFAICT it's not tested anywhere.
Once we're all-in on using the Dotdigital API, we should be able to mock responses to it and make sure it works correctly.

Won't do


Added ability to add message_properties with variables.

When adding message_properties dictionary (message_properties => {"content_type" => "const type"})
You can now use dynamic values from variables {"content_type" => "%{ContentType}"}
Thanks for contributing to Logstash! If you haven't already signed our CLA, here's a handy link: https://www.elastic.co/contributor-agreement/

hi @jsvd @sophie135,
is there any plan to release this feature?
It would be very useful to be able to set the priority of the message dynamically.
message_properties => {"priority" => "%{priority}"}

@jeslopalo I have revived this feature in the new RabbitMQ Integration Plugin: https://github.com/logstash-plugins/logstash-integration-rabbitmq/pull/9
Since the Integration Plugin requires Logstash 6.5, I'd appreciate your input on backport requirements to get this feature working for you since you contributed.


What is the license for this version of EPANET?

Most (if not all?) OS projects have a LICENSE or LICENSE.txt (to be friendly to win users) on the main repository.

touchy subject, but an important point. original EPANET is public-domain, the community version is probably MIT or similar. We need to finalize this and post the LICENSE

@samhatchett, sure BSD or MIT should both work, I guess (I am partial to MIT, but it is more a thing of tradition than nothing else)?
Thanks for the reply

For anyone tuned into this issue... After too much deliberation among the organizers, I've put up the MIT license on the repo. If any contributor has "non-trivial" code in the mix, and if you don't agree to license your code under these terms, please let me know as soon as possible.
Perhaps I posted the license too soon, without explicit acknowledgement from each and every contributor. But my reasoning was that since the original code was Public Domain (US Gov't code), I didn't think that the MIT license would be an unreasonable burden, nor grant unexpected freedoms to users of the code. We've just been sitting on this issue for so long that something needed to be done.

The license is MIT. Time to close the issue and move on... thanks all.


Provide an example for dynamically specifying allowed methods for Slim 4

I've trouble dynamically specifying the allowed methods while using Slim4 like you did in the readme file for Slim3
use Fastroute\Dispatcher;
use Tuupola\Middleware\CorsMiddleware;

$app->add(
    new CorsMiddleware([
        "origin" => ["*"],
        "methods" => function($request) use ($app) {
            $container = $app->getContainer();
            $dispatch = $container["router"]->dispatch($request);
            if (Dispatcher::METHOD_NOT_ALLOWED === $dispatch[0]) {
                return $dispatch[1];
            }
        }
    ])
);

I've tried the following for Slim 4:
$app->addMiddleware(new Tuupola\Middleware\CorsMiddleware([
  'methods' => function ($request): array {
	$routeContext = RouteContext::fromRequest($request);
	$routingResults = $routeContext->getRoutingResults();
	$methods = $routingResults->getAllowedMethods();

	return $methods;
  },
]));

But either when I put $app->addRoutingMiddleware(); in front of the code block above I get the following error:
PHP Fatal error:  Uncaught RuntimeException: Cannot create RouteContext before routing has been completed in /var/www/html/vendor/slim/slim/Slim/Routing/RouteContext.php:40
Stack trace:
#0 /var/www/html/public/index.php(103): Slim\\Routing\\RouteContext::fromRequest()
#1 /var/www/html/vendor/tuupola/cors-middleware/src/CorsMiddleware.php(175): Closure->{closure}()
#2 /var/www/html/vendor/tuupola/cors-middleware/src/CorsMiddleware.php(91): Tuupola\\Middleware\\CorsMiddleware->buildSettings()
#3 /var/www/html/vendor/slim/slim/Slim/MiddlewareDispatcher.php(147): Tuupola\\Middleware\\CorsMiddleware->process()
#4 /var/www/html/vendor/slim/slim/Slim/MiddlewareDispatcher.php(81): Psr\\Http\\Server\\RequestHandlerInterface@anonymous->handle()
#5 /var/www/html/vendor/slim/slim/Slim/App.php(215): Slim\\MiddlewareDispatcher->handle()
#6 /var/www/html/vendor/slim/slim/Slim/App.php(199): Slim\\App->handle()
#7 /var/www/html/public/index.php(127): Slim\\App->run()
#8 {main}
  thrown in /var/www/html/vendor/slim/slim/Slim/Routing/RouteContext.php on line 40

or when I put $app->addRoutingMiddleware(); after the code block, the Cors middleware doesn't get invoked.
I can mitigate the first error where it complains about the RouteContext when I implement a OPTIONS catch-all block like so:
$app->options('/{routes:.+}', function (ServerRequestInterface $request, ResponseInterface $response, $args): ResponseInterface {
  return $response;
});

But I wonder if I can get rid of the OPTIONS catch-all block altogether, what I'm missing here?

I am not sure if I have ever used that feature with Slim 4. Most API projects I have are still Slim 3. I try to find if I have example somewhere. I just remember there has never been easy way to get that info.

same 👍🏻

It works for me with the following config:
// init container
// ...

// init middleware
        $app->options('/{routes:.+}', fn ($request, $response, $args) => $response);

        $app->add(new CorsMiddleware([
            'logger' => $container->getLogger(),
            'origin' => array_merge(['http://localhost'], Settings::get('api.custom_safe_hosts')),
            'methods' => function (ServerRequest $request): array {
                $routeContext = RouteContext::fromRequest($request);
                $routingResults = $routeContext->getRoutingResults();

                return $routingResults->getAllowedMethods();
            },
            'headers.allow' => ['Authorization', 'If-Match', 'If-Unmodified-Since'],
            'headers.expose' => ['Authorization', 'Etag'],
            'credentials' => true,
            'cache' => 60,
            'error' => function (ServerRequest $request, \Slim\Psr7\Response $response, array $arguments) {
                $arguments['origin'] = $_SERVER['HTTP_ORIGIN'];

                return $response
                    ->withHeader('Content-Type', 'application/json')
                    ->withBody((new StreamFactory())->createStream(json_encode($arguments, \JSON_UNESCAPED_SLASHES | \JSON_PRETTY_PRINT)));
            },
        ]));

// init rest of middleware, including routing middleware
$app->run();


Fixed a little typo in readme

:)

I think

+A PNG, JPEG or BMP binary Buffer of an image (e.g. for storage in a database) can be generated using

sounds even better (but I'm not a native english speaker)