gi-gtk 3.0.4 build failure (ghc bug)

> /tmp/stackage-build8$ stack unpack gi-gtk-3.0.4
Unpacked gi-gtk-3.0.4 to /tmp/stackage-build8/gi-gtk-3.0.4/
> /tmp/stackage-build8/gi-gtk-3.0.4$ runghc -clear-package-db -global-package-db -package-db=/home/stackage/work/builds/nightly/pkgdb Setup configure --package-db=clear --package-db=global --package-db=/home/stackage/work/builds/nightly/pkgdb --libdir=/home/stackage/work/builds/nightly/lib --bindir=/home/stackage/work/builds/nightly/bin --datadir=/home/stackage/work/builds/nightly/share --libexecdir=/home/stackage/work/builds/nightly/libexec --sysconfdir=/home/stackage/work/builds/nightly/etc --docdir=/home/stackage/work/builds/nightly/doc/gi-gtk-3.0.4 --htmldir=/home/stackage/work/builds/nightly/doc/gi-gtk-3.0.4 --haddockdir=/home/stackage/work/builds/nightly/doc/gi-gtk-3.0.4 --flags=
Configuring gi-gtk-3.0.4...
> /tmp/stackage-build8/gi-gtk-3.0.4$ runghc -clear-package-db -global-package-db -package-db=/home/stackage/work/builds/nightly/pkgdb Setup build
Building gi-gtk-3.0.4...
Preprocessing library gi-gtk-3.0.4...
[  1 of 654] Compiling GI.Gtk.Flags     ( GI/Gtk/Flags.hs, dist/build/GI/Gtk/Flags.o )
...
[599 of 654] Compiling GI.Gtk.Objects.Widget ( GI/Gtk/Objects/Widget.hs, dist/build/GI/Gtk/Objects/Widget.o )
ghc: panic! (the 'impossible' happened)
  (GHC version 8.0.1 for x86_64-unknown-linux):
        urk! lookup local fingerprint
  $fAttrInfoWidgetMarginTopPropertyInfo4

Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

Note that gi-gtk is meant to be constrained to == 3.0.3, "since stack does not understand [custom-setup] syntax yet", but this package-and-constraint is commented out per "GHC 8".

Ping @juhp; I've disabled ghcjs-dom and jsaddle.
General note: the above occurred during the June 8 nightly (in case we need to look up specific dependency versions in order to reproduce).

Ping @garetxe, in case you haven't seen this error yet.

Here's how I attempted to reproduce on my mac:
brew install gtk+3
stack unpack gi-gtk-3.0.4
cd gi-gtk-3.0.4
stack init --resolver nightly-2016-06-08
PKG_CONFIG_PATH=/usr/local/opt/libffi/lib/pkgconfig/ stack build

This causes a different panic:
[650 of 650] Compiling GI.Gtk           ( GI/Gtk.hs, .stack-work/dist/x86_64-osx/Cabal-<IP_ADDRESS>/build/GI/Gtk.o )
ghc: panic! (the 'impossible' happened)
  (GHC version 8.0.1 for x86_64-apple-darwin):
        RnEnv.plusAvail bindingSignalWriteNext CheckButtonK{CheckButtonK}

Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

I'm able to reproduce the original panic locally by running the build on the stackage nightly docker image.
docker run -it snoyberg/stackage:nightly
cd ~
stack unpack gi-gtk-3.0.4
cd gi-gtk-3.0.4
stack init --resolver nightly-2016-06-08
stack build

Reported to ghc trac: https://ghc.haskell.org/trac/ghc/ticket/12168#ticket

Thanks for reporting this to ghc trac!
I had not seen this bug. There was a report looking somewhat similar (https://github.com/haskell-gi/gi-gtk-hs/issues/12) but I could never reproduce it locally (and the code itself is compatible with ghc 8.0, as far as I can tell). It is very good that you can reproduce this reliably.


plam_detection output

I use the netron see the model
but I don't understand output meaning.
In regressors output is 1* 2944* 18 (2944 is anchor numbers )
(18 is coords in the doc.)
but coords why has 18 numbers?
x y w h ? only 4

Hi, each anchor has (x, y, w, h) offsets for the box and and (x, y) for 7 keypoints each, thus 4 + 2 * 7 = 18.

Hi, each anchor has (x, y, w, h) offsets for the box and and (x, y) for 7 keypoints each, thus 4 + 2 * 7 = 18.

If we read the graph, we can find that the InferenceCalculator is followed by a TensorsToDetectionsCalculator. The tensors to detections decoding is done in this calculator. The tensors are converted to bbox and keypoints at https://github.com/google/mediapipe/blob/master/mediapipe/calculators/tensor/tensors_to_detections_calculator.cc#L991 and https://github.com/google/mediapipe/blob/master/mediapipe/calculators/tensor/tensors_to_detections_calculator.cc#L996.

If we read the graph, we can find that the InferenceCalculator is followed by a TensorsToDetectionsCalculator. The tensors to detections decoding is done in this calculator. The tensors are converted to bbox and keypoints at https://github.com/google/mediapipe/blob/master/mediapipe/calculators/tensor/tensors_to_detections_calculator.cc#L991 and https://github.com/google/mediapipe/blob/master/mediapipe/calculators/tensor/tensors_to_detections_calculator.cc#L996.

fanzhanggoogle and jiuqiant thank you.
I understand finally.
classificators output generate each hand probability with anchors, then we can find the compliant anchors.
Find the best anchor index using nms.
Use anchor.csv find corresponding center x,y for hand.
Use the best index regressors output find the corresponding x,y offset, plam bbox size and 7 keypoint offset in hand.
(If there is an error, please advise)
Schematic diagram

fanzhanggoogle and jiuqiant thank you.
I understand finally.
classificators output generate each hand probability with anchors, then we can find the compliant anchors.
Find the best anchor index using nms.
Use anchor.csv find corresponding center x,y for hand.
Use the best index regressors output find the corresponding x,y offset, plam bbox size and 7 keypoint offset in hand.
(If there is an error, please advise)
Schematic diagram

@u0251077 Hallo, thanks for your explanation. I wonder how to get the anchor.csv file.

@20XYSong I reference this
https://github.com/metalwhale/hand_tracking/blob/master/models/anchors.csv

Thank you very much. I've already read this repo. But I want get the anchor.csv of recent palm detection model, where the output details show that output_shape becomes 'array([   1, 2016,   18])'. You can get this info by loading 'palm_detection_full.tflite' is mediapipe python package.

For people that are looking for a csv of new  SSD anchors, I've attached the file here, along with code to generate it. Thanks @saknarak for your snippets!
https://github.com/VimalMollyn/GenMediaPipePalmDectionSSDAnchors


Colab fails to import QueryConfig

When executing first block shows:
Installing collected packages: pypika, monotonic, filetype, docx2txt, dash-table, dash-mantine-components, dash-iconify, dash-html-components, dash-core-components, brotli, antlr4-python3-runtime, XlsxWriter, Werkzeug, websockets, uvloop, unstructured.pytesseract, socksio, safetensors, retrying, rapidfuzz, pytube, python-multipart, python-magic, python-iso639, python-dotenv, python-docx, pytesseract, pypdfium2, pypdf, pypandoc, pulsar-client, portalocker, pdf2image, overrides, onnx, omegaconf, olefile, mypy-extensions, marshmallow, langdetect, jedi, hyperframe, humanfriendly, httptools, hpack, h11, emoji, ebooklib, chroma-hnswlib, beautifulsoup4, bcrypt, backoff, ansi2html, aiofiles, youtube-transcript-api, watchfiles, uvicorn, typing-inspect, tiktoken, starlette, python-pptx, posthog, msg-parser, langsmith, iopath, huggingface-hub, httpcore, h2, coloredlogs, tokenizers, pdfminer.six, openai, onnxruntime, httpx, fastapi, dataclasses-json, dash, unstructured, transformers, pdfplumber, langchain, jupyter_dash, dash-bootstrap-components, chromadb, layoutparser, fast-dash, duckduckgo-search, timm, effdet, unstructured-inference, embedchain
  Attempting uninstall: Werkzeug
    Found existing installation: Werkzeug 3.0.0
    Uninstalling Werkzeug-3.0.0:
      Successfully uninstalled Werkzeug-3.0.0
  Attempting uninstall: beautifulsoup4
    Found existing installation: beautifulsoup4 4.11.2
    Uninstalling beautifulsoup4-4.11.2:
      Successfully uninstalled beautifulsoup4-4.11.2
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
lida 0.0.10 requires kaleido, which is not installed.
llmx 0.0.15a0 requires cohere, which is not installed.
Successfully installed Werkzeug-2.2.3 XlsxWriter-3.1.8 aiofiles-23.2.1 ansi2html-1.8.0 antlr4-python3-runtime-4.9.3 backoff-2.2.1 bcrypt-4.0.1 beautifulsoup4-4.12.2 brotli-1.1.0 chroma-hnswlib-0.7.3 chromadb-0.4.14 coloredlogs-15.0.1 dash-2.14.0 dash-bootstrap-components-1.5.0 dash-core-components-2.0.0 dash-html-components-2.0.0 dash-iconify-0.1.2 dash-mantine-components-0.12.1 dash-table-5.0.0 dataclasses-json-0.5.14 docx2txt-0.8 duckduckgo-search-3.9.3 ebooklib-0.18 effdet-0.4.1 embedchain-0.0.72 emoji-2.8.0 fast-dash-0.2.6 fastapi-0.103.2 filetype-1.2.0 h11-0.14.0 h2-4.1.0 hpack-4.0.0 httpcore-0.18.0 httptools-0.6.1 httpx-0.25.0 huggingface-hub-0.17.3 humanfriendly-10.0 hyperframe-6.0.1 iopath-0.1.10 jedi-0.19.1 jupyter_dash-0.4.2 langchain-0.0.279 langdetect-1.0.9 langsmith-0.0.44 layoutparser-0.3.4 marshmallow-3.20.1 monotonic-1.6 msg-parser-1.2.0 mypy-extensions-1.0.0 olefile-0.46 omegaconf-2.3.0 onnx-1.14.1 onnxruntime-1.15.1 openai-0.27.10 overrides-7.4.0 pdf2image-1.16.3 pdfminer.six-20221105 pdfplumber-0.10.2 portalocker-2.8.2 posthog-3.0.2 pulsar-client-3.3.0 pypandoc-1.12 pypdf-3.16.4 pypdfium2-4.21.0 pypika-0.48.9 pytesseract-0.3.10 python-docx-1.0.1 python-dotenv-1.0.0 python-iso639-2023.6.15 python-magic-0.4.27 python-multipart-0.0.6 python-pptx-0.6.21 pytube-15.0.0 rapidfuzz-3.4.0 retrying-1.3.4 safetensors-0.4.0 socksio-1.0.0 starlette-0.27.0 tiktoken-0.4.0 timm-0.9.7 tokenizers-0.14.1 transformers-4.34.0 typing-inspect-0.9.0 unstructured-0.10.24 unstructured-inference-0.7.7 unstructured.pytesseract-0.3.12 uvicorn-0.23.2 uvloop-0.18.0 watchfiles-0.21.0 websockets-11.0.3 youtube-transcript-api-0.6.1
WARNING: The following packages were previously imported in this runtime:
  [pydevd_plugins]
You must restart the runtime in order to use newly installed versions.

it doesn't matter if restart or not, second block:
ImportError                               Traceback (most recent call last)
[<ipython-input-2-e4d2717a5af4>](https://localhost:8080/#) in <cell line: 5>()
      3
      4 from embedchain import App
----> 5 from embedchain.config import QueryConfig
      6
      7 from string import Template

ImportError: cannot import name 'QueryConfig' from 'embedchain.config' (/usr/local/lib/python3.10/dist-packages/embedchain/config/__init__.py)

---------------------------------------------------------------------------
NOTE: If your import is failing due to a missing package, you can
manually install dependencies using either !pip or !apt.

To view examples of installing some common dependencies, click the
"Open Examples" button below.
---------------------------------------------------------------------------

Hi @estebanavv, what embedchain version are you using?

@estebanavv Hoping the above solved your, so closing this.


reassign Readline constants

Hello all
https://github.com/ConnorAtherton/rb-readline/pull/131 was merged ,
so now can not get history command using up and down arrow in ruby-pry console. ^^
This is my test Gemfile  :
source 'https://rubygems.org'
gem 'pry'
gem 'rb-readline', '0.5.4'

bundle exec pry
　#  up/down arrow not working

This is module load steps:

load pry
1.1 load the native readline (see lib/pry/config/default.rb )
load rb-readline
2.1 can not overwrite Readline::HISTORY because it defined in native readline (see here)

So the warning warning: already initialized constant Readline::HISTORY is not a problem,
we should be reassign Readline constants as rb-readline class.

PS: This Gemfile is working , and already initialized constant warnings will not be displayed.
source 'https://rubygems.org'
gem 'pry'
gem 'rb-readline', github: 'dulao5/rb-readline', branch: 'reassign_readline_constants'

Hey, is someone? ^^


ASAR docs

While asar states it 'protects source code from users', that doesn't explain how it does, or what exactly you mean. I assume it actually means 'from easily modifying the source', but doesn't provide any real protection via obfuscation etc.

yes, asar archives have pretty simple structure and can be read just fine

I'm trying to reword the docs, but can't really come up with a reason to use it. ASAR gives you $SOME_BENEFIT at $SOME_COST? I have no idea from the docs.

it also helps a bit with long paths on windows as well as some performance gain of modules require

Closed with #2376.


Find in .jar?

Any chance we can get a find feature implemented that allows us to do a text search of all files, or at least all files in a folder/subfolder we define? Unless I'm missing an easier way, it's a pain to search through a bunch of different class files just to find where some method is called.

2e316c56cbc5b345854a662ac4b53b4a8387cd90
Ctrl-Click Navigation makes following code path a bit easier.

Great, thanks so much @deathmarine! :+1:

Closed?

I think this is resolved.


Subtyping / convertibility / assignability / unification ...

Originally posted by @netdpb in https://github.com/jspecify/jspecify/issues/275#issuecomment-1252837006:

Elsewhere I've said that the question at hand is one of convertibility and not general subtyping and that it might be counterproductive to refer to @NonNull String as a subtype of @Nullable String, much as int is convertible to Integer without being a subtype of it. I don't think it's necessary to think about type variable substitutions as being just any subtype of the upper bound, where the nullness component just gets subsumed into the general idea of subtypes. It may be possible to describe the variable nullness of type variables more specifically than that while still referring to the upper bound.

By @cpovirk in a glossary comment:

I see that you started moving away from "assignability." Yay!
I think that we should try to broaden our focus from "conversion" to "subtyping" (which applies in cases in which conversions don't, like checking type arguments).

By @scolsen on wiki/type-system:

[A type system defines] A set of calculations over those types. Most importantly, unification determines when one type can be substituted for another to make two types equal; in other words, unification can answer the question "can type A be supplied where type B is expected?".

An older issue: #80
My take on all this to follow :-)

I'm sorry I don't have all the theory and terminology, and I'm sorry that I'm going to try at this anyway.
Important: I still think that all questions involving unspecified nullness should immediately defer to many-worlds; in what follows, unspec doesn't exist.
I suppose the question an analyzer's nullness "type system" repeatedly tries to answer is
In context C, type T is needed, but type F is supplied. What do?
(Contexts: is it assignment to a variable, is it method argument passing, is it bounds checking, etc.)
I think this is normally addressed, say in Java's type system, by seeing "which type conversions are available in context C" and seeing if any work. I think we don't call it a type "check", which implies a boolean result, because we have messy cases: maybe it's fine, maybe it's fine IF you cast, maybe it's fine IF we insert some code to convert the value, maybe it's no way no how.
But it seems (??) simpler than that for a nullness analyzer: no inserting conversion code, and well #252 addresses casts.
I would think we only need to think about binary relations over types, with each context having a defined set of those relations available.

Identity - every context should support this; some only support this, saying that the types must match exactly
Subtyping - if all you care to know is that "every F is already an T", this seems right. (And I would insist that subtyping really must be reflexive/transitive/antisymmetric.)

I don't know what else we need. Not "widening" (as distinct from "subtyping"); Java "widens" ints into longs, we don't do things like that.
To me so far it seems like subtyping -- non-null <: parametric <: nullable -- works out splendidly for us. Doesn't it?

These are good questions, I might comment on this more tomorrow, but with respect to terminology at least, here are some thoughts I have.
First of all, I should warn that my understanding and use of type system terminology is largely cobbled together from knowledge of Hindley-Milner (HM) style type systems and some extensions of them. I think Java's type system deviates from the core concepts most by relying on subtyping pervasively and not really having a notion of unification, my sense is that the conceptual apparatus holds, but there may be greater differences than I'm aware of.
Here's how I view unification, conversion, and subtyping:

Unification is the process by which the type system validates or "type checks" programs. Each expression in a program is assigned initial types based on user provided type annotations and built-in type to token associations (for things like numeric literals). Unification effectively involves performing a series of substitutions between types until final types are realized, at which point all possible equations between the expected types of expressions are either satisfied or one or more is not satisfied and we have a type error.
Conversion is a different, but closely related, operation from unification that involves changing the meaning of a program by altering its types to satisfy type equations. Unification substitutions hold over an entire context. If some expression e is assigned a concrete type, it is a type error if the unification process finds that e must have an assignment of a different concrete type in the same context to satisfy some equation. Contrarily, conversion allows this for special cases. Using conversion, the same expression e may be typed at one instance as concrete type short and at another as the concrete type long. Normally this would be an invalid mapping of two distinct types under unification, but special conversion rules permit such a double usage. The unification of two types does not imply that a conversion exists between them. For example, while any two types can unify against a wholly generic type variable, this does not imply that these two types may be converted to/from each other. Likewise, that a conversion between types exists in a special case does not imply general unification support between the types (e.g. Java will convert a non-string to a string when passed to + but will refuse to unify (or convert) an int with a String when passing an argument to a function).
Subtyping is a preorder relationship among types that effects unification. For example, both int and short may be subtypes of number. This essentially provides a new rule to the unification process that tells it anywhere it expects a number type, either of these types is an acceptable substitution. Note that this does not imply that any possible conversion between such types necessarily exists.

Based on my understanding, Java differs most from this account insofar as it elects to prefer conversions across a large subtyping hierarchy and doesn't really have a notion of unification. Instead of unifying types, it performs conversions along the hierarchy. This makes sense since all reference types inherit from a common supertype. So, it may be better to leave out this notion of unification, though I personally find it easier to reason about and think it proper to conceptualize it as something different from conversion which implies the possibility of alterations at the level of complied code in a way that I feel unification doesn't. Another way to put it would be that unification respects the typing relationships expressed in your program while conversion reserves the right to blithely disregard them for your "convenience".
I do think we can probably get away with talking about subtyping only, occasionally making mention to conversions if needed. This would also put us closest to Java's own account of its type system (imo, and from what I've gathered).

Can anyone clarify what it should take to mark this issue closed?

I'm not certain what decidable question this issue covers.
Is it just about whether we should use the term "subtype of" or "convertible to"? Is it just in response to my way of thinking about parametric nullness as not a distinct kind of nullness but instead just another part of the type hierarchy over which a type parameter can vary?
If it's either of those things, I'm sorry if my comments made anyone think this is a question that blocks progress. I don't think it is.

If you and Chris can confirm that you don't see anything I said in https://github.com/jspecify/jspecify/issues/291#issuecomment-1255691555 that looks wrong to you then I think we're fine.

I do think that, in cases in which Java would normally apply a subtyping check, we need to apply an "unspecified-aware-subtyping-like" check[*], which can be non-transitive (in "lenient mode") or non-reflexive (in "strict mode"). I think your point may be that, if the result of the check is not a boolean answer, then maybe the check can still be transitive and reflexive? I'm not actually sure what that would mean for a non-boolean answer, though. Also, you go on to talk about relations, which I also think of as having a boolean answer.
I also think that the check needs to operate on full augmented types, not just on "nullness" values (even setting aside the discussion going in #238).
I see the current spec approach of defining "subtyping" and "same type" on augmented types as the way to go. That may be enough, or it may be enough when supplemented by the nudging supplied by our tests.
[*] Sorry, I know you don't like that word here.


Received both a Transfer-Encoding and a Content-Length header, refusing as recommended in RFC 7230 Section 3.3.3.

Starting with the release of mitmproxy 7.0.3, mitmproxy does not accept requests or responses anymore that include both a Content-Length and a Transfer-Encoding header. This follows the recommendation in RFC 7230 (emphasis mine):

If a message is received with both a Transfer-Encoding and a
Content-Length header field, the Transfer-Encoding overrides the
Content-Length.  Such a message might indicate an attempt to
perform request smuggling (Section 9.5) or response splitting
(Section 9.4) and ought to be handled as an error.  A sender MUST
remove the received Content-Length field prior to forwarding such
a message downstream.

We don't have any telemetry/analytics in mitmproxy, so we don't know how widespread requests with both headers are. To err on the side of being safe, we have decided to reject such requests for now. If this affects you, please comment below. If we find that messages with both headers are much more prevalent than we anticipate, we may return to a less strict handling.

Well, this is mildly annoying. In my case, the server is sending only the Content-Encoding header, but once I intercept the response and modify it, Content-Length header gets added by mitmweb and I get this error

Thanks for the feedback @cotneit! @matthewhughes934 has PR ready at #4824 that addresses this, we should have another bug fix release for this particular issue in a few days.

7.0.4 is out, which fixes this issue. :)

While this covers (some) HTTP desync scenarios, the behavior is a but harsh for an interception proxy use case (not only if you operate mitmproxy while testing for desync vulns).
I found a request - hitting this logic - in the wild (originating from Knowunity Android App):

As "aborting" such requests does not fit my mitmproxy use-cases, I tried to work around this via a requestheaders hook:
    def requestheaders(self, flow: http.HTTPFlow):
        cl: Union[None, str] = flow.request.headers.get("Content-Length")
        te: Union[None, str] = flow.request.headers.get("Transfer-Encoding")
        if cl is not None and te is not None and te.lower().find("chunked") >= 0:
            ctx.log.warn("Found CE and TL header in request to: " + flow.request.url)
            # CL and TE header, while TE allows "chunked" encoding
            # --> remove CL
            del flow.request.headers["Content-Length"]

Unfortunately, this didn't work out for me

check_body_size internally expected_http_body_size, while the latter raises the ValueError in case TE and CL headers exist in the request.
Could this be prevented via (optional) requestheaders hook from above, if the hook is placed before the check_body_size call, here: https://github.com/mitmproxy/mitmproxy/blob/7e24e77ac4d72183c2181b46364d3df3be6aed6a/mitmproxy/proxy/layers/http/init.py#L198L201 ?

Solved this for myself via monkey-patching, for now. Making it work from within the requestheaders hook, would be better IMHO.
    def load(self, loader):
        from mitmproxy.net.http import http1
        self.old_expected_http_body_size = http1.expected_http_body_size
        # monkey patch 'expected_body_size' to remove CL header in case it appears along with 'TE: ...chunked...'
        def new_expected_http_body_size(request: http.Request, response: Optional[http.Response] = None) -> Optional[int]:
            cl: Union[None, str] = request.headers.get("Content-Length")
            te: Union[None, str] = request.headers.get("Transfer-Encoding")
            if cl is not None and te is not None and te.lower().find("chunked") >= 0:
                ctx.log.warn("Found CE and TL header in request to: " + request.url)
                # CL and TE header, while TE allows "chunked" encoding
                # --> remove CL
                del request.headers["Content-Length"]
            return self.old_expected_http_body_size(request=request, response=response)

        http1.expected_http_body_size = new_expected_http_body_size

    def done(self):
        if self.old_expected_http_body_size:
            http1.expected_http_body_size = self.old_expected_http_body_size

Resulting request which got CL header removed (errored out in previous screenshot):

Thanks for sharing your workaround! If we find that there are more examples of this in practice, we should add a mitmproxy option that disables this check.
https://github.com/mitmproxy/mitmproxy/blob/df32d610869c73abf0840e148a0d7cdc64b1f61e/mitmproxy/proxy/layers/http/_http1.py#L236-L237 is where things start to go haywire.

mitmproxy 8 now has a validate_inbound_headers option which allows to disable this check. 😃

Still fails for me even with the option with https://redmine.lighttpd.net/boards/2/topics/7425.

@DUOLabs333: The following works fine for me.
mitmdump --set validate_inbound_headers=false
curl -x localhost:8080 -k https://redmine.lighttpd.net/boards/2/topics/7425

Oh, I put False, should that still work?

Weird, it allowed it for me, I'll try again.

Yes, it works for me now. Don't know why it didn't say anything the last time.


Render every row on dashboard tables

Renders every row on the My Paused Requests and My Starred Requests tables on the dashboard page.
Note that this will render every row even if there are thousands - but there won't be thousands unless a user stars every request in the system for some reason.
cc @tpetr @wolfd @kwm4385

LGTM after last piece of feedback


Wip/fbeutin/solana code improvements post refacto

Continuation of
https://github.com/LedgerHQ/app-solana/pull/40

can we merge this? so we get closer to the complete version


fitView not working on page load - Svelte Flow

What platform were you using when you found the bug?

React Flow / Svelte Flow version:
- @sveltejs/kit": "^2.0.0",
- svelte": "^5.0.0",
- @xyflow/svelte": "^0.1.24"
Browser and version: Chrome Version 130.0.6723.70
OS and version: macos sequoia 15.1.1

Live code example
No response
Describe the Bug
Just testing out Svelte flow using this example:  https://svelteflow.dev/examples/layout/horizontal-flow
When I add the example code to page.svelte the example loads fine, it just doesn't fit the view properly.  When I click the icon for fitView, the view is centered and fits the screen properly.
I don't have an online code example.  I did copy the sample code for horizontal flow into the svelte starter on stackblitz and it renders fine there.  I only see the issue when using svelte kit.
Steps to reproduce the bug or issue

generate a new svelte kit project with example code
install svelte flow
create a new fold in routes named svelteflow and add +page.ts and +page.svelte
copy example code in app.svelte and dpast into +page.svelte
create file nodes-and-edges.ts and copy sample nodes and edges into it
update sample svelte kit file Header.svelte to add a new link for svelteflow folder
navigate to the new page

Expected behavior
As a user I expect to see the horizontal view sample code centered on my page and zoomed to fit the page, but instead it is zoomed out and the diagram is at the top of the page and part of it is out of the viewing area.
Screenshots or Videos

Additional context
No response

Found a temp fix by using svelte onMount and within setTimeout clicking the control button for fit view.  I tried the useSveltFlow hook by calling fitView() in the setTimeout and that didn't work.
onMount(() => { 		const btn = document.querySelector('.svelte-flow__controls-fitview'); 		setTimeout(() => { 			btn.click(); 			//fitView(); 		}); 	});

Update...when clicking a navigation link to load the page with the diagram, the diagram is centered and fit to width as expected.  The issue seems to only occur when the page refreshed by refreshing the browser.


vLLM endpoint support

[x] I have read and agree to the contributing guidelines.

Hello, I'm trying to connect a locally hosted LLM to to a prompt engine, we are deprecating our use of HuggingFaceTextGenInference TGI, and are switching to vLLM for future use. I tried using the OpenAIPromptDriver but it looks like it attempts to connect to openai servers regardless of the base_url provided.
I'd like a prompt driver that takes in an endpoint (vLLM) and allows for parameters to be passed for tuning, and authorization. Does one already exist? I'm new to this so I might be missing something
Thanks

Hi @s-m-palmier, can you please share a minimal reproducible example of the OpenAiChatPromptDriver not using base_url correctly? We have used it with success on services like TogetherAi.

@collindutter I'm working on an endpoint that I can't send out, but basically - it's not an OpenAI endpoint. The LLM is hosted internally and was setup to run using TGI, but now it's moved to vLLM and i'm wondering if any of your prompt drivers can handle something more generic, like an inference endpoint, model_name, tuning parameters, and an api_token. The OpenAiPromptDriver seems close, but it fails to look outside of OpenAI servers.
import os
from griptape.structures import Agent
from griptape.drivers import OpenAiChatPromptDriver
from griptape.rules import Rule
from griptape.config import StructureConfig, StructureGlobalDriversConfig
from dotenv import load_dotenv

load_dotenv()

agent = Agent(
    config=StructureConfig(
        global_drivers=StructureGlobalDriversConfig(
            prompt_driver=OpenAiChatPromptDriver(
                base_url="https://[my_model_endpoint]",
                api_key=os.getenv('[API_TOKEN]'),
                temperature=0.1,
                max_tokens=2048,
                model="Mixtral-8x7B-Instruct-v0.1",
                seed=42,
            )
        )
    ),
    input_template="You will be provided with a sentence, please provide a one word classification of the sentiment. Sentence: {{ args[0] }}",
    rules=[
        Rule(
            value='Write your output in all caps'
        )
    ],
)

agent.run("I really hate it here.")

renders the following error:
WARNING:root:model not found. Using cl100k_base encoding.
failed (SSLError HTTPSConnectionPool(host='openaipublic.blob.core.windows.net')
I don't want it to look on that host, I'm trying to reach my host at my url. how can I do that?

Ah, that is coming from the OpenAiTokenizer which is trying to look up the Mixtral-8x7B-Instruct-v0.1 model. You should be able to resolve by giving OpenAiChatPromptDriver a Tokenizer that is built for Mixtral (from HuggingFace, for instance). You can also use SimpleTokenizer as a quick workaround.

Also going to point to this comment which may assist. I am going to close this issue for now but please feel free to re-open if you still face issues!


devise "flash messages" in json response in API-only rails apps

Hello there!
Is there a possibility to include devise's flash messages, into the devise json response, when using devise in an API-only rails application?
Searched the docs/issues and the web, without success.
For example when I make an http request to DELETE /users/sign_out, an empty response body gets returned.
Would be nice if there would be some confirmation in json format like "Signed out successfully." like it's returned in a normal rails app in the flash. Otherwise it's maybe not really clear for the consumer of the api if he really got logged out.
Additionally all the other messages in config/locales/devise.en.yml could be useful too, having them included in the json response.
With the advent of Rails API-only mode, and abundance of JS frontend frameworks, this would be a nice upgrade to devise to make it more suitable for API-only apps.
Thanks a lot!

For API request the http response returns if something was successful or not. Flash messages only make sense for web pages.

Hey @rafaelfranca,
thanks for your answer!
ok leaving this bit of api consumer friendliness aside of having devise messages included in the json response.
Though this would be a nice side-effect by considering the following, which was actually my main intention about including devise messages in the json response:
What if you want to use the messages, devise gives you, in your frontend that gets served by your api? You would need to duplicate all of that message functionality, depending of the status code you get back.
Wouldn't this be the opposite of DRY, since this functionality is already built-in in devise?
Additionally the messages of devise are directly intended for the user of your app, so you will most likely want to include them in your frontend. Why not just include them in devise's json response by default then, like they are included in the flash in an html response by default ( no matter if you use the flash or not )?
This would extend an api app with the same functionality as a rendered HTML app ( regarding the message system of devise ).
Or am I missing smth?
Thanks!

If you need that behavior it is easy to override the default session controller to do so. But I don't think it should be default in devise.

Could you specify the reason why you don't think this should be an option in devise?
Is an API-only use not intended for devise in general?


Message info

Message Info
Details
This plugin analyzes the post fix log file and propagates message details and statuses to the king phisher database.
Testing

[x] Allow a user to change what log file is analyzed in the server_config.yml file.
[x] The plugin will correctly analyze only the newly updated information in the log file.
[x] The plugin correctly analyzes details and statuses.
[x] Propagate the analyzed information to the correct columns in the server messages table.
[x] Enumerate user permissions to verify that the user king_phisher_server is going to be running as has the correct permissions over the post fix log file.

Landed after making some style-oriented changes in commit ae2b295db45d98bd5b25411208e96f16db163d7b and testing it. Thanks @Nib0x62s !


Shadows with icons causes low framerates on Firefox

Here is a repro JSBin demonstrating the issue: https://jsbin.com/guzulegaxu/4/edit?html,output. For completeness, here is the full example inlined into this issue too:

Full repro code
<!DOCTYPE html>

On Firefox (confirmed with 77.0.1 64-bit on MacOS - but anecdotally every version since we started using vis several years ago), trying to do any interaction (panning, zooming, moving a node) when more than 10 nodes have shadows and icons leads to an fps of around 1!
I can not reproduce the issue on either Chrome or Safari.

On further investigation it appears that drawing shadows is known to be slow in general on firefox. I suppose it makes sense this would be much worse for character icons that simple shapes.


PR: Bump pylint to 3.1 and python-lsp-server to 1.11

Description of Changes
Bump Pylint to the new version
Issue(s) Resolved
See #21936
Affirmation
By submitting this Pull Request or typing my (user)name below,
I affirm the Developer Certificate of Origin
with respect to all commits and content included in this PR,
and understand I am releasing the same under Spyder's MIT (Expat) license.

I certify the above statement is true and correct: bnavigator

Bumped pylsp because of python-lsp/python-lsp-server#543

Hello @bnavigator! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file external-deps/python-lsp-server/pylsp/python_lsp.py:

Line 116:80: E501 line too long (96 > 79 characters)


[MODULE] - Annotator split / random int generator

Please describe the module you would like to add to bricks
A simple function that uses the random library to return a random integer. This can be used to then split data for different annotators.
Do you already have an implementation?
import random
def annotator_split(record):
    n_annotators = 5
    return random.randint(0, n_annotators-1)

Additional context
This would be a generator brick module.

super simple, but really helpful as an attribute function to split data slices. I love it! Generally, we could look into grouping bricks into splits for annotation with the newly gained partOfGroups. I'm thinking for instance that

language
sentence complexity
reading time

could all be part of that group :)

I wrote a module for it :)


Reasoning should be false by default, not None

https://github.com/modelfabric/reactive-sparql/blob/e3ec7c08c56cea0b9d01fb979dd1a4483b0d8c1e/src/main/scala/com/modelfabric/sparql/api/SparqlQuery.scala#L63

The client supports arbitrary SPARQL endpoints. The fact that you can, but are not required to pass the "reasoning=[true/false]" query parameter in the outbound HTTP request is modelled through the use of this Option[Boolean]. Basically the same thing as with "offset" and "limit" parameters.
Some endpoints may treat the non-existence of the "reasoning" parameter with a different default behaviour. We want to leave that option open to anybody using this library as for their Client.
Leaving default to "None" means, don't tell the HTTP endpoint what kind of reasoning you want. Let the endpoint behave according to it's internal default behaviour. I believe in Stardog this would mean to run the query with the reasoner off.


How do we pass TraceID across other application services to get the distributed tracing context

Prerequisites
Please check the FAQ, and search existing issues for similar questions before creating a new issue.YOU MAY DELETE THIS PREREQUISITES SECTION.

[x] I have checked the FAQ, and issues and found no answer.

Just curious and to get the fundamendals right  - How do we pass TraceID across other application services to get the distributed tracing context
I am under the impression that Pinpoint propagates traceid over the HTTP headers and If yes then any specific pinpoint variable we pass along with the request header ?
@yjqg6666 @emeroad @koo-taejin

Can someone please help advice on this ?

See this article
http://naver.github.io/pinpoint/plugindevguide.html


prepare for nmdc-schema v9.4.0 release

coordinate with @aclum @eecavanna and @mbthornton-lbl wrt whether the v9.3.0 to v9.4.0 migrator is ready to include
update version strings in nmdc.yaml and pyproject.toml

I consider it ready.

see

#1701
#1702


[BLC-19940] Add breeding timer support + missing survival features

for https://github.com/BadlionClient/mc17client/pull/8063

lgtm


"latest" files in GCS buckets no longer updated

The "latest" files in the GCS bucket, e.g. https://storage.googleapis.com/symbiflow-arch-defs-gha/symbiflow-toolchain-latest point to an old version of the toolchain (currently build 597 of fb1b251a from 2022-03-22), which makes it impossible for me to download a recent archive - looking at the GCS bucket would require owning and logging into a google account. Something in the CI pipeline should probably update those files when a new archive is pushed.

Updating the "latest" pointers was broken during the transition from Kokoro to custom GitHub Actions runners, and renaming the default branch from master to main. FTR, it was reported in the slack channel last week.
We are working on it:

https://github.com/SymbiFlow/f4pga-arch-defs/commit/6529d853ff8540ab7a849f28cd68fe2d86fca209
https://github.com/SymbiFlow/f4pga-arch-defs/pull/2665

Unfortunately, CI in this repo takes ~10h. Therefore, it is tedious to have these changes tested.

#2665 was merged 6 days ago, the latest links are still not being updated. What's the current status here? Maybe the links themselves have changed with the Symbiflow->F4PGA rename?

@Xiretza, there is some issue with credentials and it's taking longer than expected to restore the procedure that updates the GCS buckets. Meanwhile, we implemented an alternative using GitHub Releases. See https://github.com/SymbiFlow/f4pga-arch-defs/releases/tag/latest.
We will update and close this issue when GCS buckets are updated again. Then, you can decide to keep using the Releases or go back to using the GCS urls.


Unable to run the code on Windows

Hi,
I've been trying to run your project on windows using my own data, the library dependencies were crazy( I was not able to install all of them using the "environment.yml" file. Anyways, I finally managed to get the code running in Google Colab, which is somehow able to install the "JAXLIB=0.3.15", but when I try to run the same code and install the libraries on my local machine(windows), I get the error below:
pip install jaxlib==0.3.15
"ERROR: Could not find a version that satisfies the requirement jaxlib==0.3.15 (from versions: 0.3.24, 0.3.25, 0.4.0, 0.4.1, 0.4.2, 0.4.3, 0.4.4, 0.4.6, 0.4.7, 0.4.9, 0.4.10, 0.4.11, 0.4.12, 0.4.13, 0.4.14, 0.4.15, 0.4.16, 0.4.17, 0.4.18, 0.4.19)
ERROR: No matching distribution found for jaxlib==0.3.15"
I'm not the expert on installing packages but I was wondering if there is a way around the issue? like changing the version of Jax to be able to use one of the compatible versions of Jaxlib which are available in windows?
Any clues are highly appreciated, thanks for taking the time in advance.

Hi there! jaxlib was not officially supported on windows until 0.3.24. As described in the readme, there is an environment_windows.yml that installs the unofficial windows build.
Regarding your comment on the "crazy" dependencies, I just deleted the version specifications in the windows environment file, which should make it easier to install. Let me know if this works for you.


Can a new version be released?

2.7.1 was released 6 months ago, but master shows the same version in the metadata. What is required to cut a 2.8 release with this change 215145c949d, from August?
Can the metadata in master be bumped to a new version after release?

To be clear, let me know how I can help. If you want me to do the tagging, verifying, uploading to the supermarket.

+1 on this. I'm waiting on the same change in a new release.

:+1: Also waiting for this to be released - anything we can do to push this forward?

2.7.2 released yesterday. @someara thanks, and feel free to close this issue!


the $ServicePrincipalDisplayName does not exist - changing to $Applic…

…ationDisplayName
the $ServicePrincipalDisplayName does not exist - changing to $ApplicationDisplayName

Hi @SubZer0MS, I'm your friendly neighborhood Azure Pull Request Bot (You can call me AZPRBOT). Thanks for your contribution!

This seems like a small (but important) contribution, so no contribution license agreement is required at this point. Real humans will now evaluate your PR.

TTYL, AZPRBOT;

@MGoedtel Matt, can you please review and let us know if this can be merged?

Approving edit.  Thanks Mihai for bringing that to my attention.

with pleasure :D I used the script for a customer and noticed this. cool article btw!


Crash: signal 4 (SIGILL), code 1 (ILL_ILLOPC) libfilament-jni.so

Describe the bug
I use Sceneform in my Android app. For some users, I have got the following crash:
signal 4 (SIGILL), code 1 (ILL_ILLOPC)
libfilament-jni.so
pid: 0, tid: 0 >>> com.mycompany.myapp <<<
backtrace:
#00 pc<PHONE_NUMBER>052eae /data/app/com.mycompany.myapp-PuYPm-sE_PEw==/lib/arm/libfilament-jni.so
To Reproduce
Steps to reproduce the behavior:
Random crash. Only on Android 9 devices (not on Android 8).
Expected behavior
No crash :)
OS
Android 9.x
Any idea how to solve this issue?
Thanks !

I am getting a lot of crash reports on the Google Play Console with the same issue. Especially from the device Motorola moto g(7) play (channel), Android 9 (as listed in Play Console).
Note that this devices has Android 9 and has a 32 bit system image (even though the hardware supports 64 bit).
Would be nice to see at least a reply on this topic, from the crash report there's nothing I can do as a developer.

@PauGuillamon I see the same problem on this specific device in our app in the Google Play Console as well. Looking forward to a reply on this issue or a fix in one of the next releases.


How query a object through relations

I can't find examples, i trying to do
Get all books with topics "adventure"
I have 3 models, books, tags and topics, books has a relation many-to-many with topics and many-to-many with tags
var Book = bookshelf.Model.extend({
    tags: function () {
        return this.belongsToMany(Tag);
    },

    topics: function(){
        return this.belongsToMany(Tag);
    }
});

var Books =  Models.Bookshelf.Collection.extend({model: Book});

var Tag = bookshelf.Model.extend({
    books: function () {
        return this.belongsToMany(Book);
    }
});

var Topic = bookshelf.Model.extend({
    books: function () {
        return this.belongsToMany(Book);
    }
});

So i need do something like
// GET /books?topic="adventure"
var query = {where: {type: 'book'}};

// this queries are optional
if (req.query.topic || req.query.tag) {
    query.andWhere = function(qb) {
        if (req.query.topic) {
            qb.orWhere('topics.name', '=', req.query.topic);
        }
        if (req.query.tag) {
            qb.orWhere('tags.name', '=', req.query.tag);
        }
    };
}

Books.query(query)
    .fetch({withRelated: ['topics', 'tags']})
    .then(function(books) {
        res.json(books);
    })
    .catch(function(err) {
        console.log(err.stack);
        res.status(500);
        res.json(err);
    });

but the generate SQL is wrong
'select "books".* from "books" where "type" = $1 and ("topics"."name" = $2)'

How i can add the required join table to this query?

Without seeing models here so we know how you've defined your relations it's impossible to say what you're doing wrong.

@bendrucker check the update

For one you need to manually specify the table name on one side. Read up on how Bookshelf assembles those queries:
http://bookshelfjs.org/#Model-belongsToMany

I have a similar question I was just about to ask. I have a Listing model that belongsTo Unit. Unit belongsTo Property and Property belongsTo Neighborhood. I need to pull listings based on a neighborhood id, for example.
https://gist.github.com/robwilkerson/e912e2c920542307875e
You can pretty much ignore what's there right now. I started trying random things at some point and still haven't gotten it to work. Maybe someone could take a look at that and point us in the right direction. I think it's the same question.

If you've got more detail or more questions happy to continue the discussion here

Was the gist, in addition to the model associations, not enough info? If not, what else can I provide that might be useful?

Referring to rkmax here. You're welcome to open a new issue with a full reproduction of your schema, models, the call you're trying to make, and the final response you'd like to send to the client.

@bendrucker how i can do the following
adminAccounts: function(access) {
    return this.belongsToMany(Account).query({where: {access: access}});
},

That looks right. Are you sure the column names are correct? And can you share the queries that are run?

@bendrucker check the update, I had forgotten to add a method

Are you ever calling getAdminAccounts(access).fetchAll()?

@bendrucker no, i'm asking if that is the correct way to do that

Yes


I staked my tokens, but I can't withdraw them now, the webpage keeps prompting an error

I staked my tokens, but I can't withdraw them now, the webpage keeps prompting an error
this is address
https://gardens.1hive.org/#/garden/0x1e2d5fb385e2eae45bd42357e426507a63597397/

I'm seeing this same behavour. I created a proposal and then canceled it. It seems I can't get my 5,000 GIV back. The topmost balance shows >5000 GIV, but when I go into create a new proposal, the 5000 is not available. Here's a picture of what I'm seeing: https://forum.giveth.io/t/proposal-draft-monitoring-tool-to-help-identify-and-mitigate-recirculating-givbacks/405/13?u=tjayrush.
Here's the profile I'm having trouble with: https://gardens.1hive.org/#/xdai/garden/0xb25f0ee2d26461e2b5b3d3ddafe197a0da677b98/collateral/deposit
Here's the proposal: https://gardens.1hive.org/#/xdai/garden/0xb25f0ee2d26461e2b5b3d3ddafe197a0da677b98/proposal/17

I found that I had to first 'unwrap' the 5,000 tokens and then it allows me to proceed. I was able to proceed after unwrapping.

I staked my tokens, but I can't withdraw them now, the webpage keeps prompting an error this is address https://gardens.1hive.org/#/garden/0x1e2d5fb385e2eae45bd42357e426507a63597397/

is this still happening? ia am able to wrap and unwrap without any issue in this garden, if still happening can you share a video and check if there is some error being displayed on the console?

I was able to create the new proposal but only after I unwrapped and then re-wrapped the tokens. It was either that, or I just waited a day and then it was possible. Not sure. You may close this if you wish.

Since seems  that you were able to proceed going to close this one, feel free to open a new one if something else happens


Add tests

We need tests to avoid regressions.
Feel free to copy Nano Colors tests (they were written from scratch, I didn’t copy them from anywhere).

Now that v0.1.0 is out with proper testing against different Node.js version, I'm keeping this issue open to track effort related to covering NO_COLOR (and related flags/vars) support.

Final piece is here: https://github.com/alexeyraspopov/picocolors/commit/9002f13a76175d6cfa489c0f47bd4322e74af34b. I've adopted the test cases from nanocolors to test isColorSupported flag. The script takes less than a second so I just put it as a part of benchmarks job just because it already uses Node v16.


httpcache: implement a redis cache for http request

Make a generic http caching system using redis. This replace the hardcoded way used in tokens.

Module httpcache use a single redis connection for all calls
Fix redisclient to use resp2 module to send request, and not the old-poor-buggy string quote separated strings
Fix redisclient on macos to not silently exit when redis is not reachable
Update tokens to use httpcache module to cache request/responses

Tested on Linux and macOS, with and without redis running :)


Mark iNat obs as imported

Update the iNat observation with a note and/or Observation field that has the MO id and a link.
Probably copy what Pulk did in his Mirror app
Tasks

[x] #2222
[ ] Add MO url to iNat ObservationField Mushroom Observer URL
[ ] Add text to iNat notes like "Imported to Mushroom Observer Mar. 23, 2024."

Notes re adding Mushroom Observer Url to iNat:
This is ofvs field_id=>5005
https://api.inaturalist.org/v1/docs/#!/Observation_Field_Values/post_observation_field_values:Parameters
Parameter
body
  "observation_field_value": {
    "observation_id": 215968871,
    "observation_field_id": 5005,
    "value": "https://mushroomobserver.org/547126"
  }
}

Parameter content type:
application/json
{
  "observation_field_value": {
    "observation_id": 215968871,
    "observation_field_id": 5005,
    "value": "https://mushroomobserver.org/547126"
  }
}

Curl
curl -X POST --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{
  "observation_field_value": {
    "observation_id": 215968871,
    "observation_field_id": 5005,
    "value": "https://mushroomobserver.org/547126"
  }
}' 'https://api.inaturalist.org/v1/observation_field_values'

Request URL
https://api.inaturalist.org/v1/observation_field_values

Add to note by modifying (PUT) the :description. It should be the old notes plus the new line.
https://api.inaturalist.org/v1/docs/#!/Observations/put_observations_id:
Parameters
id: 215968871
body:
{
  "observation": {
    "description": "\n\nImported to Mushroom Observer 2024-09-09"
  }
}

Parameter content type: application/json
Curl
curl -X PUT --header 'Content-Type: application/json' --header 'Accept: application/json' -d '{
  "observation": {
    "description": "\n\nImported to Mushroom Observer 2024-09-09"
  }
}' 'https://api.inaturalist.org/v1/observations/215968871'

Request URL
https://api.inaturalist.org/v1/observations/215968871

Copilot says:

To add an Observation Field to an iNaturalist observation using the API v1, you can use the POST /observation_field_values endpoint. Here’s how you can do it:
Endpoint: POST /observation_field_values
Headers: Ensure you include the necessary authentication headers (OAuth token).
Payload: The payload should include the observation ID, observation field ID, and the value you want to add.
Here’s an example of how you can structure your request:

curl -X POST "https://api.inaturalist.org/v1/observation_field_values" \
-H "Authorization: Bearer YOUR_ACCESS_TOKEN" \
-H "Content-Type: application/json" \
-d '{
  "observation_field_value": {
    "observation_id": 215968871,
    "observation_field_id": 5005,
    "value": "https://mushroomobserver.org/547126"
  }
}'

CoPIlot says:

Here’s how you can add an Observation Field to an iNaturalist observation using the RestClient gem in a Rails application:

def self.add_observation_field(observation_id, field_id, value, access_token)
    payload = {
      observation_field_value: {
        observation_id: observation_id,
        observation_field_id: field_id,
        value: value
      }
    }

    headers = {
      Authorization: "Bearer #{access_token}",
      content_type: :json,
      accept: :json
    }

    response = RestClient.post(INATURALIST_API_URL, payload.to_json, headers)
    JSON.parse(response.body)
  rescue RestClient::ExceptionWithResponse => e
    e.response
  end


provide overview of all created worlds

and maybe make a curation of the best versions

postponed, this has to be part of a bigger next step


support sending proper batches to EventBridge

What is your idea?
Sending proper batches to EventBridge w.r.t. 256kb size limit.
Would you be willing to make the change?
Yes.
Additional context

oh that would be great. its on @embano1 's wishlist for sure. I believe this part of the code needs to measure how big the payload sizes are https://github.com/awslabs/eventbridge-kafka-connector/blob/main/src/main/java/software/amazon/event/kafkaconnector/EventBridgeWriter.java#LL123C1-L123C89. You can see an example way to calculate size at https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-putevent-size.html . This code can be modified to make sure all batches of events are at most 256KB .

Thx for bringing this up @agebhar1 ! That's definitely something we should add soonish...
Looping in @maschnetwork since he also had thought about this.
Let me dump a quick list of things I'd like to see us tackle here:

[ ] Replace Guava dep w/ own implementation (it's literally just doing batches, but a huge dep)
[ ] Use Rishi's link as guidance but let's review this article (based on a code proposal) as it's a bit dated - we might want to update the docs based on the implementation we settle on here
[ ] We definitely need to add several unit and integration tests for this change

We're happy to help you through this!

Thanks for adding this @agebhar1.
My initial thought was to create a custom EventBridgeBatchedItemCollector that implements the Collector Interface. We could then check for both BatchSize of 10 (Max EventBridge Batch Size) + Max Payload Size in the collector and produce proper batches. The result would be a List<List<PutEventsRequestEntry>>. Which we can leverage here. This would also allow us to remove the need for Guava's Lists.partition and we can remove the lib. Also it makes testing quite easy.

Thanks for the fast reply @baldawar, @embano1 and @maschnetwork also for the hints.
I came up with the same idea @maschnetwork to implement Javas Collector interface for a size based reduction to list of list. Unfortunately the mentioned PutEvents is generated later at https://github.com/awslabs/eventbridge-kafka-connector/blob/e4854e62d12cfea516cab6a681e68521d2b043d0/src/main/java/software/amazon/event/kafkaconnector/EventBridgeWriter.java#L183-L197 which makes it harder to calculate the size while collecting. But there are a couple of ways to solve it.
@embano1, I support dropping the Guava dependency since its the only function used from the library :+1:.
What makes me wonder is why PutEvents does not have a property/method to get the actual size and it's documented as non runable code. I can think of that would be nice to have the information on the object which is aligned with the EventBridge serialization.

I agree that this would be desirable instead of creating bespoke size estimations. Not sure how the SDKs are generated and how those helper methods can be implemented (if at all). cc/ @baldawar

I think we'll probably need to do something similar to this code https://github.com/aws/aws-sdk-java-v2/blob/ad1f232de73e328a0a5c83c8a4d924baec7026ec/utils/src/main/java/software/amazon/awssdk/utils/IoUtils.java#L43

Hi @baldawar, @embano1 and @maschnetwork! I walked through the code to get an idea for a good place and structure for calculating the batches.
I'd like to propose some structural changes (first) for discussion. The Kafka Connect JDBC Connector has a similar class structure but it's in the database context. A record is written to a database, hence it's called *Writer. In the EventBridge context the record is transformed to an event and send to the EventBridge. The name *Writer does not fit best IMHO. I can think more of an adapter with an interface like:
public Result put(Collection<SinkRecord> records)

where Result contains the failed and successfully send SinkRecords separated. Currently the wrapper/adapter EventBridgeWriterRecord is mutable with a dedicated state of the send result. You're using the Java Streams API and promote a function style, so immutability might fits better. If we handle the conversion of a Kafka SinkRecord to an EventBridge PutEventsRequestEntry inside the writer (adapter/sender) before batching it's then possible to do proper batch size calculation.
WDYT  :smiley:

Hi @agebhar1,
effectively we are "writing" records to EventBridge. Hence the naming "Writer" + given the fact we are in a Sink context. We could argue we are publishing events so we could also go with *Publisher, but at this point I don't think it is misleading that it needs an immediate action. However, I do agree with the interface contract. In the initial design the WriterRecord did not contain the full SinkRecord and there are multiple occasions where a record can get into error state - the mutability evolved from there but now in the context of the new batching logic I think it makes sense to refactor it. I agree with your proposals + adapted a few details. To summarize:

EventBridgeWriter will accept a Collection of SinkRecords rather than EventBridgeWriterRecords.

 public EventBridgeWriterResult putItems(List<SinkRecord> records)

The return type of this method should be a new class EventBridgeWriterResult containing a) List successfulRecords b) List errorRecords, which can be just the current EventBridgeWriterRecord renamed to EventBridgeWriterErrorRecord. Instead of having this class mutable the error code and status should be accepted in the constructor.

public EventBridgeWriterErrorRecord(SinkRecord sinkRecord, String errorCode, String errorMessage) {
    this.record = sinkRecord;
    this.errorCode = errorCode;
    this.errorMessage = errorMessage;
  }

Batching can then happen in the writer and loop from the task can be removed / moved to the Writer. This makes sense since the batching number of 10 is a detail of the EventBridge implementation and ideally must not be known by the task. We already agreed in the previous answers that a custom Collector implementation would be a good approach (Checking for max items of 10  + size)

var batches = records.stream()
       .map(this::createPutEventsEntry)
        //Todo: filter error records
       .collect(EventBridgeBatchedItemCollector.collect());

//Loop batches

After this step we can also remove Guava.
Thought: createPutEventsEntry can also cause the record to be in an error state - it might make sense to deal with the new result class already in the context of this transformation  to be able to filter the invalid records before sending to EB - currently this is done with an Optional.

I think it makes sense to split those changes over multiple PRs. I'm happy to support you on this. Wdyt?

Hi @maschnetwork,
*Publisher sounds great but I'm fine with the current naming too, if it doesn't mislead anyone. Fully agree with all the steps and class name EventBridgeWriterResult.
Should I create a new issue for the refactoring @embano1 or is a PR sufficient? I plan to do the refactoring tomorrow if it's okay for you @maschnetwork.

@agebhar1 should be separate issue


Builld docs in CI

Useful because it is the only way for the broken intradoc links lint to be checked.

Discovered that clippy already catches this.  Closing.


Servers search box missing

Issue Type: Bug
Search box on servers side bar is missing, i cannot quick search servers by name. Does anyone have this problem or know how to bring it back?
Azure Data Studio version: azuredatastudio 1.8.0 (96e214593190071f0b520439929b218f9fa673de, 2019-06-06T01:59:30.203Z)
OS version: Windows_NT x64 10.0.18362

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz (8 x 3398)

GPU Status
2d_canvas: enabledchecker_imaging: disabled_offflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onnative_gpu_memory_buffers: disabled_softwarerasterization: enabledsurface_synchronization: enabled_onvideo_decode: enabledwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
15.94GB (4.87GB free)

Process Argv

Screen Reader
no

VM
0%

Extensions (6)

Extension
Author (truncated)
Version

admin-pack
Mic
0.0.2

admin-tool-ext-win
Mic
0.0.1

agent
Mic
0.40.0

cms
Mic
0.2.0

import
Mic
0.8.0

profiler
Mic
0.8.0

Hi @acofalc, with the release of the new Data Explorer, we got rid of the search box. However, if you still want to use the search feature, you can use the Search: Servers command from the command palette.

Thank you for info. It was an useful and fast way to quickly get to server i want.


Fix unit tests after moving to github flow

What? and Why?
Reformats the unit tests that were broken following the move to github flow
Checklist

[x] CHANGELOG.md updated? (if required)

 Unit tests failing in sdx-transform-cs (2)

Codecov Report

Merging #95 into master will increase coverage by 0.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master      #95      +/-   ##
==========================================
+ Coverage   87.23%   87.25%   +0.01%
==========================================
  Files          17       17
  Lines        1340     1342       +2
==========================================
+ Hits         1169     1171       +2
  Misses        171      171

Impacted Files
Coverage Δ

tests/test_mwss_transform.py
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d031c1a...b0f3e98. Read the comment docs.


feat: Allow specifying (optional) priorityClass

Add optional value to specificy the priorityclass to use for the pods.
See https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/

https://github.com/postfinance/kubelet-csr-approver/releases/tag/v1.2.2
and here's a new release 🙃


Data table extension

Work for integrating data table extension:
File effected:
package.json
controller.js
thanks

Sorry it got  confusing and found it easier to start again ;) didnt realise PR would cancel.
i think someone needs to sort the gitignore file out.. it shows a lot of build output as change.
Thanks

i think someone needs to sort the gitignore file out.. it shows a lot of build output as change.

If you run rm -rf ./out from the project root, everything should be cleaned up. Otherwise, try a fresh clone.

could just ignore them lol

ok yea i understand

Is there a vscode api for checking whether an extension is installed? I'm wondering if we can avoid the strict requirement on this data table extension by changing our presentation behavior based on whether the user has it installed already or not...

I was thinking the same.. but i don't know. hold fire then and ill take a look if you like.
I can see that.. not an issue for me really if it means we get all the other features?
I haven't really played around with it that much, I'm pushing to get it in a bit fast because I'm fed of getting what i need by running it locally lol but lets do it properly

I think the VSCode notebook API allows multiple formats for a single output - if that works how I think it does, could vscode-sql-notebook just provide both 'text/markdown' for its own, and 'application/json' (or whatever) for DataTable? That way users can switch between the two as needed, and it wouldn't be such a big deal if DataTable is occasionally worse.

Release 0.7.0 includes support for JSON cell output. It now should be possible to leverage any extension that registers a JSON media type cell renderer. Full release notes here: https://github.com/cmoog/vscode-sql-notebook/releases/tag/v0.7.0


Sorry for the issue

Hello! Could you please tell what do you use for screenshot creation?
THANK YOU!

We made screenshots with Cloud App, then we put it into the mockup, an example of mockups can be seen on our website.

Thank you very much!


Can't get it to work with aviutl

Hey Maverick,
I've tried to add this keeping the base structure to the aviutl path, however once i have started up aviutl, it won't show up on the list or dropdown menus.
Any ideas?

Oh c'mon man, i would answer your questions if i worked on parts of it, least you could do!
I actually managed to get a GPU version and CPU version working, but this one won't work. I would love to try it out.

Mind you attitude man, that's not the way you call for help.
The released file here still works on my machine

(Win7 64bit, Nvidia GTX970, Intel 5820k)
Show me the following or I'd have no idea what's wrong with your machine:

AviUtl base folder structure (print screen/dir/tree... whatever)
OS
Graphics chipset
a screen capture of your Filter Menu

Curious, should i keep folder structure as in zip and just extract to aviutl folder, you have an awesome program btw, i prefer it over many others like xvid4psp etc, my only gripe is slow integration of things like mkv container and mp4 etc.
Ok
1.Will attach here soon(no access right now)
2. Windows 7 (64) SP1 etc.
3. http://www.gigabyte.com.au/products/product-page.aspx?pid=4345#ov (amd 7750 2gb) and 6950 http://www.newegg.com/Product/Product.aspx?Item=N82E16814102945 2gb
4. Will attach here soon(no access right now)

Just retain the archive's folder structure. That means you are going to add 2 new folders in /aviutl/,
and a bunch of DLL and .auf side-by-side with aviutl.exe
The Sapphire should have no problem running the code (if connecting to monitor).
CPU.... you are killing it
xvid4psp... it is a transcoder... while AviUtl is primarily an Editor, so use the right tool.
The faster waifu2x implementation right now all use CUDA (and Intel CPU), and this OCL wrapper is no match with CUDA.
I'd suggest you to try this also:
[https://github.com/YSRKEN/aviutl-waifu2x-w2xc]
which can utilize CUDA, OCL, and various SIMD

If there are no more response by weekend, I'm closing this issue.


Where to place .p12 for push notification?

Refer to readme if I want to test push notification I've to add path to .p12 here.
DEV_PFX: $DEV_PFX

So I suppose to upload  .p12 somewhere in docker container am I right?
The question is which path? and how? via git? scp? to where? parse-server?
And when it go production, how to secure it?
Thanks

Place .p12 into ./volumes/production-pfx and/or ./volumes/dev-pfx.
DEV_PFX is not necessary.
ref. f13a3ab

Cool, BTW it throw error when I'm not config it yet at the moment.
fs.js:976
  return binding.lstat(pathModule._makeLong(path));
                 ^
2016-04-30T15:54:59.937599902Z
Error: ENOENT: no such file or directory, lstat '/production-pfx'
    at Error (native)
    at Object.fs.lstatSync (fs.js:976:18)
    at Object.<anonymous> (/parse/index.js:35:9)
    at Module._compile (module.js:541:32)
    at Object.Module._extensions..js (module.js:550:10)
    at Module.load (module.js:456:32)
    at tryModuleLoad (module.js:415:12)
    at Function.Module._load (module.js:407:3)
    at Function.Module.runMain (module.js:575:10)
    at startup (node.js:159:18)
[nodemon] app crashed - waiting for file changes before starting...

It didn't required before so it shouldn't throw error?
Thanks

Do you fetch latest commit of this project? I updated docker-compose.yml before, I think you need it.

@yongjhih Gotcha! working now :)

I try to send push from dash board but not success yet. Did I place it right?

It would be nice if we've some console.log to tell that certificate is there and valid.
Apart from certificate, Is there another config required to use push notifications?
How can I debug where and what made push notfication failed?
Thanks

volumes/dev-pfx is a file as .p12, so rename your Cerificates.p12 to dev-pfx

like this?

Yes

Hm weird. I got
devPfx : null
devPushConfig : undefined
pushConfig : undefined
verifyUserEmails : false
emailModule :
emailAdapter : undefined

But when I changed code to
var devPfx = '/dev-pfx';
// if (!fs.lstatSync(devPfx).isFile()) devPfx = null;

I got
devPfx : /dev-pfx
devPushConfig : [object Object]
pushConfig : [object Object]
verifyUserEmails : false
emailModule :
emailAdapter : undefined

Something wrong with if (!fs.lstatSync(devPfx).isFile()) it's always true and made devPfx null all the time.

Oh, sorry. I replied wrong answer before. Remove your dev-pfx directory, and rename your Certificate.p12 to dev-pfx as file.

I think
- "./volumes/dev-pfx:/dev-pfx"

in docker compose made it to be Docker folder.
because when I check....
console.log('fs.lstatSync(devPfx).isFile() : ' + fs.lstatSync(devPfx).isFile());
console.log('fs.lstatSync(devPfx).isDirectory() : ' + fs.lstatSync(devPfx).isDirectory());

I got
fs.lstatSync(devPfx).isFile() : false
fs.lstatSync(devPfx).isDirectory() : true

So current code will return false and always devPfx : null there
Hre's what it look like via Kitematic

And I don't think we can use it as file there?

This is working btw
var devPfx = process.env.DEV_PFX || '/dev-pfx/dev-pfx.p12';

And place dev-pfx.p12 inside dev-pfx folder also remap /dev-pfx with ral folder.
And I think you need to fix this too.
dev-pfx-cert.pem.isFile : false
dev-pfx-key.pem.isFile : false

Should it be easier if we use 1 folder and have every cers in there? e.g.
cers/dev-pfx.p12
cers/dev-pfx-cert.pem
cers/dev-pfx-key.pem

File name is unique anyway.

Anyway, now we expect that are files:
- /dev-pfx (file)
- /dev-pfx-cert.pem (file)
- /dev-pfx-key.pem (file)

If you did not place files into ./volumes/, docker will create empty directories as defaults:
- /dev-pfx/ (dir)
- /dev-pfx-cert.pem/ (dir)
- /dev-pfx-key.pem/ (dir)

So we must to check that be file in js.

if you did not place these files into ./volumes/ before, docker will create dummy directories as defaults:

That gotcha moment! This should add to README or could be better if just let Docker mount cers folder and let everyone put cers related in after that so no one stuck at this like me tho. ;)
Thanks anyway, push is fire and working perfectly.

Maybe good. I did not expect it's hard to explain that

volumes/dev-pfx is a file as .p12, so rename your Cerificates.p12 to dev-pfx as file.

https://github.com/yongjhih/docker-parse-server/commit/f45040f247493fe3f2ed0345ce6fc80b433ed33b

Sure! I'll do PR when I get it right. I'm also blog my step here.
BTW, What it will look like for multiple app? (I'm almost forget about multiple certs)

Oh, Here's another issue I found when I use dev-pfx
var devPfx = process.env.DEV_PFX || '/certs/dev-pfx';
console.log('fs.lstatSync(devPfx).isFile() : ' + fs.lstatSync(devPfx).isFile());
console.log('fs.lstatSync(devPfx).isDirectory() : ' + fs.lstatSync(devPfx).isDirectory());
```
I'll get
```
2016-05-07T13:16:41.004512973Z
Error: ENOENT: no such file or directory, lstat '/certs/dev-pfx'
    at Error (native)
    at Object.fs.lstatSync (fs.js:976:18)
    at Object.<anonymous> (/parse/index.js:62:53)
    at Module._compile (module.js:541:32)
    at Object.Module._extensions..js (module.js:550:10)
    at Module.load (module.js:456:32)
    at tryModuleLoad (module.js:415:12)
    at Function.Module._load (module.js:407:3)
    at Function.Module.runMain (module.js:575:10)
    at startup (node.js:159:18)
[nodemon] app crashed - waiting for file changes before starting...
```
But if I use `dev-pfx.p12` extension
```
var devPfx = process.env.DEV_PFX || '/certs/dev-pfx.p12';
console.log('fs.lstatSync(devPfx).isFile() : ' + fs.lstatSync(devPfx).isFile());
console.log('fs.lstatSync(devPfx).isDirectory() : ' + fs.lstatSync(devPfx).isDirectory());
```
I got
```
fs.lstatSync(devPfx).isFile() : true
fs.lstatSync(devPfx).isDirectory() : false
```
Which is weird, this maybe only happen with my machine Mac OSX EL Capitan 10.11.4 fresh install.

It's fixed, you can update image bt the following command, docker pull yongjhih/parse-server

Now I know what going on here, Mac is Hide extension!
When you rename from foo.p12 to foo via Finder
it will appear in finder as foo but will still foo.p12!

To proper rename to .p12 is use terminal instead.
$ mv Certificates.p12 dev-pfx

Cheers


New event hook for package uninstall

I think there should be a new event hook that runs when a package is uninstalled that can throw an error and cause the package to not uninstall and display an error message.
A simple use case of where this might be useful: Say that you want to have a package dependency for someone else's package. Currently, on install you can check if that person's package is installed and if not throw an exception. However, there's no way for you to prevent someone from uninstalling that package after your package has been installed, thus breaking the dependency and possibly your own package.

(If it doesn't already exist - I have not looked in 5.7), there could be a similar event on upgrade.
For example,  so an upgrade can be blocked if a dependency needs to be upgraded first.

:+1:
This can currently lead to weird dependency issues and even break the site totally in case the dependant add-on assumes something is provided by the other add-on.


Add hosts blacklist to DB_Copy App

Prevent running commands on hosts that aren't accessible from our servers using a configurable blacklist
Refactor the use of global variables in db_app

Note: Requires https://github.com/Ensembl/ensembl-prodinf-core/pull/118

What's the associated bug here?

In general, preventing to run commands on hosts that are not accessible speeds up the response. In this case also, we have scripts that use the /hosts endpoint to monitor servers every second. That's flooding the logs with errors which makes hard to track other things and it's possibly the cause for some of the timeouts we experienced.

What's the associated bug here?

In general, preventing to run commands on hosts that are not accessible speeds up the response. In this case also, we have scripts that use the /hosts endpoint to monitor servers every second. That's flooding the logs with errors which makes hard to track other things and it's possibly the cause for some of the timeouts we experienced.

Ok, thank you for the explanation @radome !

@radome: Does that mean that we need to find and list all the new hosts and put them in the config file? There are quite a lot of new servers

@radome: Does that mean that we need to find and list all the new hosts and put them in the config file? There are quite a lot of new servers

Theoretically, yes. Practically, so far only two servers we don't have access to are being repeatedly queried by a script, so I'd say we start with those two and we keep adding others as soon as they start flooding the logs.

@radome: Does that mean that we need to find and list all the new hosts and put them in the config file? There are quite a lot of new servers

Theoretically, yes. Practically, so far only two servers we don't have access to are being repeatedly queried by a script, so I'd say we start with those two and we keep adding others only when they start flooding the logs.

Sure that make sense 👍


AttributeError: module 'ctranslate2' has no attribute 'StorageView'

>>> from faster_whisper import WhisperModel
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/tmp/faster-whisper/lib64/python3.9/site-packages/faster_whisper/__init__.py", line 2, in <module>
    from faster_whisper.transcribe import WhisperModel
  File "/tmp/faster-whisper/lib64/python3.9/site-packages/faster_whisper/transcribe.py", line 75, in <module>
    class WhisperModel:
  File "/tmp/faster-whisper/lib64/python3.9/site-packages/faster_whisper/transcribe.py", line 357, in WhisperModel
    encoder_output: Optional[ctranslate2.StorageView] = None,
AttributeError: module 'ctranslate2' has no attribute 'StorageView'

I built CTranslate2 from master and installed faster-whisper with pip (in a venv).

This is not directly related to faster-whisper.
Can you open an issue in https://github.com/OpenNMT/CTranslate2 and post all the steps you followed to compile the package?

@guillaumekln thanks


Login is Broken

Looks like Niantic shut down their old 0.45 API, which is the one we're using for this app. In order to get it working, we'll need to use the newer 0.51 API, but that's a little bit of money. Here's some info: https://github.com/cyraxx/pogobuf/wiki/Using-a-hashing-server.

Well, I guess the only thing we can do here is configure the PogoBuf client to use a hashing server and let users provide their own key through a private (gitignored) file. That shouldn't be too hard, but it does mean that this won't be free any more.
I guess we had a good run.

I've played a bit with this tonight and I can't get it to work so far. I keep getting an error from pogobug.Client.init() with the message Unexpected end of JSON input. Here's how I've modified the client creation:
const client = new pogobuf.Client({
	version: privateProps.apiVersion,
	useHashingServer: true,
	hashingServer: privateProps.hashingServer,
	hashingKey: privateProps.hashingKey
});

with the apiVersion property set to 5100 and the hashingServerver property set to http://hashing.pogodev.io/api/v121_2/hash. My hashing key should be valid, as I just purchased it (~$5). I also tried providing authType and authToken properties in the config, but that's failing in the same manner - and it looks like client.setAuthInfo() sets that as expected anyway.
@DevinFalgoust - have you had any luck?

I haven't had a chance to take a look yet, but that's the same way i'd have tried to handle it. I'll see if i can do some digging as well.

Maybe this library is a good solution rather than going strait through pogobuf? https://github.com/starkevin/node-pogo-signature

Also, where did you find the hashingServer url you're using?

I think this may be a pogobuf issue? https://github.com/cyraxx/pogobuf/issues/123

It looks like pogobuf already uses that library. Also, I think you're right that this seems to be a pogobuf issue. It looks like there's already a PR open to address adding a signature cyrax/pogobuf#125
Also, each API version has its own hashing endpoint. See BuddyAuth's documentation for details.
We can probably lock in version 0.51.0 and only have users provide their key, assuming that pogobuf gets updated so that it works again.

Sounds like a good plan to me. I was trying with http://pokehash.buddyauth.com/api/v127_2/hash, and got the same result as you did (as expected). Guess we'll have to wait on pogobuf to get fixed or find a new api library.

as said on pogobuf repo, please test 'pogosandbox/pogobuf' in package.json and tell me if that correct your issue

Hey @nicoschmitt, thanks for dropping in. I tried using pogosandbox/pogobuf and I'm up against the same problem with the following config:
const client = new pogobuf.Client({
	version: 5100,
	useHashingServer: true,
	hashingServer: "http://hashing.pogodev.io/api/v121_2/hash",
	hashingKey: privateProps.hashingKey
});

I'm setting auth info later, once I've retrieved a token from Google or PTC.
I did notice that if I omit the hashingServer property, I get the following error: err="Unsupported version for hashserver: 5100/1.21.0"

5100 is now blocked by Niantic. Use 5702

And don't use hashserver option

That worked! Thanks for the support!

FYI 0.45 without hash is working. You don't absolutely have to use hashserver

Oh, sweet! I didn't know that

It work with my master at least. Not 100% sure about the #0.57.2 branch

Just tested and it's working without hash. Cool, I can provide hashing as option. Thanks again!

@Eric-Carlton I can't get it to work on mine. I added the private.json on mine with my hash key, and i'm getting this error: client.init() failed, rejecting from getClient. Any ideas?

Did you pull and then run npm update?

lol nope. classic. That fixed it, thanks

Haha, np


replace go-jose.v2 with newer version because it is vulnerable to CVE-2024-28180

Based on the advisory in this link, the go-jose.v2 is vulnerable:
https://github.com/go-jose/go-jose/security/advisories/GHSA-c5q2-7r4c-mv6g
I tried to fix in my local clone of krakend-jose, but it seems that the underlying dependency of go-auth0 also needs to be updated.
I might be able to send MRs for both of the repositories affected, but in this issue I just want to let you folks know that this vuln might be blocking some organizations that uses trivy in their CI/CD pipeline.

Thanks @gunturaf. Generally speaking, trivy is an excellent tool but it also gives a lot of false positives, so blocking your pipelines because of it IMHO is a drastic measure and not an advisable strategy.
That being said, KrakenD does not support JWE, and while this vulnerability is on the JOSE package, it does not affect KrakenD. Feel free to open MR for review, but I must warn you that the go-auth0 will be "challenging".


plot2svg, both in DynaSim and elsewhere

In the last days I dedicated quite some work to improve my forked version of plot2svg, not only but overall for matlab versions > 2014a (supporting bar graphs, legends, axes label location, tick label rotation, etc).
Please, those who use plot2svg, give it a try and report any issue here: https://github.com/kupiqu/plot2svg (thanks in advance!)
There are still few things to improve (with higher priority for adding support to scatter plots and colorbars).
Once I'm happy enough with all these enhancements (probably once colorbars and scatter plots just work), I will rename my fork to fig2svg. The reason is to make it clearer to the final user that such version will be very different from the original plot2svg (but certainly I will keep the license and will give credit to its original author as it is due, of course).

Forgot to say, the version of plot2svg in dev branch is in sync with the repo I mentioned above: https://github.com/kupiqu/plot2svg

Yes interested. How does one do it?

First, set the properties that you like for the figure and put them in "normalized" resolution units (taking 96 dpi as reference):

prevFontName = get(0,'defaultAxesFontName');
set(0,'defaultAxesFontName',fontName); % e.g. 'Arial';
resolutionScaling = 96/get(0,'ScreenPixelsPerInch');
fontSize = fontSize*resolutionScaling; % e.g. 16*resolutionScaling;
lineWidth = lineWidth*resolutionScaling; % e.g. 2*resolutionScaling;
markerSize = markerSize*resolutionScaling; % e.g. 10*resolutionScaling;

The part of the font name is not related to the resolution but it's a nice trick to use a specific font without change Matlab's default or the startup.m config (One could do similarly for the size of the figure, not shown).

Script with plotting

here
that uses fontSize, lineWidth, and/or markerSize variables and calls the forked plot2svg function mentioned above to save figures in svg format.

Finally, restore the default font name

set(0,'defaultAxesFontName',prevFontName);

First, set the properties that you like for the figure and put them in "normalized" resolution units (taking 96 dpi as reference):

where does this code go? does this work for batch sims with dsAnalyze or just for figures actively loaded in the workspace?

This is not in DynaSim at the moment, just what I use in my own analysis scripts. We could think whether we want to add it to DynaSim and how...


Migrate filebot rename and subtitles processes and workflow

Migrate from download-utils to the node-way of doing this in this project.
Apart from the regular workflow, take into account:

Take into account multiple files in one torrent
Verify subtitles are correct and with two and three name language code
Check Unsorted folder for renames
more?

Migrate to current project.

Done as part of Core SQS project.

Will close only when deployed.

This has been broken down into other issues.


Ask Nicole Raimundo to judge

Sent ask letter. She is on vacation and will likely require a followup.

Out of town on Sept 30, happy to do either of the other days. Also thinking about speakers for us!

@ChrisTheDBA  is going to ask about judging at DataPalooza finale.

Hi, Chris. Any news on Nicole judging at the Finale?

Nice work @ChrisTheDBA


Add SDK/vx.y.z tag for Go package use

For use with Go, an extra tag prefix of SDK is expected by Pulumi for the Go SDK to work.
Example here: https://github.com/pulumi/pulumi-opsgenie/tags

tag for v2.0.0 created
added automated tagging into release pipeline (https://github.com/muhlba91/pulumi-proxmoxve/blob/main/.github/workflows/release.yml#L233)

please verify and close the issue.


venv not used in initial terminal

Issue Type: Bug
VSCode automatically activates your selected Python venv in the integrated terminal, however this is currently only done for terminals opened after a Python file has been viewed. Any terminals opened before a Python file is viewed does not get the venv, including terminals opened when VSCode is started.
Extension version: 2019.9.34911
VS Code version: Code 1.38.1 (b37e54c98e1a74ba89e03073e5a3761284e3ffb0, 2019-09-11T13:30:08.229Z)
OS version: Linux x64 4.12.14-lp151.28.13-default

System Info

Item
Value

CPUs
AMD Ryzen 7 1700X Eight-Core Processor (16 x 2200)

GPU Status
2d_canvas: enabledflash_3d: enabledflash_stage3d: enabledflash_stage3d_baseline: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onnative_gpu_memory_buffers: disabled_softwareoop_rasterization: disabled_offprotected_video_decode: unavailable_offrasterization: disabled_softwareskia_deferred_display_list: disabled_offskia_renderer: disabled_offsurface_synchronization: enabled_onvideo_decode: unavailable_offviz_display_compositor: disabled_offwebgl: enabledwebgl2: enabled

Load (avg)
1, 1, 1

Memory (System)
62.84GB (46.34GB free)

Process Argv
--unity-launch

Screen Reader
no

VM
0%

Dupe of #5330


Performance for Llama-2-7b in fp8 is lower than benchmark

System Info

System Info

CPU architecture: x86_64
GPU: NVIDIA 4090 24GB
TensorRT-LLM: 0.10.0.dev2024042300
Triton Inference Server: r24.02
OS: Ubuntu 22.04

Who can help?
Quantization: @Tracin
Performance: @kaiyux
Information

[X] The official example scripts
[ ] My own modified scripts

Tasks

[X] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
[ ] My own task or dataset (give details below)

Reproduction
I hope to use w8a8 to speed up prefill infer time of LLM. I obtained satisfactory results in the benchmark phase, but the performance was decrease after using engine.
test case: input_len=1024, batch=1,2,4,8, output_len=1
Reproduce steps:
1. run benchmark:
python benchmark.py -m llama_7b --mode plugin --quantization fp8 --max_input_len 1024 --max_batch_size 8  --batch_size "1;2;4;8"  --input_output_len "1024,1"

2. run benchmark with engine:
python quantize.py --model_dir ${WORK_HF_DIR} \
                                   --dtype float16 \
                                   --qformat fp8 \
                                   --output_dir ${WORK_CKPT_DIR} \
                                   --calib_size 512

trtllm-build --checkpoint_dir ${WORK_CKPT_DIR} --output_dir ${WORK_ENGINE_DIR}   --gemm_plugin float16 --gpt_attention_plugin float16  --max_input_len=1024 --max_batch_size=8

python benchmark.py -m llama_7b --engine_dir ${WORK_ENGINE_DIR} --mode plugin  --max_input_len 1024 --max_batch_size 8  --batch_size "1;2;4;8"  --input_output_len "1024,1"

Expected behavior
Can I achieve benchmark-level performance and whether any parameters need to be adjusted.
Confirm which performance result is correct and I want to know how much improvement w8a8 can achieve compared to fp16 in this scenario.
actual behavior

There is not much difference in performance whether fp16 uses engine or not. But fp8 uses engine delay increased by 1.6 times
additional notes
Hope to use w8a8 to improve the performance of prefill stage. Looking forward to giving some suggestions

Might you try adding --kv_cache_dtype fp8 when you convert the checkpoint for the with engine case?

I tried adding parameters--kv_cache_dtype fp8, but the performance didn't seem to improve.

config.json

Hi @SidaZh ,
We can disable paged kv cache when building engine, trtllm can chose faster fp8 gemm kernels with better opt_shape and get similar performance as benchmark.py.

Hi @SidaZh , we can disable paged kv cache when building engine, trtllm can chose faster fp8 gemm kernels with better opt_shape and get similar performance as benchmark.py.

It works. Thanks for your suggestion.


Proxy Connection

I'm working on a proxy protected network, and that's drove me crazy. Is there an easy way to set proxy settings? If not, I'll do it! :)

I could finally login using this hotfix
In Laravel\Socialite\Two\AbstactProviter : 329 -> protected function getHttpClient()
I passed in the "new Client()", an array with the proxy config "new Client(['proxy' => '...'])", because the library it's not getting my HTTP_PROXY and HTTPS_PROXY from linux config as it should be doing.

Solved in #65


fix: dev server does not release ports on linux after kill

Description
After starting a dev server, killing the underlying dart process doesn't kill the server process.  As indicated by this e2e workflow run:
https://github.com/VeryGoodOpenSource/dart_frog/actions/runs/5670059745/job/15364840497

Steps To Reproduce

run dart_frog dev on a Linux machine
kill the command line with ctrl+c
There is still a local server running on :8080

Expected Behavior
The local server should be killed alongside the CLI.
Additional Context
This also happens if the dev server is started via the daemon.

Was no longer able to reproduce and the E2E seems to work correctly as well, closing this.
If you see this problem happening again please open up a new issue with a reproduce-able example <3


Fix Multiplexer POST Body Handling

The previous Multiplexer Plugin test only verified that the plugin would
load without error. This updates the test to verify it can multiplex to
multiple hosts, exercising both HTTP and HTTPS connections to the
multiplexed hosts.
While doing this, I added POST request method tests. This showed that
the plugin could not handle POST bodies and simply failed a
misconfigured assertion when the request body was processed. This change
also fixes that bug in the plugin.

Cherry-picked to v9.2.x


[WIP] Add Provisioning Reason to ForgetError

Waiting register, Waiting node provisioning is expected error when
cluster provisioning is proceeding. so these error should not be
reflected to condition as Error.
This is depending on following PR

https://github.com/rancher/norman/pull/203

This PR is depending on PR for other repository(https://github.com/rancher/norman/pull/203).
So until dependent PR got merged, we can not finish this PR.
This PR will solve https://github.com/rancher/rancher/issues/15907


Table comments truncated when exporting to SQL(mysql)

I notice some of the table comments are being truncated when exporting to mysql:
COMMENT 'design team roles (principal, project manager, civil enginee';
When the xml looks fine:
<comment>design team roles (principal, project manager, civil engineer, landscape engineer, structural engineer, mechanical engineer, plumbing engineer, electrical engineer, fire protection engineer, lab consultant, geotechnical engineer) and company roles (owner, developer, tenant, general contractor)</comment>

Right, this is probably intentional, as there is an explicit substring extraction in the relevant template: https://github.com/ondras/wwwsqldesigner/blob/master/db/mysql/output.xsl#L92
The problem is that I truly have no idea why do we need to take only the first 60 chars... :smile:

Couldn't we take out that substring statement? Want me to submit a PR?

I would gladly take it out myself. I just want to know why is the substring present right now...


v0.21.1 crash looping on docker

No logs to report, container not able to start.

Both indexer-service and indexer-agent?

Both indexer-service and indexer-agent?

I only had issues with service, but some reports of both.

Ok, that's what I'm seeing too. I'm able to reproduce on indexer-service; I'm getting an odd segfault error that doesn't produce any logs.
I'm bisecting back through git commit history to identify the issue now.

Hello! Have same issue with indexer-service, network arbitrum-sepolia

Same issue with indexer-service:

Docker container exited without logs
segfault when running from npm packages<EMAIL_ADDRESS>segfault when running from source (branch main)

same issue here with indexer-service.


refactor(frequency): ♻ extract reference as 440 Hz

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
1119

Relevant Lines:
1119

💛  - Coveralls


Crossplane - Ensure required packages installed and available to the consumers

[ ] Ensure that all packages can be installed without any conflicts
[ ] Ensure each resource apply RBAC and is working with capability roles

will be solved as part of the onboarding process


Add items for cute_files library

This is from an older verison, so might not be 100% up to date:
CompareFileTime
WIN32_FILE_ATTRIBUTE_DATA
GET_FILEEX_INFO_LEVELS
GetFileAttributesExA
MAXDWORD

If you want me to make PRs for these I can because they're pretty small and simple, though I don't know what the desired process for reverse engineering is.


Coded fsnotify constraint into build script

Addresses this.
It's a bit hacky, but should work (and be future proof).

We can clean it up when the 0.16 is more in the pipeline, thanks!


[Question] Is the library thread safe?

Hello,
Can I use Lookup method of the singleton SymSpell class object instance across multiple threads?

You may want to wait for Wolf for a definitive answer, since it's been a while since I worked on this project, but here are my thoughts. The place where there is a danger for multiple threads accessing a single instance is in the dictionary construction. Only one thread should construct the dictionary. Once the dictionary has been built, and is no longer modified, I'm pretty sure there's no internal state that's changed within the Lookup method that would cause conflicts when multiple threads utilize that method.

softwx is right. If single SymSpell instance is created, then Lookups can be consumed by in parallel by multiple threads (e.g. parallel queries from different users to a web search engine or a data base).
Lookup is thread safe, as long as there are no LoadDictionary, CreateDictionary, and CreateDictionaryEntry in parallel.
But if you want to incrementally update your dictionary with CreateDictionaryEntry, then both Lookup and CreateDictionaryEntry have to be synchronized by locks.
While you could use a simple Lock(), a ReaderWriterLockSlim ensures better performance, because it still allows multiple Lookups in parallel:
ReaderWriterLockSlim threadLock = new ReaderWriterLockSlim();

threadLock .EnterReadLock();
try
{
      symspell.Lookup(term,verbosity);
}
finally
{
      threadLock .ExitReadLock();
}

cacheLock.EnterWriteLock();
try
{
    symspell.CreateDictionaryEntry(term,frequency);
}
finally
{
    cacheLock.ExitWriteLock();
}


"all.sh" bash script kills terminal at conclusion (Ubuntu 14.04.3)

The Ubuntu instruction:
Install all required software
(cd openwhisk/tools/ubuntu-setup && source all.sh)
When executed (and all.sh is invoked) results in a lengthy series of download/install/config/setup.  Upon completion, the terminal exits so there is now way to verify success/failure.

This is fixed now https://github.com/openwhisk/openwhisk/commit/2416a3bcb486e08a8df42816f8f28f6d9f3b6f4c#diff-e126e24da7c91b97670918c1c6440cc0