Refactor DataTable implementations to pinot-common

Instructions:

The PR has to be tagged with at least one of the following labels (*):

feature
bugfix
performance
ui
backward-incompat
release-notes (**)

Remove these instructions before publishing the PR.

(*) Other labels to consider:

testing
dependencies
docker
kubernetes
observability
security
code-style
extension-point
refactor
cleanup

(**) Use release-notes label for scenarios like:

New configuration options
Deprecation of configurations
Signature changes to public methods/interfaces
New plugins added or old plugins removed

Codecov Report

Merging #9463 (36e10de) into master (e2d8aaa) will decrease coverage by 41.55%.
The diff coverage is 43.55%.

:exclamation: Current head 36e10de differs from pull request most recent head 9446a96. Consider uploading reports for the commit 9446a96 to get more accurate results

@@              Coverage Diff              @@
##             master    #9463       +/-   ##
=============================================
- Coverage     69.83%   28.28%   -41.56%
+ Complexity     5127       53     -5074
=============================================
  Files          1909     1898       -11
  Lines        101749   101401      -348
  Branches      15440    15402       -38
=============================================
- Hits          71059    28681    -42378
- Misses        25674    69926    +44252
+ Partials       5016     2794     -2222

Flag
Coverage Δ

integration1
25.98% <36.80%> (+0.02%)
:arrow_up:

integration2
24.87% <41.71%> (+0.16%)
:arrow_up:

unittests1
?

unittests2
?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...org/apache/pinot/common/data/ObjectSerDeUtils.java
46.03% <ø> (ø)

...apache/pinot/common/data/customobject/AvgPair.java
58.06% <ø> (ø)

...inot/common/data/customobject/CovarianceTuple.java
0.00% <ø> (ø)

...not/common/data/customobject/CustomSerDeUtils.java
37.09% <ø> (ø)

...pinot/common/data/customobject/DoubleLongPair.java
0.00% <ø> (ø)

.../pinot/common/data/customobject/FloatLongPair.java
0.00% <ø> (ø)

...he/pinot/common/data/customobject/IntLongPair.java
0.00% <ø> (ø)

...e/pinot/common/data/customobject/LongLongPair.java
0.00% <ø> (ø)

...inot/common/data/customobject/MinMaxRangePair.java
75.86% <ø> (ø)

...pinot/common/data/customobject/QuantileDigest.java
0.00% <ø> (ø)

... and 1499 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Deprecate Factual Consistency

Why this metric was deprecated?

@rohinish404 Metric wasn't very accurate, and we rolled out a HallucinationMetric instead

@rohinish404 Metric wasn't very accurate, and we rolled out a HallucinationMetric instead

Is that doing the same thing the factual consistency one did?

Yup! What are you trying to evaluate? (RAG or fine-tuned LLMs?)

Yup! What are you trying to evaluate? (RAG or fine-tuned LLMs?)

Hmm...im not sure..just trying to write a script which calculates factual consistency of outputs generated using openai api given input and context.

New to evaluating llms ..so if u can share some useful resources..it'll be helpful

@rohinish404 Here is a set of blog posts that gets you started: https://www.confident-ai.com/blog

@rohinish404 Here is a set of blog posts that gets you started: https://www.confident-ai.com/blog

Thanks for sharing!! :)

@penguine-ip thanks for the blogs. They are really awesome. Looking forward to more such awesome blogs in future :)
one ques i have is - why nli models are used for calculating factual consistency or hallucination?

@rohinish404 Thanks! Please join our discord for more discussions on evaluation :) https://discord.com/invite/a3K9c8GRGt
As for NLI models, they offer some insight into contradictions which is nice, although we're shifting towards using LLMs instead. Also releasing another post on that soon

@rohinish404 Thanks! Please join our discord for more discussions on evaluation :) https://discord.com/invite/a3K9c8GRGt
As for NLI models, they offer some insight into contradictions which is nice, although we're shifting towards using LLMs instead. Also releasing another post on that soon

Hey, thanks for the invite. Just joined it! I'll ask my ques there! :)


Heads-up: Copy edits ready for review

Docs Build status updates of commit 8093f79:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

mixed-reality-docs/mrlearning-base-ch6.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Feedback for “Support”

Hello dear friends
I am installing your wonderful solution AnythingLLM on my intel 7 laptop with 16 RAM formatted and I have installed:
New linux mint virginia,
python 3.12.4,
node 22.6.0,
I can see their versions in the terminal
I have followed the steps for installing docker engine and successfully executed the command sudo docker run hello-world.
The docker pull mintplexlabs/anythingllm has been completed
and when I send the following command package I get the last line which looks like an API KEY:
root@nelson-NL40-50ZU:/home/nelson# export STORAGE_LOCATION=$HOME/anythingllm &&
mkdir -p $STORAGE_LOCATION &&
touch "$STORAGE_LOCATION/.env" &&
docker run -d -p 3001:3001
--cap-add SYS_ADMIN
-v ${STORAGE_LOCATION}:/app/server/storage
-v ${STORAGE_LOCATION}/.env:/app/server/.env
-e STORAGE_DIR="/app/server/storage" \
mintplexlabs/anythingllm eda648b2e33c4954625ca38fa74648c0a0b20c7a5b23cee0c2fc9c9c1b64feb5 In the firefox browser I have NO PROXY selected in Network Settings. I try to open it at http://localhost:3001 but it says Unable to connect • Firefox can't establish a connection to the server at localhost:3001.
• The site could be temporarily unavailable or too busy. Try again in a few moments.
• If you are unable to load any pages, check your computer’s network connection.
• If your computer or network is protected by a firewall or proxy, make sure that Firefox is permitted to access the web.
What is my mistake?

Note: On another laptop on another occasion I have successfully installed anythingllm on the desktop with Linux Mint 22

This is the repo for the documentation, not the app.
You can also view the docker logs to get more specific information on what error might be preventing the application from loading.


Add Discovery Support for this Adapter

Hey,
please check out the Discovery-Adapter (https://github.com/ioBroker/ioBroker.discovery) and check if the device can be detected somehow (special url with defined answer, special port, upnp, udp or other technologies?!). If yes it would great if you could add a pull request to the Discovery-Adapter with the needed code.
Support needed? Contact me (Apollon77) in Forum as forum thread or PN ;-)
Ingo

Done.

Great!!!! Thank you or all!!


go/types, types2: revisit API behavior with respect to parameterized types

Right now, go/types has limited functionality for computing predicates "modulo type parameters": signatures are considered identical modulo renaming of type parameters in their type parameter list:
func F[A ~int](A) {}
func G[B ~int](B) {}
func H[C int](C) {}

type T[D ~int] func(D)

In this example, the types of F and G are considered identical, because they have pairwise identical type parameter constraints and their signatures are identical after substituting type parameters.  However, the signature of F is NOT identical to the signature of H (constraints don't match), and NOT identical to the underlying type of T (the signature itself does not have a type parameter list -- it uses type parameters from the declaration for T).
CL 379540 was intended to clean up some logic that existed to work around an earlier limitation that we didn't have instantiated methods. However, it inadvertently removed a "feature" that APIs like AssignableTo would somewhat work with uninstantiated types:
We ran into this example in kythe:
type Interface[T any] interface {
        Accept(T)
}

type Container[T any] struct {
        Element T
}

func (c Container[T]) Accept(t T) { c.Element = t }

Previously, we would report that Container is assignable to Interface (AssignableTo(Container, Interface) == true), because they would unify. However, this could be wrong, as in the following example:
type InterfaceInt[T ~int] interface {
        Accept(T)
}

type ContainerString[T ~string] struct {
        Element T
}

func (c ContainerString[T]) Accept(t T) { c.Element = t }

We would also have AssignableTo(ContainerString, InterfaceInt) == true, even though there is no instantiation for which this holds.
Related:

https://github.com/golang/go/issues/50658#issuecomment-1016975461. We don't expose any API for unification, and it would useful in several places.
#49912 -- an outstanding bug for comparing method signatures modulo type parameters. However, notably fixing this bug would not address the behavior with respect to the Interface type above: the methods on Interface are parameterized by the type declaration parameters, not receiver type parameters.

~Alternatively, callers could "instantiate" the types with identical arguments, but then the burden is pushed onto the caller to find type arguments that satisfy both type parameter lists, which is arbitrarily complicated.~ (EDIT: this wouldn't even really work: type parameters could be in arbitrary order, and so for the caller to even know which type parameters should correspond, they'd have to do the unification themselves).
I don't believe this affects type-checking, but matters for the API and for tools. We need to reconsider what is most useful here.
My initial thoughts are as follows: rather than what we currently do for function signatures, generalize to a limited form of unification in Identical, where type parameters are joined and then substituted in constraints.
This runs into a problem of "free type parameters"
func _[A *C, B *C, C any]() {
  var X func(A)
  var Y func(B)
}

type I[P *Q, Q any] interface{
  m(P)
}

The type of X and Y should not be identical.  What about the type of X and the signature of I.m? (EDIT: X and Y do NOT have identical types).
CC @griesemer @mdempsky

I don't believe this affects type-checking

Why do you say that? In my experience, it's always possible to write type-checker test cases that exercise identity rules. E.g., adding _ = &X == &Y to the function above will only type check if X and Y's types are identical. But maybe I misunderstand your point.

The type of X and Y should probably be identical.

I think func(A) and func(B) should only be considered identical if we also consider A and B identical. And then we should probably consider them also identical to *C.

I think func(A) and func(B) should only be considered identical if we also consider A and B identical.

Oops, that was a bad example, sorry (initially I was using local type declarations, and didn't fix the example properly). A and B are definitely not identical there. Will fix shortly.

CC @schroederc for awareness that beta2 will break kythe edges between generic types, and that we're aware of the problem.

I see. The question is about how to handle type identity for parameterized types that haven't yet been instantiated?
Is there a use case for defining identity on such types? If not, I'm inclined to say type identity is undefined on parameterized, non-instantiated types, and types.Identical should panic if it finds such a type.

Since generic types/functions can only be used in instantiated form (at which point they are not generic anymore) I believe @findleyr is correct that this shouldn't affect type checking of source code.
With respect to the API behavior, we need to determine the behavior for types.Identical as other API functions depend on its behavior.
If we have bound type parameters only, i.e., type parameters declared with the generic function/type, I believe we are in agreement that generic functions/types are identical if they can be made identical by renaming type parameters.
With respect to "free" type parameters, it seems that there are two different kinds:

type parameters declared with a generic type and used by methods of that type, and
type parameters declared in an outer scope (which must be a generic function).
And then there may be combined uses of 1) and 2).

For 1), given a generic type T, when we consider the type of a method m of T, we should be able to consider the type of the method expression T.m while ignoring the first parameter (the receiver that has become an ordinary parameter) for signature identity. That type is a function and type parameters of T are effectively "bound" with T.m. It seems that we should be able to handle this case like the bound type parameter case.
Using the example from above:
type Interface[T any] interface {
        Accept(T)
}

type Container[T any] struct {
        Element T
}

func (c Container[T]) Accept(t T) { c.Element = t }

The respective method expression types are (pseudo code):
func Interface.Accept[T any](Interface[T], T)
func Container.Accept[T any](Container[T], T)

Ignoring the first parameter for signature identity leads to two identical function signatures. Doing the same with the second (invalid) example from above will lead to two different function signatures because the constraints don't match.
Fo assignability, besides checking all methods, we also need to check assignment compatibility of the receiver type to the interface (because of type sets). Again, type parameters and constraints must be identical modulo renaming.
For case 2) I think we can only compare types/functions that are declared within the same environment, i.e., enclosed within the same generic function. There can be only one such generic function enclosing parameterized types/functions because we don't allow function-local method, or type-local type declarations. So for the purposes of comparing generic function-local types or functions, it seems that type parameters by the enclosing function simply have to match exactly. In other words, local types/functions are always different from any other type or function that is not defined in the same scope.

@griesemer had a long brainstorming session about this today, and finally arrived at the conclusion that there is no obvious way to define most predicates for uninstantiated generic types that Do The Right Thing.
For one, the two ms here must be identical:
type T1 int
type T2[P any] int

func (T1) m() {}
func (T2[P]) m() {}

and above we discuss making these ms identical:
type T1[P any] int
type T2[P any] int

func (T1[P]) m(P) {}
func (T2[P]) m(P) {}

but we don't want these m's to be identical?
type T1[P any] int
type T2[P ~int] int

func (T1[P]) m(P) {}
func (T2[P]) m(P) {}

Why do we care about constraints in the last, but not the first? This is incoherent, and more generally we concluded that any decision about the meaning of predicates on generic types runs into inconsistent and/or confusing behavior.
The problem is that there is more than one way to extend predicates to generic types: do all instantiations satisfy the predicate? Do any? Do we ignore constraints? Each of these could be useful interpretations in different situations. It's also a problem that some predicates (such as types.Implements) do not have access to an explicit type parameter list, and it's not clear that we can work around this limitation: the interpretations above rely on access to type parameter lists.
In the future we will probably want to expose APIs for substitution and unification that can efficiently answer any such question, but in the meantime we realized that the correct instantiations can answer many of them now, without additional APIs.
For example https://go.dev/cl/383094 extends types.AssignableTo in a way that we think would be useful for tools like Kythe, reporting if every instantiation V[A_1, ..., A_N] of a generic type V is assignable to the corresponding instantiation T[A_1, ..., A_N] of a generic type T.
So the good news is that we don't think any significant changes or new APIs are necessary for 1.18*, and this exercise (as well as related discussions around method identity) have given us a lot of insight into the APIs we should be thinking about for 1.19.
*well actually, we will probably want to remove the special handling that was added to types.Identical for generic signatures, since the same behavior can be replicated with instantiations similarly to CL 383094.

*well actually, we will probably want to remove the special handling that was added to types.Identical for generic signatures, since the same behavior can be replicated with instantiations similarly to CL 383094.

Actually, I think what we have now is correct. The original reason for this decision still holds: for the purposes of type identity, it is consistent to treat signature type parameters the same way we treat ordinary parameters, ignoring their names. It is also helpful: with the current behavior (ignoring names), if two generic signatures are identical they will behave the same during instantiation: func F[P ~int] (P) will behave identically in instantiation as func G[Q ~int] (Q), returning the same resulting signature identity and failing in the same cases.
If we just ignore type parameters entirely, we lose some of this useful behavior. The F and G above will be considered different, even though they behave the same, but func H[P any]() will be considered the same as func K[Q ~int](), even though they behave differently (one can be instantiated with string whereas the other can't).
Furthermore, I no longer think that it's logically inconsistent not to ignore receiver type parameter names in signatures. As we've seen, methods are not really "generic" at all: they are all methods on an instantiated type -- instantiated with their receiver type parameters. This last point is rather subtle, and warrants exposition.

Per https://github.com/golang/go/issues/50887#issuecomment-1031968549, I don't think there's anything more to do here other than try to document everything we've learned.
To preserve history, I'll change the title of this issue to track updating our documentation. I'm also going to remove the release-blocker label.


Adopt a better parsing/encoding strategy in OpenType.js

Based on the discussion seen below we currently have 2 choices:

"A binary parser generator" by  @Pomax (https://github.com/Pomax/A-binary-parser-generator)
"Restructure" by @devongovett (https://github.com/devongovett/restructure)

Manually parsing the complex datastructures of the OpenType file format is very time-consuming and error prone. And also results in verbose-code with lots of code duplication and which is tipically hard to read, understand and debug.
The approach presented by @Pomax in his "A binary parser generator" project  available at https://github.com/Pomax/A-binary-parser-generator seems very good because it relies on storing the overall file format structure in a "spec" file which is then used to automactically validate and parse OTF files (the project allows any file format to be specified, but OTF is what we're interested here, right :-D).
Several years ago I did something similar for handling the DWG file format in the GNU LibreDWG project (https://www.gnu.org/software/libredwg/). And based on our spec file we were able to generate both the format parser and the format encoder. I am not sure Pomax's implementation provides encoding as well, which would be strictly necessary for us here.
PS: As I see that Pomax has already contributed code to OpenType.js, I wonder if the proposal of incorporating the binary-parser-generator was already presented here in the past.

I've been using my restructure project for this. It supports both decoding and encoding all sorts of data types, and as a result of being developed for fontkit, it works pretty well for the OpenType format. 😜

I like the fact you seem to provide much more documentation than we get on Pomax's project.
I am also glad to see the description of the LazyArray, which is something I was expecting to see implemented in whatever solution we choose for performance reasons.
Also, it is nice to see you're working with require.js in a way that makes your solution more likely to be easily reusable as a dependency of other projects.

This looks good:
https://github.com/devongovett/fontkit/blob/master/src/opentype/GPOSProcessor.coffee
But it seems not all opentype tables are supported yet?
Also, the code seems slightly harder to read than Pomax's spec file syntax. I'll have to get used to that if we choose to go on with adopting your project here. I'm not totally convinced yet :-)

The tables are here

Pomax's project provides a test page with a binary-file element inspector which I liked a lot. Looks very good for testing & debugging.

The tables are here

I don't see a CFF.coffee file so I'm guessing that OpenType-CFF fonts are not supported.

They are, just in a different folder: https://github.com/devongovett/fontkit/tree/master/src/cff

Nice!

To be fair: my project was a PoC that I never got enough time to properly redo (there's a lot of ES6/ES2015 that can be sprinkled in to drastically increase the efficacy of a spec based approach), so the lack of docs is really because it's a lack of "this is done in any way, shape, or form" =)
(iirc there wasn't even all that much in the way of the Common Layout tables like GSUB/GPOS)

@Pomax have you already looked at @devongovett 's work on Restructure ? Would you vouch for it being used on OpenType.js instead of your binary-parser-generator ?

I also wonder what @fdb and @brawer think :)

I hadn't see restructure yet, it's now on my "to have a look at" list although I probably won't get to that until later in the week

I have been thinking about this for about a year now.  Really like Restruscture and was going to talk to Devon about it when I get back to California.
In my vision, there should be one spec of font formats that is maintained centrally, and used to generate data / code for each language (C, Python, JS).  But in the end this is so much work for no added benefit, so I won't personally be working on it.  But really like that others are interested in it and like to be involved in the design.

Note that compiling GSUB /GPOS needs table sharing logic. Devon, do you implement that?

I haven't tried compiling the GSUB/GPOS tables yet, but they should work since they are defined using types from restructure. As for table sharing logic, restructure doesn't currently have a way for two different pointers to point to the same place.

Basically we would need to calculate a checksum for each subtable and then optimize the serialization when checksums are equal. Or maybe some other procedure equivalent to that.

I guess that encoding the tables without this optimization would still produce valid OTF files. It's just that their size would be larger, which is undesirable. But I don't see programs being unable to handle reading the unoptimized tables. So it seems to be an issue of size optimization, instead of a matter of compliance to the file format spec.

OK. I just registered an issue about that:
https://github.com/devongovett/restructure/issues/20

Another good thing about restructure is that it's got lots of mocha tests (a total of 234).

So... after spending the afternoon inspecting the source code of these projects I came to the conclusion that it is not really restructure that we would use here, but instead we would actually be using fontkit, which has restructure as a dependency.
But then, fontkit's description is much like the opentype.js project description... Are these two projects actually trying to solve the same thing? How does fontkit differ from opentype.js ?

that is certainly an important question to answer.

Initially, when developing OpenType.js, I also had table specs (check out the first commit for example). Even though I'm a fan of the declarative approach, I soon discovered that a purely declarative approach is problematic. For some tables, e.g. head, the declarative approach works really well. However, things like CFF are too difficult to encode declaratively and do actually need logic for parsing. That's why I think it's difficult, if not impossible, to have a "universal spec" as @behdad suggested. I see that Fontkit solved some issues quite nicely, e.g. using a VersionedStruct to encode both long and short loca offsets, and I must say I like the approach.
OpenType.js and Fontkit were developed independently. I developed OpenType.js to "scratch my own itch": for NodeBox Live we needed access to the glyph shapes of a font, and nothing was available. Later on, through the efforts of @Pomax (and @louisremi), we also added font exporting.
The two projects do have a lot in common and I'm playing with the idea of merging efforts. It seems Fontkit has more support for things like WOFF, which we currently don't support.
I'm a bit torn at the moment. I like the things that Fontkit is doing, but I also think it would be silly to give up the effort that went into creating OpenType.js.

@davelab6 had listed a few projects trying to solve the same thing in byte-foundry/prototypo#115
@bramstein's opentype is still actively being developed.
I’d also add fonteditor-core by @kekee000 used in fonteditor.

From a personal perspective, it's also nice to have options as a user. Right now, I can use harfbuzz or freetype2 or roll something quick and dirty using a TTX export of a font, and having that same freedom in JS land is great: I can use OpenType.js or Fontkit, and they have their respective strengths and weaknesses (format and specific shaping support, memory footprints, etc). So I'm tempted to pitch my 2 cents as "even if they do the same, the way they do things are different enough to be valueable as checks on each other". One library to rule them all always sounds tempting, but multiple libraries to divide and conquer based on the tool you need, to me, has stronger appeal.

Some history about fontkit for those who don't know: it was originally written for PDFKit, my PDF generation library, over 4 years ago, and existed as a part of that project for a while. Then, I kept getting a ton of bug reports about font issues, much of which were related to complex script support. So, a couple years ago I started writing fontkit as it exists today, and I extracted it as a separate project so it could be useful outside of just PDFKit. It is finally getting to state where I can ship it as the default font engine in PDFKit, and I plan to do so shortly.
Fontkit and opentype.js do have slightly different goals as far as I can tell. Fontkit was originally developed to solve the layout problem, along with PDF subsetting, and although it has grown to support other things like glyph decoding, layout remains its primary goal in my opinion.
On the other hand, from what I can tell, opentype.js is mostly focused on encoding and decoding glyph paths, rather than glyph layout. Fontkit currently does not support glyph encoding from paths like opentype.js does, just decoding. I may need to do something in this area in the future though, in order to support variation fonts in PDFs...


Remove distro-provided JRE

We don't need Java 8 from Ubuntu, we have 15 from AdoptOpenJDK.

Keeping in mind java8 may be required in some situations (spigot + denizen plugin).

Good comments but all is good.
The mainline will remain at 8 to maximize compatibility.
The targeted branch is named "multiarch-latest" since it derives the same Ubuntu base and multi-processor architecture support from "multiarch".
The "adopt" variants are amd64 only and Alpine, so that they're smaller.

Ah gotcha, wasn't 100% sure on the structure of the branches and hub images or that this was an adopt based Dockerfile, guess I should have read clearly. Nice change.


Can't run pyglet loop in a separate process: "XIO:  fatal IO error 0 (Success) on X server ":0.0""

I'm trying to run pyglet event loop in a separate process, and it mostly works (input events, scheduled events), however nothing is displayed and python exits with code 1
Program output is the following:
starting pyglet
pyglet started
PygletProcess started
update
draw
update
draw
PygletProcess stopped
XIO:  fatal IO error 0 (Success) on X server ":0.0"
      after 166 requests (166 known processed) with 11 events remaining.

The program in question:
from multiprocessing import Process

import pyglet

class PygletProcess(Process):
    def __init__(self):
        super(PygletProcess, self).__init__()
        self.event_loop = pyglet.app.EventLoop()

        self.window = pyglet.window.Window(fullscreen=True)
        self.window.set_caption('test')

        label2 = pyglet.text.Label(text='hello',
                                   x=0, y=0)

        @self.event_loop.event
        def on_window_close_loop(win):
            self.event_loop.exit()
            return pyglet.event.EVENT_HANDLED

        @self.window.event
        def on_window_close(win):
            self.event_loop.exit()
            return pyglet.event.EVENT_HANDLED

        @self.window.event
        def on_key_press(symbol, modifiers):
            if symbol == pyglet.window.key.ESCAPE:
                self.event_loop.exit()

        @self.window.event
        def on_draw():
            print('draw')
            pyglet.gl.glClearColor(0.92, 0.78, 0.60, 1)
            self.window.clear()
            label2.draw()

        def update(dt):
            print('update')

        interval = 1
        self.event_loop.clock.schedule_interval(update, interval)

    def run(self):
        print('PygletProcess started')
        self.event_loop.run()
        print('PygletProcess stopped')

if __name__ == "__main__":
    print('starting pyglet')
    pyg = PygletProcess()
    pyg.start()
    print('pyglet started')

If main section is changed to call pyg.run() instead, everything works as expected, because it runs in single-process mode.
Versions tested:
Python 3.8.1
pyglet==1.5.0
pyglet==1.4.9
$ xdpyinfo | grep version
version number:    11.0
X.Org version: 1.20.7

Is this a bug or multiprocessing is not supported?
How do I debug this further?

multiprocessing is known to work, but it's not specifically supported.
The biggest issue you might face is that Window and OpenGL resources won't work correctly across processes. What you can try to do is move the pyglet import into your other process, such as the top of the run method.

Thanks @benmoran56, that has helped on Linux too.
Though, I'm surprised that openGL resources get acquired on module import. Wouldn't it be more straightforward to acquire them upon creating a window instead?

pyglet creates a background OpenGL context (and invisible "Shadow Window") to allow loading resources (images/textures) before creating your Window. You can disable this behavior by setting the following option immediately after import:
pyglet.options['shadow_window'] = False
That might also work for this case.

@3demax If you are able to find a solution, could we close the ticket?

Oh, for sure!
Thanks again for the help!


Google.OrTools.dll not found

I use VS2019 on Windows to add Nuget Package Google.OrTools to my .Net Standard project. Running the code I experience following exception:
System.IO.FileNotFoundException: 'Could not load file or assembly 'Google.OrTools, Version=7.3.7083.0, Culture=neutral, PublicKeyToken=77370d3086a71853' or one of its dependencies. The system cannot find the file specified.'
I found that others used to have similar problems and added a reference to Google.OrTools.dll manually. As I was not able to find that file anywhere it was not a solution to me. After downloading the windows package I checked if I can find the missing file there...unfortunately not.
Can you please provide direction for that problem.
Andreas

Make sure your project targets dotnet core 2.0
Le sam. 7 sept. 2019 à 20:33, CubusAnderl<EMAIL_ADDRESS>a
écrit :

I use VS2019 on Windows to add Nuget Package Google.OrTools to my .Net
Standard project. Running the code I experience following exception:
System.IO.FileNotFoundException: 'Could not load file or assembly
'Google.OrTools, Version=7.3.7083.0, Culture=neutral,
PublicKeyToken=77370d3086a71853' or one of its dependencies. The system
cannot find the file specified.'
I found that others used to have similar problems and added a reference to
Google.OrTools.dll manually. As I was not able to find that file anywhere
it was not a solution to me. After downloading the windows package I
checked if I can find the missing file there...unfortunately not.
Can you please provide direction for that problem.
Andreas
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/1558?email_source=notifications&email_token=ACUPL3N4H4FBYRM2U3SXJDTQIPXW5A5CNFSM4IUQ5EAKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HJ67J3Q,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACUPL3MDO53IYD4I3AB7HPDQIPXW5ANCNFSM4IUQ5EAA
.

And that your project is 64 bit.
Le sam. 7 sept. 2019 à 20:46, Laurent Perron<EMAIL_ADDRESS>a écrit :

Make sure your project targets dotnet core 2.0
Le sam. 7 sept. 2019 à 20:33, CubusAnderl<EMAIL_ADDRESS>a
écrit :

I use VS2019 on Windows to add Nuget Package Google.OrTools to my .Net
Standard project. Running the code I experience following exception:
System.IO.FileNotFoundException: 'Could not load file or assembly
'Google.OrTools, Version=7.3.7083.0, Culture=neutral,
PublicKeyToken=77370d3086a71853' or one of its dependencies. The system
cannot find the file specified.'
I found that others used to have similar problems and added a reference
to Google.OrTools.dll manually. As I was not able to find that file
anywhere it was not a solution to me. After downloading the windows package
I checked if I can find the missing file there...unfortunately not.
Can you please provide direction for that problem.
Andreas
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/1558?email_source=notifications&email_token=ACUPL3N4H4FBYRM2U3SXJDTQIPXW5A5CNFSM4IUQ5EAKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HJ67J3Q,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACUPL3MDO53IYD4I3AB7HPDQIPXW5ANCNFSM4IUQ5EAA
.

Works with .Net Core
However I need to target .Net Standard or .Net Framework. Any chances to get Google Or Tools run for these?

Next release should target 4.5.2 and above.
See https://github.com/google/or-tools/pull/1500
Laurent Perron | Operations Research |<EMAIL_ADDRESS>| (33) 1 42 68 53
00
Le sam. 7 sept. 2019 à 23:37, CubusAnderl<EMAIL_ADDRESS>a
écrit :

Works with .Net Core
However I need to target .Net Standard or .Net Framework. Any chances to
get Google Or Tools run for these?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/1558?email_source=notifications&email_token=ACUPL3KQW7SMV76SPUS2IHLQIQNJXA5CNFSM4IUQ5EAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6FCW5I#issuecomment-529148789,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACUPL3IM2MMFT45ZGE4DUGLQIQNJXANCNFSM4IUQ5EAA
.

That's awesome. I am excited to have it available for my project soon. According to latest release schedule I am positive that we find a new version with in the next few days.
Thanks for your amazing work Laurent!
Andreas

When will the next version be released? I'm in need for the support for .NET Framework (instead of .NET Core).

Normally today, provided the builds are OK.
Laurent Perron | Operations Research |<EMAIL_ADDRESS>| (33) 1 42 68 53
00
Le ven. 11 oct. 2019 à 15:16, Merlijn<EMAIL_ADDRESS>a écrit :

When will the next version be released? I'm in need for the support for
.NET Framework (instead of .NET Core).
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/1558?email_source=notifications&email_token=ACUPL3OCRAGATF6DBHYEEMDQOB4D3A5CNFSM4IUQ5EAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEA762HA#issuecomment-541060380,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACUPL3J4DQQWHG25X7WB6ILQOB4D3ANCNFSM4IUQ5EAA
.

Same scenario as OP, new version, still getting "File not found Exception"
.Net Framework 4.7.2

Are you in 64 bit ?
Le ven. 11 oct. 2019 à 18:55, John Alejandro Zapata <
<EMAIL_ADDRESS>a écrit :

Same scenario as OP, new version, still getting "File not found Exception"
.Net Framework 4.7.2
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/1558?email_source=notifications&email_token=ACUPL3MJX67KVJ5WDOXZC2TQOCVZFA5CNFSM4IUQ5EAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBAS43I#issuecomment-541142637,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACUPL3MRYS5WOLWN3NHR7X3QOCVZFANCNFSM4IUQ5EAA
.

Are you in 64 bit ? Le ven. 11 oct. 2019 à 18:55, John Alejandro Zapata <<EMAIL_ADDRESS>a écrit :
…
Same scenario as OP, new version, still getting "File not found Exception" .Net Framework 4.7.2 — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1558?email_source=notifications&email_token=ACUPL3MJX67KVJ5WDOXZC2TQOCVZFA5CNFSM4IUQ5EAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEBAS43I#issuecomment-541142637>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACUPL3MRYS5WOLWN3NHR7X3QOCVZFANCNFSM4IUQ5EAA .

I was running in Any CPU, my bad,
i actually had to start a new wrapper with python as my project cant run in 64 bit because of another dependency.
Thank you Mr Perron.


Issue on page /notebooks/05-Python_I.html

Errors in your Python code
Errors that cannot be debugged
tips
This works for expressions as well! Try something like the following to see: from math import cos, radians theta = 30 f"{theta=}  {cos(radians(theta))=:.3f}"

better to be like this
This works for expressions as well! Try something like the following to see:
from math import cos, radians
theta = 30
f"{theta=}  {cos(radians(theta))=:.3f}"

Fixed, thanks!


Did you try to fine-tune transformers LM with Ranger?

Recent transformers architectures are very famous in NLP: BERT, GPT-2, RoBERTa, XLNET. Did you try to fine-tune them on some NLP task? If so, what was the best Ranger hyper-parameters and learning rate scheduler?

Testing for XLnet should be prioritarised as it is the current best state of the art.
ERNIE 2.0 would be interesting too.

@avostryakov I tried fine-tuning a BERT based model for joint NER and relation classification. It performs about ~1.5% worse for my tasks than the AdamW implementation in Transformers:
AdamW

lr: 3e-5
betas: (0.9, 0.999)
eps: 1e-6
weight_decay: 0.1
correct_bias: True

Ranger

lr: 3e-4
betas: (0.95, 0.999)
eps: 1e-6
weight_decay: 0.1

It is possible that with more tunining I might be able to close the gap.

I'm working with DETR which is object detection with transformer internally and will test it out there soon.
Note that Ranger now has GC (gradient centralization) and will be interesting to see if that helps for transformers.

How does ranger perform for Detr?


Use version from VERSION for app -v and macOS bundle version

Use version from VERSION for app -v and macOS bundle version. I've bumped the version in VERSION to match the one that was hard-coded in the application. This should solve #20.

I agree. I copied what Marian did, but the cmake documentation also seems to recommend using ${CMAKE_CURRENT_BINARY_DIR}.


Showcase Submission for 131-bouncing-dvd-logo

Thank you joseph for your contribution! A member of the Coding Train team will review it shortly.

Bouncing Box Game
joseph

Great work!


Custom Tensorflow lite model integration in flutter app

Use case
I have a tensorflow lite ML model that I'd like to use in a flutter application.
One of my first options was to test plugins available on pub dev. Unfortunately, I was unable to find one that is still working in 2023.
Both of the major two plugins appear to have been abandoned. tflite_flutter and tflite

Proposal

Is there an official flutter package available for this use case?
Or are there any other options for integrating Tensorflow models into a Flutter app?

@baimamboukar
Check this documentation and see if it helps for your use case.
Also, your question is better suited for support channels like StackOverflow, so please ask the same there.
Closing from here.


chore: Release v0.4.1

release v0.4.1 to make IonExtension pub
command run:
cargo release patch --no-publish --no-push -x

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

partiql crates updated to v0.4.1: https://crates.io/teams/github:partiql:ion?sort=recent-updates


Add validators

Validator to exclude unwanted characters from the name. Apparently the //// is needed to actually exclude a backslash. This is because it is also used as the escape character

Looks good, I think it would be nice if we put the pattern string in a var and use that throughout the code so we can easily change all of the regexpatterns if we want. We could also write a field validator and re-use it https://docs.pydantic.dev/2.0/usage/validators/#reuse-validators

What makes it a bit weird is that all models are in different files. What we could do is make our own version of the base model that always includes the name (and maybe description since those 2 are the only ones used everywhere) or make a seperate utils file for standard validators, but I'm not 100% sure if that's worth it if is just for checking the name


Doesn't work in a monorepo if the package.json in the root of the project doesn't have a "name" property

npx knip
Reading workspace configuration(s)...
Missing package name in (project directory)/package.json

I'm not sure why I would have to set a "name" property in the package.json in the root of my project. As no other tools require this.

Since Knip requires/uses a root workspaces, and it can be targeted in features like --directory, there must be a name. Also for features like #538.

It should still work if the root package.json doesn't have a name. It's very uncommon in a monorepo for the root to have the name set.

If I'm not mistaken, you're the first to mention this. Feel free to open a PR, happy to review!

:rocket: This issue has been resolved in v5.1.2. See Release 5.1.2 for release notes.

Alright, here we go :)


Path Validation and autocompletion

The goal is to create Path Validation based on this great example: https://discord.com/channels/314103695568666625/314115147184603137/1145171814314688598
Milestones:

Path V.1

[x] Validation for &Identifier
[x] Support for Autocompletion &Identifier

Path V.2

[x] Validation for <>
[ ] Support for Autocompletion <>
[x] Requires a overview of all rules files (To Create)

Path V.3

[ ] Validation for the other open options
[ ] Support for the other options autocompletion

Will be handled in separate issues


Bounding grid updates on properties panel change of scale

Check that the bounding box is regenerated when an object is rescaled

This is probably a scenery thing @skalarproduktraum

The problem is that generateBoundingBox in scenery doesn't use scale factors

This is still a problem but it manifests differently.
To reproduce:

Add an object like a sphere
Toggle bounding box of the sphere
Set the scale of the sphere

The bounding box adjusts for the scale, but the font and legend does too. See:

@skalarproduktraum should we make a corresponding issue in scenery?


Support commit SHA in --remote-branch option

Related:

https://github.com/yamadashy/repomix/issues/195?notification_referrer_id=NT_kwDOByTsI7UxMzc1NzczNzE5ODoxMTk4NjAyNTk

https://github.com/yamadashy/repomix/pull/196

https://github.com/yamadashy/repomix/pull/199

The older PR claims to support branch names, tags, and commit hashes, but commit hashes are not actually supported.
Checklist

[ ] Run npm run test
[ ] Run npm run lint

Hi, @tranquochuy645 !
Thank you for the implementation!
Your changes look good, but I wanted to add support for short commit hashes, so I've added a commit to handle that. I've verified that all the following cases now work:
# branch
npm run cli-run -- --remote yamadashy/repomix --remote-branch main

# full hash
npm run cli-run -- --remote yamadashy/repomix --remote-branch bc4f6fd6eb1a680c6d88404da5eb0844ee795e6e

# short hash
npm run cli-run -- --remote yamadashy/repomix --remote-branch bc4f6f

How does this look to you?

Sorry for the delay in reviewing this! I've rebased and force pushed to keep the branch up to date.
I'll start reviewing it now.

@tranquochuy645
The code looks perfect! I've tested it out and everything seems to be working as expected.
Thanks for your improvements. I'll merge this now.

I added an example of using a commit hash with the --remote-branch flag to make it clearer that this flag supports not just branches but also tags and commit hashes. #232


JSON encode support for struct objects

Add StructJsonSerializeTrait to struct objects, to be able to json_encode objects

First of all: Thanks for the PR! It needs to be discussed, though.
Json-Encoding the complete struct will expose all private properties (and will use them as object keys in the resulting json) - we won't be able to change them any longer. Second, every addition in the struct will be present in the resulting json, causing changed datasets on client side every time we add/change the internals.
@WorksDev Didn't we talk about adding multiple output formats in the soap connector?

Hi,
thanks also from my side for the contribution.
@usox I mean yes, in future development stages - as long as there is a valid scenario for it.
From my point of view, this is currently not the case for the reasons mentioned above.
@sradinger what are you trying to achieve with it?

Thank you for your answers.
I understand your doubts.
I need the response for logging purposes as JSON.
For the further processing of the data I would like to use the created response object.
If the adjustments are not ok for you, you can ignore them and I will program an alternative without adjustments to this component.

I need the response for logging purposes as JSON.

Well, I guess we could consider adding a Logger to the connector to simply log all outgoing/incoming data. From my pov, it shouldn't be that hard.

For the further processing of the data I would like to use the created response object.

I can image lots of use cases for structured json data and I think, it's a valid request. Imo the best way would be a mapping of the internal keys to external ones. This has to be done for every single method, though.
I.e. for the AccountingItem-Struct:
public function jsonSerialize(): array {
  return [
    'externalKeyForDescription' => $this->description,
    'externalKeyForMandatorId' => $this->mandatorId,
    'externalKeyForAccountingDate' => $this->accountingDate,
    'externalKeyForChargeCount' => $this->chargeCount,
  ];
}

@usox yes it's a valid request, that's out of the question - but IMHO I don't think that these "structures" should be inflated further.  There are already possibilities to arrange this elsewhere (ReflectionClass) if needed.
Maybe I lack creativity, I can think of a few cases why I need a different output format (connector-side!), with the objects you can do what you want in the end.
However, it makes sense for me to implement a logger in the connector.


panic: runtime error: slice bounds out of range

I'm doing a very big push operation (the same as in #999), and the pre push lfs transfers go on fine for a couple of hours, but at some point, this happens:
errors2.txt
Any ideas?

What does git lfs version show?

git-lfs/1.1.1 (GitHub; linux amd64; go 1.4.2)
But this is actually a version where I have added those Waits to get rid of the zombies. But I did encounter this behaviour already with a version without those changes.

Thanks. Having the exact version ensures I'm looking in the right place. The problem is here on line 374: https://github.com/github/git-lfs/blob/05aa87dbf5c6c7feced04bb43c907cc3ba4e1f76/lfs/scanner.go#L368-L374:
For some reason, git cat-file --batch-check is returning a bad line. Maybe it's not reporting the size for some reason? Here's a diff that would print the bad line, if you have a way to replicate this easily:
diff --git a/lfs/scanner.go b/lfs/scanner.go
index 87abed1..0c0f866 100644
--- a/lfs/scanner.go
+++ b/lfs/scanner.go
@@ -371,6 +371,7 @@ func catFileBatchCheck(revs chan string) (chan string, error) {
                        // type is at a fixed spot, if we see that it's "blob", we can avoid
                        // splitting the line just to get the size.
                        if line[41:45] == "blob" {
+                               tracerx.Printf("BLOB: %q", line)
                                size, err := strconv.Atoi(line[46:len(line)])
                                if err != nil {
                                        continue

If you could help track it down, that'd be swell. If you just want to get on with your work, you could try out my fix at #1006 :)

Awesome, thanks! I'll try to replicate it with the traces. I suppose I will need to set GIT_TRACE=1 for this trace to show? Or is there a way to show this with some less verbose debugging level?

I think this was after all caused by my own changes (similar to #999). Sorry for the inconvenience! I'll try to create a better PR!

No worries. The code was too trusting of git's behavior, anyway.


NOTE: Optimizing OOM on a single GPU

The following is investigating / follow up on a single GPU facing high memory fragmentation. Causing OOM issues despite clearly having sufficient space.
This manifest to either of the following
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 186.00 MiB (GPU 0; 22.13 GiB total capacity; 16.56 GiB already allocated; 164.62 MiB free; 16.78 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

or simply PYTORCH_CUDA_ALLOC_CONF configured
RuntimeError: The following operation failed in the TorchScript interpreter.
Traceback of TorchScript (most recent call last):
  File "/home/picocreator/rwkv-proj/picocreator-memory-experiment/RWKV-v4wavenet/src/model.py", line 323, in forward
        xr = x * self.time_mix_r + xx * (1 - self.time_mix_r)
        k = self.key(xk)
        k = torch.square(torch.relu(k))
                         ~~~~~~~~~~ <--- HERE
        kv = self.value(k)
        return (torch.sigmoid(self.receptance(xr)) * kv,
RuntimeError: Allocation on device 0 would exceed allowed memory. (out of memory)
Currently allocated     : 16.66 GiB
Requested               : 31.27 MiB
Device limit            : 22.13 GiB
Free (according to CUDA): 4.62 MiB
PyTorch limit (set by user-supplied memory fraction)
                        :<PHONE_NUMBER>4.00 GiB

Subsequently, it is suspected that high fragmentation is occurring when required to keep track of a large number of forward loss objects, before needing to do backprop.
So instead of
main segment loop
    forward pass on chunk

reverse segment loop
    backward pass on chunk

We do the following in a single loop
main segment loop
     forward pass on chunk
     backward pass on chunk

This seems to confer a speed advantage as well (atleast in the case of a single 4090, finetuning 1B5 model, at ctx 2048)

This has been confirmed with the changes here : https://github.com/PicoCreator/RWKV-LM-LoRA/commit/e555c6e85381f639492e61b78ccc202816b87c9c?diff=split
Not much to see in the loss curve, the model is able to continue training beyond the usual spot where it OOM's

Additionally the estimate training time dropped from 38 hours, to 29 hours, meaning there is an overall speed up as well


rework the aws cli docs

After a discussion with @MarcelStranak, we will discuss refactoring of the AWS CLI v2 docs next week. I will merge this PR to give way to some new docs where the AWS profile needs to be mentioned.


Support Query Param Matching

What would you like to be added:
It may be helpful to extend matching to support query parameters. Although not as common as header matching, this could be useful in some cases.
Why is this needed:
From what I can tell, some proxy implementations support query param matching but definitely not all. If we support this it will likely have to be an extended feature.

This and HTTP method based matching -- these two good candidates that could have custom conformance and use the extensionRef inside HTTPRouteMatch. If anyone takes this up, would love to get some feedback on the extension point.

Traefik is able to do QueryPath matching. We're currently doing a base implementation on the Spec. We could think about experimenting with the extension afterwards to support Query Param Matching?

/remove-lifecycle stale
/lifecycle frozen

/remove-lifecycle stale
/lifecycle frozen


add release-notes sts

need: https://github.com/chainguard-dev/chainguard-devops/pull/144

ready!

thanks hector
i will run some smoke test


Address minor warnings and lint issues

A series of minor changes/fixes:

De-capitalize error messages in line with guidance
Export a func needed in #1467 (draft)
Remove a test func that has become unused

Submission Checklist
Please take the following actions before submitting this pull request.

Fork your PR branch from the Toolkit "develop" branch (not main)
Test all changes with pre-commit in a local branch #
Confirm that "make tests" passes all tests
Add or modify unit tests to cover code changes
Ensure that unit test coverage remains above 80%
Update all applicable documentation
Follow Cloud HPC Toolkit Contribution guidelines #

I decided to remove one commit that is strictly necessary for #1467.


Rancher error syncing 'fleet'

Hello
I have installed rancher rancher-2.5.5 on kubernetes 1.19, the pods rancher is reporting this error:
[ERROR] error syncing 'fleet': handler mcm-start: error running the jail command: exit status 2, requeuing
The option used was:
helm install rancher rancher-stable/rancher
--namespace cattle-system
--set hostname=rancher.ptmindra.net
--set ingress.tls.source=secret
What kind of request is this (question/bug/enhancement/feature request):
Steps to reproduce (least amount of steps as possible):
Result:
Other details that may be helpful:
Environment information

Rancher version (rancher/rancher/rancher/server image tag or shown bottom left in the UI):
Installation option (single install/HA):

Cluster information

Cluster type (Hosted/Infrastructure Provider/Custom/Imported):
Machine type (cloud/VM/metal) and specifications (CPU/memory):
Kubernetes version (use kubectl version):

(paste the output here)

Docker version (use docker version):

(paste the output here)

Hello
My details are:
Image rancher: rancher/rancher:v2.5.5
Installation option is single install
Kubernetes version is V1.19 in bare metal
I use CRIO as runtime of container:
OS-IMAGE		           CONTAINER-RUNTIME
CentOS Linux 7 (Core)        cri-o://1.19.0

Same here, did you figure out your problem ?

Same here, did you figure out your problem ?

If you haven't found workaround yet, it is here: https://github.com/rancher/rancher/issues/27160#issuecomment-643653677


Address Bias in RandomValueStringGenerator

Fixes #639.
I added the URL safe characters hyphen (-) and underscore (_) as recommended.  If there are other characters that are preferred let me know.

Thanks for the PR @jokerr. This is now in master.


UIImageView+AFNetworking: Prevent stuck state for malformed urlRequest

Addresses issue within setImageWithURLRequest:placeholderImage:success:failure:.
Issue:
If a NSURLRequest object is initialized with a malformed url (eg. passing an uninitialized string into URLWithString:), and is passed into setImageWithUrlRequest:, neither blocks (success or failure) get fired. When the method encounters a nil URL property of the NSURLRequest argument, cancelImageDownloadTask is called, but it's functionality depends on the existence of af_activeImageDownloadReceipt. In this case af_activeImageDownloadReceipt is nil, as the early return prevents a download task from starting, thus leaving the method caller waiting.
Proposal:
Replace cancelImageDownloadTask with the firing of the failure block.

@SlaunchaMan from what I can tell, build job #5 failed due to a fastlane error: https://travis-ci.org/AFNetworking/AFNetworking/jobs/328142514
I'd restart the build job but do not have the proper permissions to do so; would you be able to restart the build job?

Restarted!

@SlaunchaMan got my fork UTD w/ master, and pushed a merge into the PR and saw all build jobs failing
it seems master has a handful of failing tests:

Looks like some certificates expired that we use for tests. I'll update those shortly!

Tests have been updated. Once you merge in master and the tests pass this is good to go!

master merged and seein' ✅s. 👍


hw: Added fpga vregfile

Content:
Added FPGA specific register file for Spatz.
Status:
To be tested in the Carfield CI.

Linting SV fails, should we use Bender or SV macros for this?


Add query parameter on coverage badge image url

This should be tested by manually updating the badge on S3.

I'm still missing cache control headers from cloudfront. I think I need to deploy a cloudfront function to do this.

infra changes were merged in https://github.com/artichoke/project-infrastructure/pull/336. The badge in the readme re-rendered.


Segmentation Fault in OpenSSL-1.1.1 CRYPTO_THREAD_write_lock when using an engine for RSA

I am running openssl-1.1.1k on RHEL 8.10 on s390x architecture. I understand 1.1.1k is quite out of date, but it is the latest available in the RHEL8.10 public repos, and I do not see any relevant changes in the source tree that would suggest a fix in 1.1.1w. I am using the ibmca engine in my openssl configuration.
When RSA is handled by the engine, there is a segmentation fault caused by openssl originating in RSA_free, in the call to CRYPTO_THREAD_write_lock. It appears as though the global_engine_lock may already be NULL here, and very similar to (if not the same as) issue #18578 which was resolved in OpenSSL-3.0.
Since RHEL has no plans to migrate to openssl-3.0, this issue would ideally also be resolved in openssl-1.1.1.
To reproduce, using openssl-1.1.1, configure an engine to handle RSA algorithms. The segmentation fault is triggered upon SSL_CTX_free() which is invoked at the end of a sudo invocation, so calling any executable with sudo should trigger it. For example:
$ sudo echo hello world
hello world
Segmentation fault

Here's a stack trace:
$ sudo gdb --args  /usr/bin/sudo  echo hi
GNU gdb (GDB) Red Hat Enterprise Linux 8.2-20.el8
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "s390x-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/sudo...Reading symbols from .gnu_debugdata for /usr/bin/sudo...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Missing separate debuginfos, use: yum debuginfo-install sudo-1.9.5p2-1.el8_9.s390x
(gdb) run
Starting program: /usr/bin/sudo echo hi
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
[Detaching after fork from child process 574675]
hi

Program received signal SIGSEGV, Segmentation fault.
0x000003ffa300ce84 in pthread_rwlock_wrlock () from /lib64/libpthread.so.0
(gdb) bt
#0  0x000003ffa300ce84 in pthread_rwlock_wrlock () from /lib64/libpthread.so.0
#1  0x000003ffa1d0c0fa in CRYPTO_THREAD_write_lock () from /lib64/libcrypto.so.1.1
#2  0x000003ffa1c8ed08 in ENGINE_finish () from /lib64/libcrypto.so.1.1
#3  0x000003ffa1cf1712 in RSA_free () from /lib64/libcrypto.so.1.1
#4  0x000003ffa1caaafe in EVP_PKEY_free_it () from /lib64/libcrypto.so.1.1
#5  0x000003ffa1cab8f8 in EVP_PKEY_free () from /lib64/libcrypto.so.1.1
#6  0x000003ffa1d27e0a in pubkey_cb () from /lib64/libcrypto.so.1.1
#7  0x000003ffa1c16c42 in asn1_item_embed_free () from /lib64/libcrypto.so.1.1
#8  0x000003ffa1c16e48 in asn1_template_free () from /lib64/libcrypto.so.1.1
#9  0x000003ffa1c16c1c in asn1_item_embed_free () from /lib64/libcrypto.so.1.1
#10 0x000003ffa1c16e48 in asn1_template_free () from /lib64/libcrypto.so.1.1
#11 0x000003ffa1c16c1c in asn1_item_embed_free () from /lib64/libcrypto.so.1.1
#12 0x000003ffa1c16d50 in ASN1_item_free () from /lib64/libcrypto.so.1.1
#13 0x000003ffa1d1b20e in X509_OBJECT_free () from /lib64/libcrypto.so.1.1
#14 0x000003ffa1d08248 in OPENSSL_sk_pop_free () from /lib64/libcrypto.so.1.1
#15 0x000003ffa1d1b7d2 in X509_STORE_free () from /lib64/libcrypto.so.1.1
#16 0x000003ffa1ebb3e4 in SSL_CTX_free () from /lib64/libssl.so.1.1
#17 0x000003ffa21b97de in ldap_int_tls_destroy () from /lib64/libldap-2.4.so.2
#18 0x000003ffa3304ef0 in _dl_fini () at dl-fini.c:141
#19 0x000003ffa2dc9e0a in __run_exit_handlers () from /lib64/libc.so.6
#20 0x000003ffa2dc9ef8 in exit () from /lib64/libc.so.6
#21 0x000002aa29d877d2 in main ()

System information:
$ uname -a
Linux MYHOST 4.18.0-553.8.1.el8_10.s390x #1 SMP Fri Jun 14 02:46:29 EDT 2024 s390x s390x s390x GNU/Linux

$ openssl version
OpenSSL 1.1.1k  FIPS 25 Mar 2021

$ cat /etc/pki/tls/openssl.cnf
openssl_conf = default_modules

[ default_modules ]
engines = engine_section

[engine_section]
ibmca = ibmca_section

[ibmca_section]
dynamic_path = /usr/lib64/engines-1.1/ibmca.so
engine_id = ibmca
init = 1
default_algorithms = RAND,DH,DSA  #,RSA -- adding RSA currently causes a segfault on ssl context free.

can you set breakpoints in the following functions:
do_engine_lock_init
engine_cleanup_int
and re-run your test
it would seem the engine subsystem either never got initialized, or was cleaned up prior to the exit handlers running

Yup ... it does trigger the breakpoint in engine_cleanup_int:

(gdb) break engine_cleanup_int
Function "engine_cleanup_int" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (engine_cleanup_int) pending.

(gdb) break do_engine_lock_init
Function "do_engine_lock_init" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 2 (do_engine_lock_init) pending.

(gdb) run
Starting program: /usr/bin/sudo echo hello world
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib64/libthread_db.so.1".
[Detaching after fork from child process 596760]
hello world

Breakpoint 1, 0x000003ff8350f152 in engine_cleanup_int () from /usr/lib64/libcrypto.so.1.1
(gdb) bt
#0  0x000003ff8350f152 in engine_cleanup_int () from /usr/lib64/libcrypto.so.1.1
#1  0x000003ff8353986a in OPENSSL_cleanup () from /usr/lib64/libcrypto.so.1.1
#2  0x000003ff84649e0a in __run_exit_handlers () from /usr/lib64/libc.so.6
#3  0x000003ff84649ef8 in exit () from /usr/lib64/libc.so.6
#4  0x000002aa3ce077d2 in main ()
(gdb) cont
Continuing.

Program received signal SIGSEGV, Segmentation fault.
0x000003ff8488ce84 in pthread_rwlock_wrlock () from /usr/lib64/libpthread.so.0
(gdb) bt
#0  0x000003ff8488ce84 in pthread_rwlock_wrlock () from /usr/lib64/libpthread.so.0
#1  0x000003ff8358c0fa in CRYPTO_THREAD_write_lock () from /usr/lib64/libcrypto.so.1.1
#2  0x000003ff8350ed08 in ENGINE_finish () from /usr/lib64/libcrypto.so.1.1
#3  0x000003ff83571712 in RSA_free () from /usr/lib64/libcrypto.so.1.1
#4  0x000003ff8352aafe in EVP_PKEY_free_it () from /usr/lib64/libcrypto.so.1.1
#5  0x000003ff8352b8f8 in EVP_PKEY_free () from /usr/lib64/libcrypto.so.1.1
#6  0x000003ff835a7e0a in pubkey_cb () from /usr/lib64/libcrypto.so.1.1
#7  0x000003ff83496c42 in asn1_item_embed_free () from /usr/lib64/libcrypto.so.1.1
#8  0x000003ff83496e48 in asn1_template_free () from /usr/lib64/libcrypto.so.1.1
#9  0x000003ff83496c1c in asn1_item_embed_free () from /usr/lib64/libcrypto.so.1.1
#10 0x000003ff83496e48 in asn1_template_free () from /usr/lib64/libcrypto.so.1.1
#11 0x000003ff83496c1c in asn1_item_embed_free () from /usr/lib64/libcrypto.so.1.1
#12 0x000003ff83496d50 in ASN1_item_free () from /usr/lib64/libcrypto.so.1.1
#13 0x000003ff8359b20e in X509_OBJECT_free () from /usr/lib64/libcrypto.so.1.1
#14 0x000003ff83588248 in OPENSSL_sk_pop_free () from /usr/lib64/libcrypto.so.1.1
#15 0x000003ff8359b7d2 in X509_STORE_free () from /usr/lib64/libcrypto.so.1.1
#16 0x000003ff8373b3e4 in SSL_CTX_free () from /usr/lib64/libssl.so.1.1
#17 0x000003ff83a397de in ldap_int_tls_destroy () from /usr/lib64/libldap-2.4.so.2
#18 0x000003ff84b04ef0 in _dl_fini () at dl-fini.c:141
#19 0x000003ff84649e0a in __run_exit_handlers () from /usr/lib64/libc.so.6
#20 0x000003ff84649ef8 in exit () from /usr/lib64/libc.so.6
#21 0x000002aa3ce077d2 in main ()

The problem appears to be that libldap is cleaning up OpenSSL via a "fini" handler:
#17 0x000003ff83a397de in ldap_int_tls_destroy () from /usr/lib64/libldap-2.4.so.2
#18 0x000003ff84b04ef0 in _dl_fini () at dl-fini.c:141
#19 0x000003ff84649e0a in __run_exit_handlers () from /usr/lib64/libc.so.6
#20 0x000003ff84649ef8 in exit () from /usr/lib64/libc.so.6
#21 0x000002aa3ce077d2 in main ()

But this happens after OpenSSL's "atexit" handler has already been invoked. It is a requirement that once OpenSSL has been cleaned up that you make no further calls to OpenSSL.
Since 1.1.1 is EOL there's not much that can be done there. libldap could resolve it by using the OPENSSL_INIT_NO_ATEXIT option to suppress OpenSSL's atexit cleanup functions:
https://www.openssl.org/docs/man1.1.1/man3/OPENSSL_init_crypto.html

@mattcaswell thank you for catching that ... I didn't notice libldap in the stack. We do use ldaps for authentication through sssd.
I think for now I will leave the engine disabled in the default openssl.cnf, and put a separate openssl configuration file alongside it with the engine enabled ... I can then use the OPENSSL_CONF variable to toggle between, making the engine configuration "opt-in" for processes that need to use it. That seems to be working well in local testing, and effectively avoids the issue described above.

Just a quick follow up in case it helps anyone else in the future... I found that /etc/nsswtich.conf contains the following line:
sudoers:    ldap files

Since I do not use ldap to store sudo rules, though, there is no need for us to have ldap there. Modifying the nss rule to be:
sudoers:    files

eliminates ldap from the sudo workflow, avoiding this issue.
Thanks again for the quick assistance... much appreciated


Feature/implement the forgot UI

Related issues
Resolved #5
Summary

Create template forgot password UI light-dark mode
Added new translation keys and new color codes
Added color at interface and constranst
Added library Expo LinearGradient

How to test
step

Open app/_layout.tsx
Commet  <TamaguiProvider />
Import tempalte import ForgotTemplate from '~/components/templates/PorgotPasswordTemplate'
Call <ForgotTemplate/>

Concern points
Screenshots

Fix it

@KMTus Fix conflicts


Question: What is the correct way to add request-specific properties to telemetry context?

I want to add request-specific data to the telemetry context, so that it will be included in the default AI Trace and Request log messages. This data is retrieved from the request body in my Azure Function (v2). Ideally, I want to be able to add this data to the context inside my Function, and not in a TelemetryInitializer (as that would force me to parse the request body in both the Function and the initializer).
I looked at the TelemetryClient to do this, but this one doesn't seem to have an applicable property for it:

TelemetryClient.Context.GlobalProperties is meant for global properties, and not for request-scoped properties;
TelemetryClient.Context.Properties is obsolete, and I don't see how I can use the recommended replacement ISupportProperties.Properties there.

Note: I've also opened a question on StackOverflow for this.

For anyone who ends up here in the future, if you're in AspNetCore one answer is to do this:
var requestTelemetry = context.HttpContext.Features.Get<RequestTelemetry>();
requestTelemetry.Properties["Foo"] = "Bar";

@lmolkova:

You can update request telemetry properties by adding tags on Activity.Current like
Activity.Current?.AddTag("my-prop", ExtractPropFromRequest());

Hi - is this still the case with Azure Functions on .NET 5 (isolated process)? Activity.Current seems to always be null:
[Function("Foo")]
public HttpResponseData Run(
    [HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = RoutePrefix + "foo")]
    HttpRequestData req)
{
    // `Current` is null, `AddTag` doesn't get executed.
    Activity.Current?.AddTag("Bar", "Baz");

    return req.CreateResponse();
}

Do I need to start an activity manually using TraceParent and TraceState from req.FunctionContext.TraceContext?

@lmolkova:

You can update request telemetry properties by adding tags on Activity.Current like
Activity.Current?.AddTag("my-prop", ExtractPropFromRequest());

Hi - is this still the case with Azure Functions on .NET 5 (isolated process)? Activity.Current seems to always be null:
[Function("Foo")]
public HttpResponseData Run(
    [HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = "foo")]
    HttpRequestData req)
{
    // `Current` is null, `AddTag` doesn't get executed.
    Activity.Current?.AddTag("Bar", "Baz");

    return req.CreateResponse();
}

Do I need to start an activity manually using TraceParent and TraceState from req.FunctionContext.TraceContext?

I just started a new 3.1 function app with Microsoft.NET.Sdk.Functions (3.0.13) and Microsoft.ApplicationInsights.AspNetCore (2.18.0) and Activity.Current seems to be null as well.
I also tried to use the "HttpContext"-way but I can't seem to get an HttpContext from the IHttpContextAccessor, it's always null.


added note on how to get started

I added a note on how to get started because it took me quite some time to figure out how to use the app

Thanks for the contribution. Love to hear any feedback you have. Send to<EMAIL_ADDRESS>


Proposal for initial best practices for ICRC7 metadata

To provide a standardized approach to handling metadata for NFTs within the Internet Computer ecosystem, the ICRCX metadata standard has been designed. This standard facilitates multiple methods for specifying and verifying metadata, accommodating diverse NFT use cases and ensuring interoperability with existing standards. Below is a comprehensive description of the ICRCX metadata standard's specifications:
ICRCX Metadata Standard Specification
Metadata Retrieval Methods
The ICRCX standard supports several methods for associating metadata with NFTs. Each method caters to different storage preferences and application requirements:

URI-based Metadata (IcrcX:metadata:uri):

Description: Metadata is hosted externally and accessible via a specified URI. The URI points to a JSON file containing the metadata.
Structure: #Text containing the URI.

Embedded JSON Metadata (icrcX:metadata:json):

Description: Metadata is directly embedded within the NFT as a JSON string.
Structure: #Text containing JSON data.

Direct Value Metadata (icrcX:metadata:value):

Description: Metadata is provided directly as a map within the NFT, using the Value type to support complex structures.
Structure: Value type, typically a map containing key-value pairs.

Image Only Metadata (icrcX:metadata:image):

Description: Only an image URL is provided as the NFT's metadata.
Structure: #Text containing the image URL.
or `Array[#Text(url),#Text(mimetype)]

Canister-based Metadata (icrcX:metadata:canister):

Description: Metadata is fetched from a specified Internet Computer canister.
Structure: Array containing #Text(canisterID) and #Text(path) and an optional #Text(path) format which shuld be json or candid.
This type is important because IC based smart contracts can pull these resources via http_request without an http outcall.

Optional Verification via Hash
To ensure the integrity and authenticity of the metadata, especially when it is stored externally, a hash can optionally be provided:

Metadata Hash (icrcX:metadata:hash):

Description: Accompanies any of the metadata retrieval methods to provide a verifiable hash of the metadata content.
Structure: #Blob containing the hash value.

Standard Items for icrcX:metadata:value
When using the direct value method for metadata (icrcX:metadata:value), the following standard items are recommended within the metadata map:

icrcX:name: Text — Name of the NFT.
icrcX:description: Text — Description of the NFT.
icrcX:image: Text — URL or data URL pointing to an image representing the NFT.
icrcX:preview: Text — URL or data URL pointing to a thumbnail image of the NFT.
icrcX:experience: Text — URL or data URL pointing to a dapp for interacting with the NFT.
icrcX:attributes: Array of Maps — Each attribute is a map containing properties like trait type, value, and display type.

JSON Format Compliance
When metadata is represented in JSON format (either embedded or via URI), it should adhere to widely accepted standards such as those established for ERC-721 or ERC-1155:

The JSON structure should generally omit the "icrcX:" prefix used in direct value maps.
An optional field, icrcX:metadata:format, may be used to specify the metadata standard or version being followed (e.g., "erc721", "enjin", "opensea").

Summary
This standardized approach ensures flexibility in how metadata is stored and accessed while promoting consistency in its structure and verification. It supports both on-chain and off-chain storage strategies, caters to different levels of complexity in metadata content, and facilitates interoperability with existing blockchain ecosystems through compliance with popular standards like ERC-721 and ERC-1155.

Updated in response to the 20240521 Working Group meeting:
To provide a standardized approach to handling metadata for NFTs within the Internet Computer ecosystem, the ICRC-76 metadata standard has been designed. This standard facilitates a bi-modal for specifying and verifying metadata, accommodating diverse NFT use cases, and ensuring interoperability with existing standards. The initial standards are purposely simple and MAY be extended through additional ICRC Extensions to target specific systems, standards, and protocols. Below is a comprehensive description of the ICRC-76 metadata standard's specifications:
ICRC-76 Metadata Standard Specification
Metadata Retrieval Methods
The ICRCX standard supports two methods for associating metadata with NFTs. Each method caters to different storage preferences and application requirements:

URI-based Metadata (IcrcX:metadata:uri):

Description: Metadata is hosted externally and accessible via a specified URI. The URI points to either a JSON file containing the metadata or, alternately, a data URI as specified in RFC 2397.
Structure:

#Text containing the URI.

Limits: Response from the Internet Computer are currently limited to 2MB in size. Any data of significant files may need to be held off-chain or at an IC-URI(described later).
Examples

Text Data: #Text(data:text/plain,Hello%20World)
IPFS: #Text(ipfs://bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi/dir/file.json)
JSON Data: #Text("data:application/json,{'name':'example','value':'data'}")
Image Data: #Text("data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA...")
IC-URI: #Text(icdata://2222s-4iaaa-aaaaf-ax2uq-cai/item/64/data.json) - An IC canister can retrieve this data in a trusted manner using http_request without having to resort to http-outcalls.
URL: #Text("https://foo.com/id/64/data.json") - When a URL is encountered, the metadata should be loaded from this URL.

Direct Value Metadata (icrcX:metadata:value):

Description: Metadata is provided directly as a map within the NFT, using the Value type to support complex structures.
Structure: Value type, typically a map containing key-value pairs.
Off-chain nodes:  Text values that correspond to an internet addressable URI, IC-URI, or IPFS locations should be followed to retrieve additional metadata.

//todo: Some outstanding questions on what to do about data in metadata that is Text, a URI, but NOT supposed to be traversed and included in the metadata.  (Choice one is to have a specific tag in a map for this type of data ie icrc76:no_follow or two to have a tag for followable links ie icrc76:metadata:follow).
IC-URI specification
Locations that hold data on the IC that are both addressable from the IC and from https can use the IC-URI specification as follows:
ic-http://{canister_id}/{path}?[__icrc76hash={hash}&__icrc76mime={mimetype} where

The query string parameter __icrc76hash is optional but should match the hash of the retrieved file if present.
The query string parameter __icrc76mime is optional and should have no effect but to tell the retrieving program how to treat the data.
The data should be accessible via an IC HTTP gateway and via a call the the indicated canister's http_request method.

Optional Verification via Hash
To ensure the integrity and authenticity of the metadata, especially when it is stored externally, a hash can optionally be provided:

Metadata Hash (icrcX:metadata:hash):

Description: Accompanies any data URL that needs that does not support a query string parameter of __icrc76hash.
Structure: #Blob containing the sha256 hash value.

Query String Paramater (__icrc76hash):

Description: Appended to URLs that can be retrieved. Tells the server the expected has so that it can be matched server side if desired.
Structure: `?__icrc76hash={sha256 hash} in hex representation

Standard Items for icrcX:metadata:value and/or JSON structure.
ICRC-76 does not specifically specify these structures but encourages the extension of ICRC-76 via other ICRC proposals that will allow for standardization of data useful for parsing and interoperating with other systems.
Examples:

A standard that matches the OpenSea Standard for NFTs.
A standard that matches the Enjin Standard for NFTs.
A custom standard that extends basic NFT standards with features usually only available to the Internet Computer Blockchain.

Summary
This standardized approach ensures flexibility in how metadata is stored and accessed while promoting consistency in its structure and verification. It supports both on-chain and off-chain storage strategies, caters to different levels of complexity in metadata content, and facilitates interoperability with existing blockchain ecosystems through compliance with popular standards like ERC-721 and ERC-1155.

This is already in a great shape, please find some comments in the forum to further improve it.

External metadata
ICRC-3 extension that defines a value that is a record that points to external metadata:
Map {
   icrc123_url: Text; // Path to external json
   hash?: '8o3vfuaowrgva' // Optional SHA-256 hash of external json UTF-8 string
   path?: 'a.c.[2].d' // Optional path to json value within above json
}

Standardized urls for http assets on the IC
The following url format:
ic-http://{canister_id}/{path}

Details like mime etc are not included since main purpose is to have urls that point to IC http urls without relying on a gateway in the url.

Updated in response to the 20240602 Working Group meeting:
To provide a standardized approach to handling metadata for NFTs within the Internet Computer ecosystem, the ICRC-76 metadata standard has been designed. This standard facilitates specifying and verifying metadata, accommodating diverse NFT use cases, and ensuring interoperability with existing standards. The initial standards are purposely simple and MAY be extended through additional ICRC Extensions to target specific systems, standards, and protocols. Below is a comprehensive description of the ICRC-76 metadata standard's specifications:
ICRC-76 Metadata Standard Specification
Metadata Retrieval Methods
The ICRC-76 standard supports three methods for associating metadata with NFTs. Each method caters to different storage preferences and application requirements:

Value based metadata

Description: Metadata is hosted in the NFT as a tree of Value based variants.
Structure:

The key used to specify ICRC-76 metadata is icrc76:metadata.
The Value used for Value based metadata should be a Map and that map may contain unbounded vec {Text;Value;} pairs provided the whole does not exceed the maximum return value for an Internet Computer query(currently 2MB).
The top-level Map MUST not be an ICRC-61 map(see 3.).
Uses standard Value types as defined in  ICRC-3.
Descendent Map nodes that conform to ICRC-61 should be followed by the client and the Map replaced with the remote file as described in ICRC-61.
Examples

Pure Value  [("icrc76:metadata", #Map([
      ("id","#Text("item-1"),
      ("preview", #Text("https://foo.bar.com/images/preview/item-1")
    ])
  )]

Value with ICRC-61 Replace

    [("icrc76:metadata", #Map([
        ("id","#Text("item-1"),
        ("preview", ("file", #Map[
           ("icrc61_url", #Text("https://foo.bar/data.json"),
           ("icrc61_hash, #Text("UNhYo7NpAQ61sEpPbHXH4B5C4tLwF87qNaAg17Qn8Jw=")),
         ]))
      ])
    )]

Rendered as:   [("icrc76:metadata", #Map([
       ("id","#Text("item-1"),
       ("preview", #Text("data:application/json;charset=utf-8;base64,ew0KICAiZm9vIjoiYmFyIiwNCiAgImZpenoiOiJidXp6Ig0KfQ==)"))
     ])
   )]

DataURI based metadata

Description: Metadata is hosted in the NFT as a data URI as specified in RFC 2397.

Example:
  [
    ("icrc76:metadata", #Text("data:image/json;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA...")
  ]

External based metadata

Description: Metadat is hosted outside the nft in a remote location
Structure:
The key used to specify ICRC-76 metadata is icrc76:metadata.
The Value used for Value based metadata should be an ICRC-61 standard Map that points to a valid external resource.
Example:

External
[("icrc76:metadata",  #Map[
("icrc61_url", #Text("https://foo.bar/data.json"),
("icrc61_hash, #Text("UNhYo7NpAQ61sEpPbHXH4B5C4tLwF87qNaAg17Qn8Jw=")),
)]

Rendered as:

 [("icrc76:metadata", #Text("data:application/json;charset=utf-8;base64,ew0KICAiZm9vIjoiYmFyIiwNCiAgImZpenoiOiJidXp6Ig0KfQ==)"))
 )]

Standard Items for icrc76:metadata  keys and/or JSON structure.
ICRC-76 does not specifically specify these structures but encourages the extension of ICRC-76 via other ICRC proposals that will allow for standardization of data useful for parsing and interoperating with other systems.
Examples:

A standard that matches the OpenSea Standard for NFTs.
A standard that matches the Enjin Standard for NFTs.
A custom standard that extends basic NFT standards with features usually only available to the Internet Computer Blockchain.

Summary
This standardized approach ensures flexibility in how metadata is stored and accessed while promoting consistency in its structure and verification. It supports both on-chain and off-chain storage strategies, caters to different levels of complexity in metadata content, and facilitates interoperability with existing blockchain ecosystems through compliance with popular standards like ERC-721 and ERC-1155.

Didn't get around to define the whole OpenSea inspired spec in full detail yet but here's the current definition of the spec. The spec mentions ICRC-99 as number, this is just a placeholder.

Metadata entry point
The following metadata property MUST be defined in the root of the token metadata.

Property
ICRC-3 Type
JSON Type
Description

icrc99:metadata
variant { Map }
object
Contains below metadata properties.

Metadata properties

Property
Optional
ICRC-3 Type
JSON Type
Description

external_url
Yes
variant { Text }
string
URL that allows the user to view the item on your site.

name
Yes
variant { Map }
{ [string]: string }
Key value pairs where key is a RFC 5646 language tag and description is plain text.

description
Yes
variant { Map }
{ [string]: string }
Key value pairs where key is a RFC 5646 language tag and description is Markdown.

assets
Yes
variant { Array = vec variant { Map } }
object[]
List of assets ordered by priority ascending.

attributes
Yes
variant { Array = vec variant { Map } }
object[]
List of attributes ordered by priority ascending.

Asset properties

Property
Optional
ICRC-3 Type
JSON Type
Description

url
No
variant { Text }
string
URL that returns the asset e.g. a PNG image (protocol is not limited to HTTP).

mime
Yes
variant { Text }
string
Mime type as defined in RFC 6838.

hash
Yes
variant { Blob }
string (base64)
SHA-256 hash of HTTP response body bytes returned from above url.

purpose
Yes
variant { Text }
string
Indicate purpose of asset e.g. icrc99:preview.

Purpose values
Below list of purpose values are part of the ICRC-99 standard, this list could be extended by other standards.
Purpose values can define additional properties in the attribute.

Value
Description

icrc99:image
Original size image that is shown on e.g. item details page. Additional optional width and height properties define the dimensions in number of pixels.

icrc99:preview
Small image meant as preview within e.g. a list of items. Additional optional width and height properties define the dimensions in number of pixels.

Attribute properties

Property
Optional
ICRC-3 Type
JSON Type
Description

value
No
variant { Text; Nat; Int }
string | number
Value of the trait.

trait_type
No
variant { Text }
string
Name of the trait.

display_type
Yes
variant { Text }
string
Indicate how attribute should be displayed.

Display type values
Below list of display types are part of the ICRC-99 standard, this list could be extended by other standards.
Display types can define additional properties in the attribute.

Value
Description

icrc99:property
Shows attribute as property with e.g. rarity, this is the default display type.

icrc99:date
Shows attribute as date, expects UNIX timestamp number.

icrc99:time
Shows attribute as date with time, expects UNIX timestamp number.

icrc99:rank
Shows attribute as progress rank e.g. 4 of 10, expects number and additional max_value property with a number.

icrc99:stat
Show attribute as stat e.g. 1 out of 2, expects number and additional max_value property with a number.

icrc99:boost
Shows attribute as boost e.g. +10, expect number and optional additional min_value and max_value properties. Numbers can be either positive or negative.

icrc99:boost_percentage
Same as icrc99:boost but shows values with %.


RFC: Make JsonRpc to NOT swallow exceptions

This is once again a continuation of #25.
There were several issues with the previous version.
Catching exceptions in server's code and turning them into LocalRpcExceptions probably isn't the best because:

Notifications don't require a response, hence no error will be shown
to the user. Effectively, this means any exceptions including fatal
can get swallowed.
The context of the original exception is lost in conversion.

This is a breaking change and may not be desirable for servers that use exception for control flow/recovery. But I think the spirit of the library is such that

Recoverable (user reportable) errors are encoded as LspResult's Error.
Exceptions are non-recoverable and should terminate the server.

Note: since now we're awaiting jsonRpc.Completion which completes when connection is closed, I could clean up a workaround for VSCode's incorrect shutdown sequence.

As an example, with this change I'm getting an error and a stack trace like this in lsp logs, when the server hits failwith or other kind of unrecoverable error:
Unhandled exception: System.AggregateException: One or more errors occurred. (Exception has been thrown by the target of an invocation.)
 ---> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.Exception: Planted error!
   at Marksman.Server.MarksmanServer.TextDocumentDidChange(DidChangeTextDocumentParams par) in /Users/arr/dev/marksman/Marksman/Server.fs:line 411
   at Ionide.LanguageServerProtocol.Server.Pipe #1 input at line 183@187-4.Invoke(server s, DidChangeTextDocumentParams p) in /Users/arr/dev/marksman/LanguageServerProtocol/LanguageServerProtocol.fs:line 187

Whereas before that same error would be transmitted to the client and just show the exception message?

@baronfel not quite. In this case the exception happened in textDocument/didChange and previously it ;would just get swallowed by JsonRpc because notification don't require a response.
They actually mention this here:

Well overall I'd prefer to have the information, so let's take this change :)

Coolio! 🤘 As I see it, the only questionable part in this PR is:
use jsonRpc =
       { new JsonRpc(jsonRpcHandler) with
           member this.IsFatalException(ex: Exception) = true }

In case dying on exceptions will prove to be no bueno, we could make this thing configurable.

Coolio! 🤘 As I see it, the only questionable part in this PR is:
use jsonRpc =
       { new JsonRpc(jsonRpcHandler) with
           member this.IsFatalException(ex: Exception) = true }

In case dying on exceptions will prove to be no bueno, we could make this thing configurable.

uh.. there are actually some problems with this, regarding to hanlding of LspResult.Error -- see #33


fix(deps): update github.com/ibm-cloud/bluemix-go digest to 28d386b

This PR contains the following updates:

Package
Type
Update
Change

github.com/IBM-Cloud/bluemix-go
require
digest
4d6050d -> 28d386b

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

[ ] If you want to rebase/retry this PR, click this checkbox.

This PR has been generated by Renovate Bot.

:tada: This PR is included in version 1.3.5 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


how to set the ratio for different slave accounts?

how to set the ratio for slave 1 or slave2 accounts?
for example,the main account invest 100 usd and the slave 1 1000 and slave2 50.
is it only support the same amount on the copy trade for the main accounts?

This feature is already enabled. When you create an order on the main account for $ 50, an order of about $ 500 will be created on the slave account 1, and the order will be created on the slave account 2 ~ 25 US dollars.
But be careful if you create small orders. For example, when you create a $ 10 order in the master account, the program will try to create a $ 5 order in the slave2 account, but the minimum order in the BTCUSDT trading pair is $ 10, so you will get an error and this may lead to other unexpected errors.

This feature is already enabled. When you create an order on the main account for $ 50, an order of about $ 500 will be created on the slave account 1, and the order will be created on the slave account 2 ~ 25 US dollars.
But be careful if you create small orders. For example, when you create a $ 10 order in the master account, the program will try to create a $ 5 order in the slave2 account, but the minimum order in the BTCUSDT trading pair is $ 10, so you will get an error and this may lead to other unexpected errors.

is it a auto setting base on the balance on different accounts?

Yes
On Thu, Aug 6, 2020, 13:35 gzf445<EMAIL_ADDRESS>wrote:

This feature is already enabled. When you create an order on the main
account for $ 50, an order of about $ 500 will be created on the slave
account 1, and the order will be created on the slave account 2 ~ 25 US
dollars.
But be careful if you create small orders. For example, when you create a
$ 10 order in the master account, the program will try to create a $ 5
order in the slave2 account, but the minimum order in the BTCUSDT trading
pair is $ 10, so you will get an error and this may lead to other
unexpected errors.
is it a auto setting base on the balance on different accounts?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/MohammedRashad/Crypto-Copy-Trader/issues/25#issuecomment-669850506,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AKBV6NUPLE6DZ4C555BRY2TR7KBNNANCNFSM4PWMFZEA
.

ok.thanks.


Add Ollama as a LLM option

What?
Add Ollama as an LLM option.
Why?
Ollama allows you to run an LLM model/service locally with minimal effort. This is can be especially important, for example, when demoing the project and derivative projects in a network-disconnected environment.
Implementation Considerations

Initial PR: #276


Grouping pane error if the query ends with a comment

System information:

Operating system (distribution) and version: Windows 10
DBeaver version: <IP_ADDRESS>101311042
Additional extensions: n/a

Connection specification:

Database name and version: Oracle
Driver name: ojdbc 19
Do you use tunnels or proxies (SSH, SOCKS, etc)? n/a

Describe the problem you're observing:
If the last line of the query is a comment, using the grouping panel will lead to an error
Steps to reproduce, if exist:
Run this query:
SELECT * FROM dual
-- comment

Drag the "DUMMY" column to the grouping panel; the following invalid query will be generated (closing parenthesis gets commented out):
SELECT DUMMY, COUNT(*) FROM (
SELECT * FROM dual
-- comment) src
GROUP BY DUMMY

Include any warning/errors/backtraces from the logs

Thank you for the report @p91paul

Fixed.

verified


Improvements to prometheus and Grafana

Purpose
Update grafana dashboard with Prometheus configuration entered by user. With improvements to Prometheus deployed using k8s
Remarks

PR introduces (a) breaking change(s): no
PR introduces (a) deprecation(s): no

The imported grafana instrumentation process is as follows


Update the npm repo to v1.5.3

Need to update to support the lvgl9 font

Could you please make a new tag? The latest tag is 1.5.2 and it goes back to 2021. This tag is not compatible with v9.

I can tag it to v1.6? Do you agree with that?


PointRCNN config

Dear @sshaoshuai
I would love to have your advice on the parameter tuning in the pointrcnn config. I am currently training on the JRDB Dataset  for the human/pedestrian class. I have the feeling that the parameters in the target_config for the proposal_target_layer play a huge role (especially BG_THRESH, FG_THRESH in the classification and regression branch). How did you tune these parameters?
Also looking at your uploaded pretrained models, what is the main difference between the PointRCNN and the PointRCNN-IoU model? Not many parameters have been changed in the configs but the performance on the pedestrian class of the PointRCNN-IoU model is significantly better according to your result table.
Also, I have been trying to tune the learning rate parameters for the one-cycle decay. I have been reducing the div_factor to decrease the max_lr, do you think this makes sense?
I would highly appreciate your input/thoughts. Thank you in advance.

It is hard to say how to tune these parameters, which depends on the data and specific problem.
PointRCNN-IoU just uses the IoU metric for predicting the confidence of each box. You could refer to the codes for more details.
In my experiments slightly modifying the learning rate doesn't significantly affect the final results.


How to early stop the update of branch weights as the paper said?

You can split the parameters into 2 groups in the optimizer as below:

Then you can set the learning rate of SW to zero in the middle of the training.

Can you show some results about how serious is the overfitting problem?

@JiyueWang The initial version of SW is implemented with SVD decomposition, where the lack of early stop would lead to a decrease of about 0.5 scores. The current version of SW uses newton's iteration, which naturally has stochasticity, so the overfitting is alleviated. We observe that the effect of early stop is negligible in this case.


I have found a small issue though: it's possible to insert formulas into the spreadsheet. This could end up being a little annoying and given the fact that google Sheets is so powerful, it could lead to some vulnerabilities.
Anyways, in order to prevent formulas, I have added an apostrophe (') in front of every cell except for the timestamp using the following function:
if(header !== 'timestamp') {
  return "'"+e.parameter[header]
}
inserted on line 26 (or 68 in the commented version).

Yes, good spot. This could be a problem. Maybe add in some validation preventing = in any of the fields

also in addition to the apostrophe you could replace the = sign from the string
e.parameter[header].replace(/=/g, '')