Undefined fields passed as None

If field is not specified in database _deserialize method define it as None. But that solution produce confusing moment when you declare new field in model with default value but it still takes None and raise exception. I think better ignore not specified fields, so them could be set as default or processed in validators. I maybe will do it myself later and suggest PR

I think I understand what you are saying. I can slightly change the deserialize implementation to handle explicit Nones and missing keys separated and let pydantic set the fields to default values or None. If the field doesn't have a default value, it will be set to None anyway by pydantic. If the field is require and does not have a default value, pydantic will error.

Take a look at #11 and see if that behavior makes sense to you

I mean such situation, when I don't have "order_manage_mode" field in Data Base and need set default.
Yep, #11 fix it, thanks

#11 produce new bag. Now extra fields in constructor skipped and warning not shown.

Can you provide a code example? I'm not sure what you mean by extra fields. Most of the constructor behavior is handled by pydantic. I didn't touch the constructor or serializer in the patch, so no behavior has changed there, only in the deserializer.

Ok, I check and found out that it is correct pydantic behavior


welcome to modern era article is a dead link

it 404s, that doesn't sound right

out of scope, that's up to writers, I forgot to mention that


C/C++ style Struct and union initialization MIXED UP

According to IBM Initialization of structures and unions
the C-style initialization of structures and unions should be ONLY in  C files, you CAN'T mix up using it in CPP files.

According to cppreference Struct and union initialization, I tested with CPP and C style respectively, if named as .c file, the following piece of code builds. Otherwise, if named as .cppfile, the following code failed to build.

#include <stdio.h>
typedef struct { int k; int l; int a[2]; } T;
typedef struct { int i;  T t; } S;
T x = {.l = 43, .k = 42, .a[1] = 19, .a[0] = 18 };
 // x initialized to {42, 43, {18, 19} }
int main(void)
{
    S l = { 1,          // initializes l.i to 1
           .t = x,      // initializes l.t to {42, 43, {18, 19} }
           .t.l = 41,   // changes l.t to {42, 41, {18, 19} }
           .t.a[1] = 17 // changes l.t to {42, 41, {18, 17} }
          };
    [printf](http://en.cppreference.com/w/c/io/fprintf)("l.t.k is %d\n", l.t.k); // .t = x sets l.t.k to 42 explicitly
                                    // .t.l = 41 would zero out l.t.k implicitly
}

For instance, line 148~150 of file PostProcessManager.cpp

Can you please give  me some suggestions?
Thank you

Looks there is an answer on StackOverflow ??

It's a C++20 feature supported by clang and msvc.

@romainguy
Great... it seems there is a little progress... But, I still have that isse:
Take a look at line 40~42 of FrameGraphRenderPass.h
In file included from ....../filament/src/fg/details/Resource.h:22,
                 from ....../filament/src/fg/Resource.cpp:17:
....../filament/src/fg/FrameGraphRenderPass.h:40:49: error: member ‘filament::FrameGraphId<filament::FrameGraphTexture> filament::FrameGraphRenderPass::Attachments::<unnamed union>::<unnamed struct>::depth’ with constructor not allowed in anonymous aggregate
   40 |                 FrameGraphId<FrameGraphTexture> depth;
      |                                                 ^~~~~
....../filament/src/fg/FrameGraphRenderPass.h:41:49: error: member ‘filament::FrameGraphId<filament::FrameGraphTexture> filament::FrameGraphRenderPass::Attachments::<unnamed union>::<unnamed struct>::stencil’ with constructor not allowed in anonymous aggregate
   41 |                 FrameGraphId<FrameGraphTexture> stencil;

It looks problem goes on??

For instance: Line 145~148 of file RendererUtils.cpp

RendererUtils.cpp:145:42: error: no matching function for call to ‘filament::FrameGraph::Builder::declareRenderPass(const char [18], <brace-enclosed initializer list>)’
  145 |                 builder.declareRenderPass("Color Pass Target", {
      |                 ~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~
  146 |                         .attachments = { .color = { data.color, data.output }, .depth = data.depth },
      |                         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  147 |                         .samples = config.msaa,
      |                         ~~~~~~~~~~~~~~~~~~~~~~~
  148 |                         .clearFlags = clearColorFlags | clearDepthFlags });

It's a C++20 feature supported by clang and msvc.

Yeah... After having it activated, and use CLANG, it builds...


Instructions for running your project ?

I am new into this and there are some files name main.py and others , can you give me instructions how to run and how to train your chatbot with datasets?
Apart those question i want to build a domain specific chatbot means it answer questions realted to a domain like medical shop and also if user ask something else rather than medical products then it chat normally like a chatbot , How can i do this ?
Suppose user asked :
user : I want medicine for fever.
bot : Here are available medicines for fever {medical shop api call and search medicine there}
{Now question is if i train bot for some FAQ then how call medical shop api between questions ?
Another example :
User : can you show me flavor of cold-drinks ?
Bot : Here are available cold-drinks flavours {call medical shop api for keyword search}
{now how i call api in the middle of answer if user ask something realted to medical}
So my idea is :
If user ask something realted to 'medical api or shop' ==> reply something like above examples
if user ask normal questions like 'how are you ?' , 'How  is weather?' , 'i am feeling bored','tell me a joke' something these types questions then just talk normally like a chatbot from any dialogue dataset. How can i do this ?

The docstring of main.py provides some usage examples:
main.py: Train and/or chat with a bot. (work in progress).
Typical use cases:
    1.  Train a model specified by yaml config file, located at
        path_to/my_config.yml, where paths are relative to project root:
            ./main.py --config path_to/my_config.yml
    2.  Train using mix of yaml config and cmd-line args, with
        command-line args taking precedence over any values.
            ./main.py \
                --config path_to/my_config.yml \
                --model_params "{'batch_size': 32, 'optimizer': 'RMSProp'}"
    3.  Load a pretrained model that was saved in path_to/pretrained_dir,
        which is assumed to be relative to the project root.
            ./main.py --pretrained_dir path_to/pretrained_dir

Regarding your medical shop example, do you have a dataset ready to train on? Note that sequence-to-sequence models (the ones this project is based on) require a considerable number of training examples to produce good results. Typically hundreds of thousands, or millions. That would be a pretty big FAQ! Hah. For something that domain specific, you'd be better off using something like a retrieval-based model for response ranking. Here is a nice blog post that goes over such models.
For the other part of your question, you'd need some binary classifier to determine whether to use the domain-specific medical-shop-bot or the dialogue bot. An SVM trained on (sentence, T/F) pairs where the label is either True (sentence is related to medical) or False (sentence is not related to medical -- use dialogue bot) should do the trick. There are of course many ways to approach this that are outside the scope of this repository.

Did that address your issue? I'm going to mark this as closed in a few days if no response.


Feat/fix ci publish

Codecov Report

Merging #314 (2f225a3) into main (a0f1ed5) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head 2f225a3 differs from pull request most recent head 63e4d0c. Consider uploading reports for the commit 63e4d0c to get more accurate results

@@           Coverage Diff           @@
##             main     #314   +/-   ##
=======================================
  Coverage   95.89%   95.89%
=======================================
  Files          12       12
  Lines         341      341
  Branches       50       50
=======================================
  Hits          327      327
  Misses         14       14

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a0f1ed5...63e4d0c. Read the comment docs.


Win Editor: Could not find a working python interpreter

[READ] For Firebase Unity SDK issues, please report to Firebase Unity open-source
Once you've read this section and determined that your issue is appropriate for this repository, please delete this section.
[REQUIRED] Please fill in the following fields:

Unity editor version: Unity2019.4.17f1
Firebase Unity SDK version: 9.1.0
Source you installed the SDK: .unitypackage
Problematic Firebase Component: Analytics
Other Firebase Components in use: Crashlytics
Additional SDKs you are using:
Platform you are using the Unity editor on: Windows
Platform you are targeting: iOS and Android
Scripting Runtime:  IL2CPP

[REQUIRED] Please describe the issue here:
I have error on every compiling:
"Could not find a working python interpreter. Please make sure one of the following is in your PATH: python3 python3.8 python3.7 python2.7 python2 python"
Steps to reproduce:
Update any cs script.
I saw sources. Firebase App don't use python interpritater on windows platform. Firebase App use binaries: generate_xml_from_google_services_json.exe
network_request.exe
Please remove this error message for windows platform.

It looks like you are correct - an exe should be used instead of python on windows, but it seems that the current logic is such that it is performing the check to find python even though it is not required.
As a temporary workaround until this is fixed, you can either install python locally, or try an older version of the SDK (8.8.1 or older, I think)

This error message don't affect building. But we have many team members who panic when see any errors.


Weird message behavior

I am experiencing some weird message behavior.
I am using your library since i like its API and the ability to run 20 MHZ crystals on the mcp2515 which is what I am using on my custom boards.
On the bus I have 4 cm of wire terminated on both boards with 120R resistors. The boards have common grounds.
Running the example can_write and can_read on each board respectively, I am getting only one of the two messages on the receiver side, and some times, especially when i reset the sender, I get the messages but as seen in the screenshot. Any idea why this might happen? Thanks in advance.
Imgur

Try to add a pause between two sendMessage. Maybe library trying to put frame in same output buffer

Actually I tried to change my crystal to 16MHZ (and of course the change to MCP_16MHZ) and everything worked fine. I am gonna test with another board at 20MHZ to confirm this

If you found valid configuration please add numbers to library https://github.com/autowp/arduino-mcp2515/blob/master/mcp2515.h#L124


[RFC][COMMUNITY] Nomination Org Principle

Dear TVM community
TVM is a community project, we always put community collaboration at the top of our priority, this is especially true for collaborations with folks who we do not interact daily. I the meanwhile, I do have to admit that have the natural tendency to talk more to our colleagues.
To further encourage such collaboration and more importantly recognition of community members who are not in our physical circle. We propose to adopt the principle that a PMC member will only nominate candidates from a different organization. This principle will not bar anyone with merit from being nominated(since folks will be nominated by PMC members who are not their colleague at work). It is a principle to encourage folks to interact more with people who are beyond our circles and work together with other folks in the Apache way. Of course, it is by no means a way to prevent all kind of conflict of interest, but it encourages our PMC members to look broadly in the community and bring a better community for everyone.
As you may have noticed in the past community nominations, the current PMC has already done a soft run of this principle in the past five months and it worked quite well. We want to formalize this principle and would like to hear the community’s thoughts on this matter.
Please share your thoughts.
cc @dmlc/tvm-committer

+1.
I like the idea that don't allow to nominate candidate from the same org. I believe that it would be helpful to achieve a diverse community.

+1 From oberservation of a very similar cross org DL project, every org has its own interest and trys to push project follow their best interest. This will bring deadly effect when interests are very different and there is POTENTIAL unreasonable PMC to screw up contributors from other org. This proposal is able to avoid potential painful situation we have seen in other project.

+1. From historical observation, this will make project healthier, bring more diversity and help make project fair to every org. Also it will be helpful in the merit evaluation process.

Formal voting thread here https://github.com/dmlc/tvm/issues/2717


Add [disable] button to overlord console for remote workers

There was a PR recently that added a handy [kill] button to the overlord console for specific tasks. This ask is that, in the remote worker section, there is a button added that allows an operator to disable a specific middle manager from the overlord console.

Would love to add this to the new console #6832

Resolved by #7642


set camera to continuousAutoFocus

in func resetCameraToContinuousExposureAndFocus() set to input.device.focusMode = .continuousAutoFocus

in func resetCameraToContinuousExposureAndFocus() set to input.device.focusMode = .continuousAutoFocus where it used to be set to .autoFocus.  I believe this is a simple fix for a copy-paste bug.
Description

Motivation and Context

#99
How Has This Been Tested?

Tap to focus the camera, wait 1 second, verify that continuous autofocus mode for the camera resumes successfully.
Screenshots (if appropriate):
Types of changes

[X] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[X ] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ X] I have read the CONTRIBUTING document.

Please disregard my earlier PR #100 which is wrong and should remain closed or removed. This is the correct fix for issue #99.


Please update npm @pyviz/jupyterlab_holoviews

The npm package at https://www.npmjs.com/package/@pyviz/jupyterlab_holoviews is still at 0.5.2.  It would be nice to not have to build it locally.

jupyterlab_holoviews should no longer be used please use https://www.npmjs.com/package/@pyviz/jupyterlab_pyviz instead.


Open access link broken on front-office

Bug description
Open access link broken on front-office
Steps to Reproduce

Go to https://biblio.ugent.be
Scroll to footer, click on "Open Access"
See error

Desired link
https://www.ugent.be/intranet/en/research/impact/schol-publishing

Fixed by this PR https://github.ugent.be/Universiteitsbibliotheek/biblio/pull/74


Update for Spine 4.0+

Spine version 4.0 and above have changed the curves in the JSON file. Before it was 4 variables (curves,c2,c3,c4), now there is a array called curve with 8 floats:
curve": [ 0.889, -169.87, 1.114, -108.54, 0.924, -264.13, 1.114, -373.86 ]
Spine now has x and y for translate,rotation and scale.
Can you please update the runtime so that it works with newer versions?

The 4.x version of Spine has not been released yet, and the version of the spine-runtime-cpp they provided is 4.0-beta which was updated 3 monthes ago with 3.8 version code merged. I prefer to wait for the official release version of spine-runtime. You could try to override the spine-cpp folder with the newer version of spine-runtime-cpp and build the engine.

The 4.x version of Spine has not been released yet, and the version of the spine-runtime-cpp they provided is 4.0-beta which was updated 3 monthes ago with 3.8 version code merged. I prefer to wait for the official release version of spine-runtime. You could try to override the spine-cpp folder with the newer version of spine-runtime-cpp and build the engine.

I've update it to spine version 4.0.
Check it out!


[bug] case statement update optimization failure

Describe the bug
This is going to be hard to reproduce as a test case since it relies on weird ordering in a case statement. Thus, I will describe what I'm trying to do and what code is getting generated and why I think it is failing.
What I'm trying to do is "move" a PSU (power supply) from one rig (machine box) to another rig.
The code is effectively this:
		if (oldRig.psu) {
			newRig.psu = oldRig.psu;
			newRig.psu.status = RigStatus.MOVED;
			oldRig.psu = null;
		}
		oldRig.status = RigStatus.REMOVED;

		await this.em.persistAndFlush([oldRig, newRig]);

The SQL that is generated is this:
update "Rig"
set "status"    =
        case when ("id" = '00E067229BC8') then 'REMOVED' else "status" end,
    "psu"       = case
                      when ("id" = '00E067229BC8') then NULL
                      when ("id" = '00E0671F7819') then 'PS970312'
                      else "psu" end,
    "updatedAt" = case
                      when ("id" = '00E067229BC8') then '2020-11-02T22:28:41.749Z'
                      when ("id" = '00E0671F7819') then '2020-11-02T22:28:41.749Z'
                      else "updatedAt" end
where "id" in ('00E067229BC8', '00E0671F7819')

This runs fine in my unit tests against my local postgres... I can see everything generated just fine:
    [query] update "Rig" set "status" = case when ("id" = 'RANDOMID') then 'REMOVED' else "status" end, "psu" = case when ("id" = 'RANDOMID') then NULL when ("id" = 'ID2') then 'psu:1:e52c9f25d976' else "psu" end, "updatedAt" = case when ("id" = 'RANDOMID') then '2020-11-02T23:56:10.058Z' when ("id" = 'ID2') then '2020-11-02T23:56:10.058Z' else "updatedAt" end where "id" in ('RANDOMID', 'ID2') [took 0 ms]

But in production, I get this error:
 "message": "update \"Rig\" set \"status\" = case when (\"id\" = '00E067229BC8') then 'REMOVED' else \"status\" end, \"psu\" = case when (\"id\" = '00E067229BC8') then NULL when (\"id\" = '00E0671F7819') then 'PS970312' else \"psu\" end, \"updatedAt\" = case when (\"id\" = '00E067229BC8') then '2020-11-02T23:32:32.752Z' when (\"id\" = '00E0671F7819') then '2020-11-02T23:32:32.752Z' else \"updatedAt\" end where \"id\" in ('00E067229BC8', '00E0671F7819') - duplicate key value violates unique constraint \"Rig_psu_unique\"",

The error being:
duplicate key value violates unique constraint \"Rig_psu_unique\"",
The Mikro generated DDL for Rig.psu looks like this:
    psu         varchar(255)
        constraint "Rig_psu_unique"
            unique
        constraint rig_psu_foreign
            references "Psu"
            on update cascade on delete set null

The Rig.psu entity definition looks like this (it hasn't changed since Mikro 3.x):
	@OneToOne(() => Psu, (psu) => psu.rig, { owner: true, nullable: true })
	@Field(() => Psu, { nullable: true })
	psu?: Psu | null;

Expected behavior
No error of course. =) This also ran just fine in Mikro 3.x, so I suspect that something is changed in 4.x... namely generating these case statements.
Workaround
I can probably change this to run the two modifications separately in a transaction.
Additional context
I suspect that the PG query planner doesn't know how to deal with the fact that I'm going to remove something later.
Versions

Dependency
Version

typescript
4.0.5

mikro-orm
4.2.3

pgsql
8.2.1

Ah, I see you did some big change for case statements to optimize things. Well, this is one 'case' where it won't work. I need some way to turn this off.

Interesting... Anyway, you can disable batch updates (globally) via useBatchUpdates: false.

This is pretty much the same issue:
https://stackoverflow.com/questions/5403437/atomic-multi-row-update-with-a-unique-constraint
In pg we could use SET CONSTRAINTS ALL DEFERRED. The fact that we are batch updating 1:1 property can be detected easily, so it would be there only when needed.
But looks like mysql does not support that 🤷
https://stackoverflow.com/questions/5014700/in-mysql-can-i-defer-referential-integrity-checks-until-commit
Or we could split it into 2 queries automatically for sure :]

The solution in 4.3.0 was bad (let's say too aggressive) and had some side effects, I took a bit different approach that should hopefully be applied only in the right time. Will be shipping 4.3.1 soon, would be great if you could verify it still works fine with latest dev - 4.3.1-dev.9.


Can this library easily work for non-KMP projects?

I am trying to use this SpriteView in a non-KMP project but am stuck in SpriteSheet that demands an image: DrawableResource parameter that I cannot provide in non-KMP project structure (AFAIK).

This project is made primarily for KMP projects. However you can fork this repository and modify it at your convenience if you're not planning to use KMP at all.

Ok, thanks. On the surface, it seems like making an alternate constructor for SpriteSheet() that takes an R.drawable.* might work. It doesn't seem like KMP is a fundamental part of this library (althought it is nice that it drops right into a KMP project structure).

I'll take a look in the next few days. This should be left here open for the time being though. Thanks for the suggestion.

Try something like this to convert your Android's resource id into ImageBitmap. This was generated by ChatGpt, I haven't tried it though:
@Composable
fun loadImageResource(resourceId: Int): ImageBitmap {
    val context = LocalContext.current
    val drawable = AppCompatResources.getDrawable(context, resourceId)
    return drawable?.toBitmap()?.asImageBitmap() ?: error("Resource not found")
}

Thanks! Maybe I am missing something, but the SpriteSpec() I am trying to use requires a image = SpriteSheet( image:DrawableResource.,...)


Chatbot not responding - Facebook Messenger message.user is not correct

I was testing my platform and then I decided to delete the conversation that I had with my chatbot but after deleting the conversation I tried to access my information using message.user  but found out later that it was not the same.
So what I happens is if I delete conversation new id is being given which I can't even use with the facebook profile API, the profile API is returning the error saying the user does not exist.
The following link  is the one I use to get other information concerning the user :

"https://graph.facebook.com/v2.6/" + this.fbId + "?fields=first_name,last_name,email,locale,profile_pic&access_token=" + process.env.page_token;

And the chatbot is no longer responding to me (Deleted conversation), it is giving the error :

An error occurred while sending a message:  { message: '(#100) No matching user found',

How can I fix this problem that problem of message.user returning different id ?

Botkit version: 7
Messaging Platform: Messenger
Node version:  8
Os: Centos

I have finally find the issue. It was the page token was not correct. I have fixed.


Update testing setup for python 3.12 and django 5.0

Update python, remove redis, update mariadb

 Upgrade version of Django to current version (LTS if avaiable)


Support customization of Open Graph Metadata

Is your feature request related to a problem? Please describe.
Not a problem; desired functionality.
Describe the solution you'd like
Ability to customize applicable Open Graph Metadata, namely to enable nice link sharing in Apple iMessages (e.g. to add an image to a link). See documentation.
Describe alternatives you've considered
None
Additional context
None

That seems to track with how you've set up customization thus far, although I do not have the knowledge to consider pitfalls of that approach, if any. Thanks for being active and taking user feedback.

I mean OG tags are already working. Try it out with one of Blowfish's page - or my personal homepage. So I was assuming that you want article level control over OG tags. Did I get it right?

So I was assuming that you want article level control over OG tags. Did I get it right?

If this is what you, @c-elliott96, want, you can look up the documentation of hugo on open graph here. They describe how to change the open graph data for each file in your content directory.

Ah, shame on me for not realizing that before creating an issue! Thank you both, @nunocoracao and @arunoruto for your help. Closing, as it looks like my request already exists.


[question] 🤔 can we use VS Code user code-snippets?

I created a bulk of code-snippets when I was using vscode, the path is: ~/Library/Application Support/Code/User/snippets/*.code-snippets
So, can I load these code-snippets via this plugin? I've read the doc, but I didn't see any guide for this(maybe I missed).
here is the vscode snippet template:
{
	// Place your global snippets here. Each snippet is defined under a snippet name and has a scope, prefix, body and
	// description. Add comma separated ids of the languages where the snippet is applicable in the scope field. If scope
	// is left empty or omitted, the snippet gets applied to all languages. The prefix is what is
	// used to trigger the snippet and the body will be expanded and inserted. Possible variables are:
	// $1, $2 for tab stops, $0 for the final cursor position, and ${1:label}, ${2:another} for placeholders.
	// Placeholders with the same ids are connected.
	// Example:
	// "Print to console": {
	// 	"scope": "javascript,typescript",
	// 	"prefix": "log",
	// 	"body": [
	// 		"console.log('$1');",
	// 		"$2"
	// 	],
	// 	"description": "Log output to console"
	// }
}

Yeah, do you have the package.json accompanying them? Pass the path to the directory it's in to luasnip.loaders.from_vscode.load (see readme, or search for it in DOC.md), that should do it

OK, now works according to the doc.

It seems can not work like this:
~/.config/nvim/my-snippets/package.json
~/Library/Application Support/Code/User/snippets/dropcode.code-snippets

-- luasnip.lua
require("luasnip.loaders.from_vscode").lazy_load({
  paths = {
    "./my-snippets", 👈
  }
})

// my-snippets/package.json
{
  "name": "my-snippets",
  "contributes": {
    "snippets": [
      {
        "language": [
          "all"
        ],
        "path": "~/Library/Application Support/Code/User/snippets/dropcode.code-snippets" 👈
      }
    ]
  }
}

@2nthony I've two questions:

Are scope fields explicitly supported?
Does the "package.json#language" field affects the scope field when snippets are loaded into luasnip engine?

Please, give answers in docs, thanks!

@hinell

I didn't see it works.(if I do scope in javascript, it will still have the snippet in vue file)
not sure

We don't support .code-snippets files as of now (mostly because I don't use vscode so I forgot (or didn't know about) them. But adding them should be quite easy, a PR will be appreciated :smiley:

Supported actually by adding ~/Library/Application Support/Code/User/snippets/package.json and do some figure.

The problem is that I have snippets for across different languages: javascript,typescript,javascriptreact,typescriptreact in one place that I would like to import into luasnip.
@leiserfg If I have time on hands!


Manage local models through application

Solving #45.
Idea is to:

[x] Add a table to the settings dialog to see & delete local models

[x] only the ones in .config/translateLocally can be deleted. Do not attempt to delete models found in pwd.
[x] Make delete option prominent so the intention of that table becomes immediately clear
I'm thinking adding a column with a delete button per row, but might end up with a button at the bottom of the table that is only enabled if deletable modes are selected. Like how the context menu works. Because of space.

[x] Add an import button to load (copy) local models into the local "library" (the .config/translateLocally folder)

[x] validate before importing (check whether there is a model_json file?)
partially done, model is not imported but extracted files stay

This won't affect:

Reading local models from the start-up directory. That should keep working. Those models should not be imported into the local library.

I compiled and tried.

We should do something about auto-resizing the widget when switching to the languages tab, as the user needs to resize things to see the full name.
We should give the user the information why some models are undeleteable (eg, they were distributed with the exe, and the user can just remove them from the current directory)

In terms of validation, we this is already done by this function:
https://github.com/XapaJIaMnu/translateLocally/blob/db88f0d571055ae7750a72f35413575e2dad6f6f/src/ModelManager.cpp#L144
Difference being, it takes a parent folder. Maybe some of it can be abstracted to work on a specific model file. (OR you know use this function: https://github.com/XapaJIaMnu/translateLocally/blob/db88f0d571055ae7750a72f35413575e2dad6f6f/src/ModelManager.cpp#L37 )
As a side note, we probably should explain in more details the change dir shenanigans that go on in here: https://github.com/XapaJIaMnu/translateLocally/blob/db88f0d571055ae7750a72f35413575e2dad6f6f/src/ModelManager.cpp#L189

A problem with the current approach to models is that we really trust the .tar.gz files.
We assume

that the name of the .tar.gz file is also the name of the folder it contains
that it contains a single folder with all the model files
that no two different models shall have the same folder name
that it doesn't contain any symlinks or paths with ".." in it (I've seen that libarchive has some flags to guarantee this, but we don't explicitly set them)

Additionally, right now if you upgrade a model it does so by extracting over the current model. This could leave files from the old model that haven't been overwritten lying around. It also corrupts the old model when the updated model fails to extract completely.
I think we could solve all of the above if:

We always extract models to a unique folder (say like UUID based folder names)
If all entries in the .tar.gz indeed have the same prefix folder, we substitute that for the unique folder. Otherwise we extract all files as is to the unique folder (so models can still have subdirectories if they need to)
If extraction fails half-way through, or the model_info.json file is not found, we can just delete the unique folder
If it does extract & loads correctly, and it is the same model (same long name?) but a newer version of an already installed model, we can delete the old model completely by just removing that unique folder. It won't affect the updated model.

Additionally we might want to add checksum to the model index at some point? Then we can just point to models on S3 or 3rd party sites and validate the .tar.gz files before extracting them. Won't solve the above issues for locally imported models (e.g. by using the "Import models…" button I added.

I'm all for md5 checksums being distributed alongside the tar.gz possibly inside the models.json which we fetch from the server. Then we can download and compute the md5sum.
Can we not do opaque folder names? We could add a timestamp to them, I guess. I see little benefit of it though, we could just delete the previous folder before extracting?

md5 is broken.  Let's do something more secure.

Any of these would be fine with me: https://doc.qt.io/qt-5/qcryptographichash.html#Algorithm-enum

Can we not do opaque folder names?

I don't trust the archives to always have a folder with a unique name in them, or a consistent name across upgrades. Like, that would be okay if it was just a single source for models. But I just added a button to import your own .tar.gz files, opening it up for all kinds of shenanigans.

we could just delete the previous folder before extracting?

I'm thinking of the case extraction might fail, or it might be an incomplete archive. Then we also just deleted the old working version.

We could add a timestamp to
Them

That would be sufficient to get around the above failure case

We are the single source models for now, but yeah, we might not be in the future. I don't see users being too keen to prepare their own models, but hopefully that will change. And when they do, they would expect models with the same folder names to override the previous model they tried to import. I guess confirmation for "Folder X, exists, replace?" would work here.
We also have to teach people how to prepare their model_info.json, I guess.
I'm not sure adding an extra timestamp to the folder name is absolutely necessary. It is defnitely not harmful.

Delete models was somewhat buggy for me. I had 5 models and I deleted them one by one and at first, multiple models disappeared at once, then later on I kept having 1 model left, but as I deleted it, another one appeared. Could it because I had the .tar.gz files inside the directory, as per the behaviour of the master branch?
The initial model manager window is a bit tiny, especially if one has many models installed.

Same run, when I try to download a new model after deleting everything:

Oooh interesting! Why would it not be able to move the folder? Hmm… I'll experiment a bit more.
I'll also look into the weird deleting/popping back models. Probably a signalling issue in ModelManager. Making the QAbstractTableModel emit the right signals is a bit tricky.

Same issue, if I try to import a model. I assume it works for you on mac?

Yeah, worked fine for me. Not sure why it doesn't work for linux. Does the ~/.config/translateLocally path exist when you encounter this error?

Yes, it does exist. It is created automatically on the first run if it doesn't exist.

I have no clue why it would not just work. QDir::rename() and QTemporaryDir both don't do anything magical. I'll try it in my Ubuntu VM.

I haven't been able to replicate the rename() failure. What OS are trying this on (looks like something with KDE)
Do you have any special setup around your temp and home directory? Like, could there be any reason why a directory in /tmp could not be moved to ~/.config/translateLocally? Different filesystem maybe?
Lastly, could you try the commented out line in ModelManger.cpp that creates a temp directory in the translateLocally directory directly, and see if that solves it? I.e. change ModelManager.cpp:91 to:
// Initially extract to to a temporary directory. Will delete its contents
// when it goes out of scope
QTemporaryDir tempDir(configDir_.filePath(".extracting-XXXXXXX"));
// QTemporaryDir tempDir;

I'll take a look over the weekend. I suspect it might be an issue with the working directory not being clean due to it being created by a previous version of translateLocally.
I use arch Linux

I think i found the issue. Across SO:

...
Do remember that even with giving a full path QDir::rename will fail if source and destination path's are not on the same partition or for a few more reasons as mentioned in the documentation.
...
My tmp is not on the same partition.

This will potentially be a problem on a number of systems that use different drive configurations. With some brief googling, it seems everyone recommends the use of QDir::rename which will fail in my case. Not sure what's the next best thing to do. Maintain our own temp in the configuration dir? Use C++17 filesystem?

I think the safest work-around is just creating a temp directory inside the config dir. This also saves us from creating temp files and then ending up trying to move them to a filesystem that's full or has a quota on it that causes the move to fail.
I've explicitly excluded any of the temp directories from being scanned. Not that they should still exist at that point, but who knows what weird crashes or bugs may cause them to survive their pruning. I thought about just making them hidden directories and excluding them that way from the scan in scanForModels(), but I'm not sure that works on Windows.

Yes, this finally works for me.
How about deleting any extracting-XXXX directories when the program is run? That should do a proper cleanup in case we had a crash during model extraction on a previous run.

Or is this done automatically by the temporary dir function?

Yes. the QTemporaryDir will delete the directory it made (recursively) when it's deallocated. It's only when the application segfaults (e.g. somewhere in libarchive) that the extracting-XXXX folder would stay.
I'm not a fan of the idea of deleting these folders on start-up. I think Linux and Windows there's nothing preventing you from starting a second instance of the program. If you would do that exactly while another instance is extracting, it would break that process.

Fair point, merging.


#7 Fix UI blink for passive key

the fix, proposed in https://github.com/hCaptcha/react-native-hcaptcha/issues/7#issuecomment-996946369 works
I have verified the fix with 0.63.4

Issues

Yesterday I faced with JCenter down issue https://status.gradle.com/
Example contains RangeError: Maximum call stack size exceeded. loop between onMessage -> hide -> onMessage -> ... (screenshot attached). But this error is invisible in production mode only during debug

Open question

Should I continue the investigation on 4
I still have some issues with the 0.66.1 version, @e271828- could you please update your local example to confirm this issue, also I need to know if this important
Is there a reason why https://github.com/react-native-modal/react-native-modal used instead import { Modal } from "react-native";?
I think it makes sense to do some integration testing with react-native init && yarn add file:.. just to check that compilation works

the fix, proposed in Webview fades on / off for passive sitekey #7 (comment) works
I have verified the fix with 0.63.4

Issues

Yesterday I faced with JCenter down issue https://status.gradle.com/
Example contains RangeError: Maximum call stack size exceeded. loop between onMessage -> hide -> onMessage -> ... (screenshot attached). But this error is invisible in production mode only during debug

Open question

Should I continue the investigation on 4

Yep.

I still have some issues with the 0.66.1 version, @e271828- could you please update your local example to confirm this issue, also I need to know if this important

Please try to fix for yourself locally, will take a look if that fails.

Is there a reason why https://github.com/react-native-modal/react-native-modal used instead import { Modal } from "react-native";?

Probably for some convenience API like dismiss-on-background-click, don't recall for sure.

I think it makes sense to do some integration testing with react-native init && yarn add file:.. just to check that compilation works

+1

I think this requires a switch like "passive=true" (quick/ugly fix) or a different approach, e.g. unhide modal if visible challenge is presented, as determined by open callback firing.

I wanted to ask the same) BTW there is no way to check if a key is passive (maybe it have some prefix or specific pattern)?

I think this requires a switch like "passive=true" (quick/ugly fix) or a different approach, e.g. unhide modal if visible challenge is presented, as determined by open callback firing.

I wanted to ask the same) BTW there is no way to check if a key is passive (maybe it have some prefix or specific pattern)?

There is no way, and some will be mixed. The open callback is your only option there.

I think this requires a switch like "passive=true" (quick/ugly fix) or a different approach, e.g. unhide modal if visible challenge is presented, as determined by open callback firing.

I wanted to ask the same) BTW there is no way to check if a key is passive (maybe it have some prefix or specific pattern)?

There is no way, and some will be mixed. The open callback is your only option there.

Ok I will add passiveSiteKey option which will be false by default, but if true passed it will apply these changes

Done


Simplify archived? and orphaned? to not use associations

In doing so, the TreeBuilderVmsAndTemplates no longer needs
to preload the EMS.
https://bugzilla.redhat.com/show_bug.cgi?id=1281999
@kbrock Please review.

Fixed failed spec.

:+1:

This saves us 1.6% on vm show page with 13k vms


error: no matching package named atty found (required by cargo)

`ERROR: cargo-native-0.25.0-r0 do_compile: Function failed: do_compile (log file is located at /home/user/balenaOS/rpi/build/tmp/work/x86_64-linux/cargo-native/0.25.0-r0/temp/log.do_compile.69181)
ERROR: Logfile of failure stored in: /home/user/balenaOS/rpi/build/tmp/work/x86_64-linux/cargo-native/0.25.0-r0/temp/log.do_compile.69181
Log data follows:
| Report bugs to<EMAIL_ADDRESS>| NOTE: rustc =
| NOTE: /home/user/balenaOS/rpi/build/tmp/work/x86_64-linux/cargo-native/0.25.0-r0/cargo-0.24.0-x86_64-unknown-linux-gnu/cargo/bin/cargo build -v --target x86_64-linux --release
| error: no matching package named atty found (required by cargo)
| location searched: registry https://github.com/rust-lang/crates.io-index
| version required: = 0.2.11
| WARNING: /home/user/balenaOS/rpi/build/tmp/work/x86_64-linux/cargo-native/0.25.0-r0/temp/run.do_compile.69181:1 exit 101 from '"/home/user/balenaOS/rpi/build/tmp/work/x86_64-linux/cargo-native/0.25.0-r0/cargo-0.24.0-x86_64-unknown-linux-gnu/cargo/bin/cargo" build -v --target x86_64-linux --release "$@"'
| ERROR: Function failed: do_compile (log file is located at /home/user/balenaOS/rpi/build/tmp/work/x86_64-linux/cargo-native/0.25.0-r0/temp/log.do_compile.69181)
`

@thufuxu what did you do to resolve this problem?
I have exact the same

@siredmar it’s because of network issue usually. I setup a mirror for crates, then solved.

What do you mean exactly by saying network issue?
And did you change the crate URIs from the recepies to your mirror?

network issue means that your network may not be able to download the crates listed here. You can try downloading again or setup proxy or setup a mirror of the crates.io.

The system was able to download the crates as they were present in the download section of bitbake. I tried several times, also forcing another version of atty (e.g. 0.2.11 instead of 0.2.6)

@siredmar so did they download work after all?

@floion , yes they did.
$ ls /opt/projects/balena-raspberrypi/build/tmp/work/x86_64-linux/cargo-native/0.25.0-r0/cargo_home/bitbake/atty-0.2.6
appveyor.yml  Cargo.toml  Cargo.toml.orig  CHANGELOG.md  examples  LICENSE  README.md  rustfmt.toml  src

Also the crate used in layers/meta-rust/recipes-devtools/cargo/cargo_0.25.0.bb is atty-0.2.6

SRC_URI += " \
...
crate://crates.io/atty/0.2.6 \

I suggest you try a clean build for yourself and see what happens.
I cloned the repo, pulled submodules and called
$ ./balena-yocto-scripts/build/barys -m raspberrypi3

Kind of pinning down the problem here.
I found out that atty is not the problem here. No package can be fetched with cargo. I tried that by using the …/cargo-native/0.25.0-r0/temp/run.do_compile scripts and modifying it to build a minimalist example using only one dependency.
What i also found out was that the cargo_home is not
balena-raspberrypi/build/tmp/work/x86_64-linux/cargo-native/0.25.0-r0/cargo_home/

but
balena-raspberrypi/build/tmp/work/x86_64-linux/cargo-native/0.25.0-r0/git/cargo_home/

Inside is a config file that contains:
...
[source.bitbake]
directory = "/opt/projects/balena-raspberrypi/build/tmp/work/x86_64-linux/cargo-native/0.25.0-r0/git/cargo_home/bitbake"
[source.crates-io]
replace-with = "bitbake"
local-registry = "/nonexistant"
...

This file comes from the meta-rust layer (https://github.com/meta-rust/meta-rust/blob/f2f17c58b00d12e795fc9c5f66c1bb82759bf72f/classes/cargo_common.bbclass#L46)
This change came in in this commit https://github.com/meta-rust/meta-rust/commit/48e2a7db2588d55ee5c43cf5242ad41aa16a2c1c#diff-86fe95c88946300256cccb34400231e7
However commenting those lines out downloads the crates in the ../git/cargo_home
@thufuxu you said you mirrored the crate server. Did you came accross anything i found out here?

Update: I succeeded in building the balena-raspberry image (not tested if it actually boots up) with the current master on meta-rust layer (https://github.com/meta-rust/meta-rust/tree/0621067c560f6e3e1a4c4c08ac173393bbafad5c) and removing those files as the version of cargo is now different:
balena-raspberrypi/layers/meta-resin/meta-resin-sumo/recipes-devtools/cargo/cargo_0.25.0.bbappend

balena-raspberrypi/layers/meta-resin/meta-resin-sumo/recipes-devtools/cargo/files/Cargo.lock file

I just tried on my development machine and was able to have it finish without this error for the same machine as you did:

./balena-yocto-scripts/build/barys -m raspberrypi2

[000000004][LOG]Run build for raspberrypi2: MACHINE=raspberrypi2 bitbake resin-image
[000000004][LOG]This might take a while ...
Parsing recipes: 100% |#############################################################################################################################################################################| Time: 0:00:35
Parsing of 2152 .bb files complete (0 cached, 2152 parsed). 3049 targets, 314 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies
Build Configuration:
BB_VERSION           = "1.38.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "ubuntu-16.04"
TARGET_SYS           = "arm-poky-linux-gnueabi"
MACHINE              = "raspberrypi2"
DISTRO               = "balena-os"
DISTRO_VERSION       = "2.30.0"
TUNE_FEATURES        = "arm armv7ve vfp thumb neon vfpv4 callconvention-hard cortexa7"
TARGET_FPU           = "hard"
meta
meta-poky            = "HEAD:24a833f40d837407baa4e8192f372d998a776b6c"
meta-oe
meta-filesystems
meta-networking
meta-python          = "HEAD:be79b8b111a968efdbe5e1482d0c246d0b24763e"
meta-raspberrypi     = "HEAD:2d40b000021bc8a9ef7f329ed0ad410f8d227b97"
meta-resin-common
meta-resin-sumo      = "HEAD:1aba4a4a644f74a80636b98e70407c06c8db6f33"
meta-resin-raspberrypi = "master:5ec5540bbc3d67b86f9b0c8a58da3939c9bac700"
meta-rust            = "HEAD:3efa0f069ffe4dfcb3531ea89834e69cd2257157"
NOTE: Fetching uninative binary shim from http://downloads.yoctoproject.org/releases/uninative/2.2/x86_64-nativesdk-libc.tar.bz2;sha256sum=e3b77208169bf1ac4e89496f3cdbf27695f5b18a2694a908a793390f28b67f83
Initialising tasks: 100% |##########################################################################################################################################################################| Time: 0:00:03
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
WARNING: bzip2-native-1.0.6-r5 do_fetch: Checksum mismatch for local file /home/floion/builds/teste_headers/balena-raspberrypi/build/downloads/bzip2-1.0.6.tar.gz
Cleaning and trying again.
WARNING: bzip2-native-1.0.6-r5 do_fetch: Renaming /home/floion/builds/teste_headers/balena-raspberrypi/build/downloads/bzip2-1.0.6.tar.gz to /home/floion/builds/teste_headers/balena-raspberrypi/build/downloads/bzip2-1.0.6.tar.gz_bad-checksum_f4b5249c17053f5c4c85d870feef4f48
WARNING: bzip2-native-1.0.6-r5 do_fetch: Checksum failure encountered with download of http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz - will attempt other sources if available
WARNING: expat-native-2.2.5-r0 do_fetch: Failed to fetch URL http://downloads.sourceforge.net/expat/expat-2.2.5.tar.bz2, attempting MIRRORS if available
WARNING: freetype-2.9-r0 do_fetch: Failed to fetch URL http://downloads.sourceforge.net/freetype/freetype-2.9.tar.bz2, attempting MIRRORS if available
WARNING: libusb-compat-1_0.1.5-r0 do_fetch: Failed to fetch URL http://downloads.sourceforge.net/libusb/libusb-compat-0.1.5.tar.bz2, attempting MIRRORS if available
WARNING: tcl-native-8.6.8-r0 do_fetch: Failed to fetch URL http://downloads.sourceforge.net/tcl/tcl8.6.8-src.tar.gz, attempting MIRRORS if available
WARNING: lsof-4.89-r0 do_fetch: Failed to fetch URL http://www.mirrorservice.org/sites/lsof.itap.purdue.edu/pub/tools/unix/lsof/lsof_4.89.tar.bz2, attempting MIRRORS if available
WARNING: linux-raspberrypi-1_4.14.98+gitAUTOINC+5d63a4595d-r0 do_kernel_resin_checkconfig: Checking for CONFIG_IP_NF_TARGET_LOG=m in the kernel configs failed for misc.
NOTE: Tasks Summary: Attempted 3827 tasks of which 10 didn't need to be rerun and all succeeded.
Summary: There were 9 WARNING messages shown.
[000011361][LOG]Build for raspberrypi2 suceeded.
[000011361][LOG]If build for raspberrypi2 succeeded, final image should have been generated here:
[000011361][LOG]   build/tmp/deploy/images/raspberrypi2/resin-image-raspberrypi2.resinos-img
[000011361][LOG]Done.

@floion now i am completely confused. Did you clone the repo freshly with all its submodules?
What DNS server did you configure in your host?
What OS are you building with?
I am running Ubuntu 18.04 - and getting this warning
WARNING: Host distribution "ubuntu-18.04" has not been validated with this version of the build system; you may possibly experience unexpected failures. It is recommended that you use a tested distribution.
My primary DNS Server is google <IP_ADDRESS>
Network connection is stable and not running through a proxy or VPN.
What does your balena-raspberrypi/build/tmp/work/x86_64-linux/cargo-native/0.25.0-r0/git/cargo_home/config look like?

The build was done in a fresh clone (with git clone --recursive)

cat /etc/resolv.conf

nameserver <IP_ADDRESS>
nameserver <IP_ADDRESS>
nameserver <IP_ADDRESS>
nameserver 2a01:4f8:0:a102::add:9999
nameserver 2a01:4f8:0:a0a1::add:1010
nameserver 2a01:4f8:0:a111::add:9898

I built on a 16.04.2 LTS (Xenial Xerus).

cat build/tmp/work/x86_64-linux/cargo-native/0.25.0-r0/git/cargo_home/config

paths = [
]
[source.bitbake]
directory = "/home/floion/builds/teste_headers/balena-raspberrypi/build/tmp/work/x86_64-linux/cargo-native/0.25.0-r0/git/cargo_home/bitbake"
[target.x86_64-linux]
linker = '/home/floion/builds/teste_headers/balena-raspberrypi/build/tmp/work/x86_64-linux/cargo-native/0.25.0-r0/wrapper/target-rust-ccld'

Your cargo_home/config file does look different. Mine does have the section in it:
[source.crates-io]
replace-with = "bitbake"
local-registry = "/nonexistant"

Do you have any idea why? Could it be because i am building with Ubuntu 18.04?

Can't say I know why it looks different for you.
Would you have a 16.04 to build on? @spanceac did you manage to get the build going on 18.04?

@floion i did a clean build on a new setup vanilla Ubuntu 16.04.06 and it was running without issues.

Great. So I guess 18.04 will need to get official support from Yocto until this can be fixed on that OS version


Check for null child node/Ids

I've been seeing the following error intermittently when running puppeteer-dart, which is causing test run failures.
Null check operator used on a null value
package:puppeteer/src/page/accessibility.dart 548:53  _AXNode.createTree
package:puppeteer/src/page/accessibility.dart 69:31   Accessibility.snapshot

I was able to recreate this scenario with the following code:
class AXNodeId {
  final String value;
  AXNodeId(this.value);

  factory AXNodeId.fromJson(String value) => AXNodeId(value);
}

class AXNodeData {
  final List<AXNodeId>? childIds;

  AXNodeData({this.childIds});

  factory AXNodeData.fromJson(Map<String, dynamic> json) {
    return AXNodeData(
        childIds: json.containsKey('childIds')
            ? (json['childIds'] as List)
            .map((e) => AXNodeId.fromJson(e as String))
            .toList()
            : null
    );
  }
}

class _AXNode {
  final AXNodeData _payload;
  final _children = <_AXNode>[];
  _AXNode(this._payload);
}

void main() {
  final nodeData = AXNodeData.fromJson({'childIds':['123']});
  var nodeById = <String, _AXNode>{};
  final axNode = _AXNode(nodeData);

  for(var childId in axNode._payload.childIds!){
    axNode._children.add(nodeById[childId.value]!);
  }
}

The issue appears to be related to the use of the null assertion operator (!) on a value that can in fact be null in this case.  From what I can tell nodeById does not contain the childId.value in the map, thus resulting in null being returned.  This appears to indicate that the child node is missing? from the FullAXTree(removed during processing?) or we've been given a bad id.  Let me know if the FullAXTree missing a node is cause for concern.
My proposal is to explicitly check for null after retrieving the value, prior to adding it to the list of children.  I also updated the node._payload.childIds! to prevent the nesting from getting any deeper.
Overall it feels like the null assertion operator ! should be used sparingly, as the analyzer simply shuts off, even if you didn't check for null first.
@xvrh for your review

Thanks for the contribution!


How to run for entire project

Well I still don't get it.
I have followed the how to do
  "scripts": {
    "lint-staged": "lint-staged",
    "precommit": "lint-staged"
  },
  "lint-staged": {
    "*.{js,ts,tsx}": [
      "standard --fix",
      "tslint --fix",
      "git add",
      "jest --bail"
    ]
  },

... and it's quite working. What I still missing is the tests
 ✔ Running tasks for *.{js,ts,tsx}
[0.1.11 7a1daa8] testing
 1 file changed, 4 insertions(+)

it says okay, but not all tests have ran. I don't want to run only for the committed files, I want run all.
What am I missing here?

lint-staged deals with staged files. If you want to test all files, change your pre-commit hook command to lint-staged && jest --bail and remove jest --bail from lint-staged commands

I don't know what I was thinking.
Thank you.


Tableview widget janks execessively with >250k rows

Is there an existing issue for this?

[X] I have searched the existing issues
[X] I have read the guide to filing a bug

Steps to reproduce

Update the example app in the 2d-scrollables package to start with 1.5 million rows by editing int _rowCount = 1500000;
Run the app flutter run -d mac --release
Click the "Jump to bottom"
Attempt to scroll

Expected results
Scroll without janking
Actual results
Getting ~1 frame every 1-2 seconds
Code sample
Update the example app in the 2d-scrollables package to start with 1.5 million rows by editing int _rowCount = 1500000;
Screenshots or Video

Screenshots / Video demonstration
https://github.com/flutter/flutter/assets/22898670/43e49aba-9c4d-4709-97a9-4253d27a7620

Logs
No response
Flutter Doctor output
Doctor output
[✓] Flutter (Channel stable, 3.13.9, on macOS 14.1 23B74 darwin-arm64, locale
    en-US)
[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.0)
[!] Xcode - develop for iOS and macOS (Xcode 15.0.1)
    ✗ Unable to get list of installed Simulator runtimes.
[✓] Chrome - develop for the web
[✓] Android Studio (version 2021.2)
[✓] VS Code (version 1.84.0)
[✓] Connected device (2 available)
[✓] Network resources

Love the package thank you!
I was hopeful that supporting large sets of rows would come with lazy loading nature of the package, as Android or iOS virtualizing views do. However, scrolling performance degrades in the lower row in the table.

+1 this is a great package!
I notice that when you scroll from the bottom, _reifyChildren appears to get called on every animation frame.  It has this implementation:
for (int minorIndex = _leadingYIndex!; minorIndex <= _trailingYIndex!; minorIndex++) {
  // Major
  for (int majorIndex = _leadingXIndex!; majorIndex <= _trailingXIndex!; majorIndex++) {
    final ChildVicinity vicinity = ChildVicinity(xIndex: majorIndex, yIndex: minorIndex);
    previousChild = _completeChildParentData(
      vicinity,
      previousChild: previousChild,
    ) ?? previousChild;
  }
}

If you're at the bottom of a 1.5 million row table, _leadingYIndex is zero, _trailingYIndex is 1,500,000 and the leading and trailing X values is 0 and 5. (for my table)
This means that while scrolling, 100s of millions of iterations of these loops is firing, and it's not really usable, unfortunately.

Thanks for the report. Reproduced this issue on Android and macOS platforms in profile mode.
After clicking on "Jump to bottom" button, there are a lot of janks:

Janks observation demo video (macOS)
https://github.com/flutter/flutter/assets/104349824/5e6b6b63-5f9d-450b-b0f8-64884269e78f

Timeline trace json file: dart_devtools_2023-11-13_12_41_23.152.json

flutter doctor -v (stable and master)
[✓] Flutter (Channel stable, 3.13.9, on macOS 14.1 23B74 darwin-x64, locale en-VN)
    • Flutter version 3.13.9 on channel stable at /Users/huynq/Documents/GitHub/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision d211f42860 (11 days ago), 2023-10-25 13:42:25 -0700
    • Engine revision 0545f8705d
    • Dart version 3.1.5
    • DevTools version 2.25.0

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at /Users/huynq/Library/Android/sdk
    • Platform android-34, build-tools 34.0.0
    • ANDROID_HOME = /Users/huynq/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b802.4-9586694)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 15.0.1)
    • Xcode at /Users/huynq/Downloads/Xcode.app/Contents/Developer
    • Build 15A507
    • CocoaPods version 1.13.0

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2022.3)
    • Android Studio at /Applications/Android Studio Giraffe.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b829.9-10027231)

[✓] Android Studio (version 2022.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • android-studio-dir = /Applications/Android Studio.app/
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b802.4-9586694)

[✓] VS Code (version 1.84.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.76.0

[✓] Connected device (4 available)
    • RMX2001 (mobile) • EUYTFEUSQSRGDA6D                         • android-arm64  • Android 11 (API 30)
    • iPhone (mobile)  • d9a94afe2b649fef56ba0bfeb052f0f2a7dae95e • ios            • iOS 15.7.9 19H365
    • macOS (desktop)  • macos                                    • darwin-x64     • macOS 14.1 23B74 darwin-x64
    • Chrome (web)     • chrome                                   • web-javascript • Google Chrome 118.0.5993.117

[✓] Network resources
    • All expected network resources are available.

• No issues found!

[!] Flutter (Channel master, 3.17.0-5.0.pre.64, on macOS 14.1 23B74 darwin-x64, locale en-VN)
    • Flutter version 3.17.0-5.0.pre.64 on channel master at /Users/huynq/Documents/GitHub/flutter_master
    ! Warning: `flutter` on your path resolves to /Users/huynq/Documents/GitHub/flutter/bin/flutter, which is not inside your current Flutter SDK checkout at /Users/huynq/Documents/GitHub/flutter_master. Consider adding /Users/huynq/Documents/GitHub/flutter_master/bin to the front of your path.
    ! Warning: `dart` on your path resolves to /Users/huynq/Documents/GitHub/flutter/bin/dart, which is not inside your current Flutter SDK checkout at /Users/huynq/Documents/GitHub/flutter_master. Consider adding /Users/huynq/Documents/GitHub/flutter_master/bin to the front of your path.
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 2e088178e8 (4 hours ago), 2023-11-12 18:45:26 -0500
    • Engine revision a18ee3c7f5
    • Dart version 3.3.0 (build 3.3.0-120.0.dev)
    • DevTools version 2.30.0-dev.1
    • If those were intentional, you can disregard the above warnings; however it is recommended to use "git" directly to perform update checks and upgrades.

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at /Users/huynq/Library/Android/sdk
    • Platform android-34, build-tools 34.0.0
    • ANDROID_HOME = /Users/huynq/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b802.4-9586694)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 15.0.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 15A507
    • CocoaPods version 1.13.0

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2022.3)
    • Android Studio at /Applications/Android Studio Giraffe.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b829.9-10027231)

[!] Android Studio (version unknown)
    • Android Studio at /Applications/Android Studio Preview.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    ✗ Unable to determine Android Studio version.
    • Java version OpenJDK Runtime Environment (build 17.0.8+0-17.0.8b1000.22-10799086)

[✓] Android Studio (version 2022.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • android-studio-dir = /Applications/Android Studio.app/
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b802.4-9586694)

[✓] VS Code (version 1.84.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.76.0

[✓] Connected device (3 available)
    • RMX2001 (mobile) • EUYTFEUSQSRGDA6D • android-arm64  • Android 11 (API 30)
    • macOS (desktop)  • macos            • darwin-x64     • macOS 14.1 23B74 darwin-x64
    • Chrome (web)     • chrome           • web-javascript • Google Chrome 119.0.6045.123

[✓] Network resources
    • All expected network resources are available.

! Doctor found issues in 2 categories.


May I know the example that I can follow to use SignalR?

I tried the example below:
https://github.com/aspnet/Docs/tree/master/aspnetcore/signalr/get-started/sample/
But, nothing happened after I clicked "Send Message".

Can you provide information requested in the issue template so we can properly investigate? https://github.com/aspnet/SignalR/blob/dev/.github/ISSUE_TEMPLATE.md

I'm guessing you didn't follow the docs?
If you don't follow step 4 and 5 of https://docs.microsoft.com/en-us/aspnet/core/signalr/get-started?tabs=visual-studio#create-an-aspnet-core-project-that-hosts-signalr-client-and-server then you won't have the signalr client setup

Rick-Anderson just changed get-started\sample. Now, I cannot open the project at all.
VS:

VS Code:
C:\Users\xxx\Documents\Docs\aspnetcore\signalr\get-started\sample\SignalRChat.csproj(14,3): error MSB4067: The element <#text> beneath element  is unrecognized.
GitHub:

It looks like there was a bad merge. I filed https://github.com/aspnet/Docs/issues/5943 to track that.


The results obtained from dspy are all in English, how should the results be obtained in Chinese

可以在声明dspy.Signature时，指明输出为中文，比如：
answer: str = dspy.OutputField(desc='使用中文描述答案')

Adapters in 2.5 make this easy in other ways too!


DON-965: Cookie banner changes

I think we need to add a matching #cookies id into the content at https://biggive.org/privacy/ before merging this.
For the our applications a user clicking "Reject all" will be indistinguishable from a user clicking "Choose Preferences" and then rejecting every available option there.
Also having trouble getting the link-components script to work again, so I haven't tested this in context of donate-frontend (or wordpress). Will test when making the PRs to those repos:
Before:

After:

This PR is blocked by https://github.com/thebiggive/wordpress/pull/175 - that needs to be merged first.

Maybe it's OK to say this isn't actually blocked by the wordpress deploy, but the PRs that pull it into FE & WP will be. We shouldn't need to do any other components changes before end of GMF.


Consider publishing to Maven Central

Hi @mtamme, thumbs up for your useful library! 👍  Would you please consider $subj? It would make a life a little bit easier for anyone using it :).
If there's anything I can help you with regarding this, feel free to let me know.
Thanks and keep up good work!

Hi @dreo, since I don't consider the project to be ready for production it makes no sense for me to publish the artifacts. Sorry for that but it's just a prototype.
I started the project before Java 8 came out, the dependencies are outdated and there is further cleanup and refactoring required.
Cheers, Martin

Thanks Martin. My original intention why I wrote you was that - be it production ready or not - your library seems to be working (at least in my use case), unlike the original (and seemingly abandoned)  client from Microsoft. But I also understand your reasons.


Don't overwrite custom user defined event id

In some cases, we already set an event id provided via payloadPairs and it's important for us to keep that event id. However, currently in the core tracker the event id is overwritten no matter what. One very simple solution to this issue is to switch the order of adding the event id and the payloadPairs. In the normal way of using the tracker without providing an own event id, nothing changes.

Thanks for the PR @mcepok
We're well on the way to v3 now which sees a significant reshuffle of the codebase. I'm happy to include this change in v3.
If you want to build your own version of sp.js whilst you wait for v3, then you should be able to run: npm run build:all and it'll build core and the tracker then output a new sp.js to /dist.

Thanks for the PR @mcepok
We're well on the way to v3 now which sees a significant reshuffle of the codebase. I'm happy to include this change in v3.
If you want to build your own version of sp.js whilst you wait for v3, then you should be able to run: npm run build:all and it'll build core and the tracker then output a new sp.js to /dist.

We actually want this behavior for the snowplow-nodejs-tracker which, as far as I understood, is based since version 0.4.0 on the core-tracker of this repository. If the next release of the core-tracker is coupled with the javascript-tracker release, that's also fine.
Thanks a lot!

We actually want this behavior for the snowplow-nodejs-tracker which, as far as I understood, is based since version 0.4.0 on the core-tracker of this repository. If the next release of the core-tracker is coupled with the javascript-tracker release, that's also fine.
Thanks a lot!


update 3.0 print_only_common_headers

#322 Now I'm deleting approx 30 lines in code.

Do we need to test this?
Thanks,
Hari
On Aug 11, 2022, 8:43 AM -0400, Pradeeban Kathiravelu @.***>, wrote:

@pradeeban requested your review on: #336 update 3.0 print_only_common_headers .
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because your review was requested.Message ID: @.***>

@haritrivedi thanks, that will be helpful.
We don't want to break any functionality.

@Nitesh639 can you bring your dev branch to sync with Niffler's dev branch? By doing that, the test failure in this pull request should be fixed.

OK, I will do it today.

Hari and I reviewed the code, and changes are proposed.


Save model to pb and pbtxt

I would like to use a model trained on python on C++
tried by tflite but cmake needed don't work
by the past, I had to use pb file and pbtxt file that works.
In TF 2.0 how could I save a Model to pb and pbtxt file to use it on C++
Or is there a better way that doesn't involve Cmake ?
Thanks a lot,

@GaranceRichard, Please follow this guide to load a savedmodel in C++. Let us know if it helps. Thanks!

with my model (as model) I did :
tf.saved_model.save(model,"") which gives me a saved_model.pb
sent to C++ dev, I'll tell you if it's ok...

@GaranceRichard, Any update!


places-auto-complete

Input-html: places-auto-complete types="['geocode']" on-place-changed="myCallback(place)"
Angular: $scope.myCallback = function(place){
console.log(place);
};
Output: undefined

Please add a plunker example.

or just give me the source of this
https://rawgit.com/allenhwkim/angularjs-google-maps/master/testapp/places-auto-complete.html

         $scope.placeChanged = function() {
           $scope.place = this.getPlace();
           console.log($scope.place.geometry.location.lat(), $scope.place.geometry.location.lng());
           $scope.map.setCenter($scope.place.geometry.location);
         }

places auto compete not respon list places in web admin...

not list place

Please add a plunger that reproduces your problem

well i found ans tanks @allenhwkim
ans is:  $scope.place = this.getPlace();


Gated progression

Hi, I ve got some ideas so this really fun pack does not became another gtnh-like abomination. Starts with applied at start, it makes this pack and gregtech playable, make it gated like, you can access basic terminals and basic automatization at start, but you cant use wireless terminals and bigger pattern providers at start, like it may be opened at lv or at least at mv. This will make gameplay more like, I can breathe, than it is takes 3 hours to craft 2 machine yaaaay.
Another idea is make better recipes for steel and aluminum, let it take 4x or 3x raw resourse but make goal metal faster, the example: i spend 16 coal and 32 wrought iron to make 4 steel ingot, or 16 aluminum dust and 8 buckets of nitrogen to make 8 ingots of aluminum. Third thing is make quests more useful for newbies, like you can access nichrome coils when you get HV energy hatch, but quest that it's requires at the end of EV. Things like this all over the quest book and it is really disappointing, because the idea of pack is really cool. Another idea is return back the platinum processing, moon miner module that allows to mine sheldonite, like you did with End mining module.
Another suggestion is make coin shop more useful, add there models from deep learning, like peaceful mobs, just for fun and add for doge coin shop more materials to buy, so you can buy bunch of aluminum ingots when you got at mv or buy steel when you at lv.

There some bugs with ender IO energy conduits, like they're voiding energy or smth.I suggest adding PipeZ, lock upgrades at stages and you have energy conduits alike enderIO

Nice for  the suggestions :D
Pipez will not be added as we want people to use things like gregtech pipes and ae2 subnets, same reason as why we disabled functional storage upgrades.
Also I dont know why you suggest GTNH "gated progression" as on gtnh you get ae2 in a similar way by mining titanium-ores on the moon, like ilmenite. The pack is literally called Sky of Grind, early stages shouldn't be too hard but lategame uv-uev should be hard.
You also need to do the platline, so you will need to setup the processing for iridium, osmium, rhodium and palladium.

I dont mean GTNH like progression, just allow use applied at start, cause it will make it less grindy at start, as you say you want to make easy start and make hard late game

as suggestion for pipez, disable netherite and diamond upgrade so people dont use them at late game

And if you port GTNH feature that you can use fluid solidifier to make pipes for gt will be huge

As me and my friend come to LuV, we see some strange decisions. Those solar panels work very very very very strange and there is not so many alternatives, except GT energy generation

About Solar panels, you will start to use plasma soon.
You can make pipes on a extruder, I dont know what do u mean about the fluid solidifier
Also I dont want to make the early game easy, just not annoying like mining on early game gtnh.


how to connect brackets with google-chrome in linux

Am having troubles using the live preview in linux. How can i specify the default location or the default user account brackets should use to start google chrome ?

I have the same problem on Sabayon Linux 16.02 (based on Gentoo)
When i click the "Live Preview" button, "Launching browser error" is showing.
All these advices doesn't works:
I was installed all these dependences:
google-chrome: to enable Live Preview
gnuplot: to enable node benchmarking
ruby: to enable LiveDevelopment Inspector
hicolor-icon-theme: for hicolor theme hierarchy


Double-clicking in the file explorer has a race condition

Issue Type: Bug

Have the "OPEN EDITORS" section of the explorer open, and above the explorer's main section.
Double-click one of the files in the main section, but delay the second click slightly.

The file opens on the first click and gets added to the OPEN EDITORS section. This moves everything below it down by the height of one line, so if that happens before the second click, you're clicking on a different file. Often, instead of opening the second file, it opens a text entry field near the bottom of the explorer's main section (not necessarily anywhere near the mouse cursor) as if I'm trying to create a new file at the top-level of my project.
 However I could not find the duplicate so let's keep this one open.

I did subsequently find the issue that this duplicates, but it had been closed by a bot: #25911.

The limit is at 250ms currently but in theory can be changed to a higher value:
https://github.com/microsoft/vscode/blob/ae86e4c50532d12309dbc1bb7c9579b962fbe42e/src/vs/workbench/contrib/files/browser/explorerViewlet.ts#L197-L229


train ffhq

Hi, thanks for the excellent work. I need to train my own face dataset, Can you provide scripts for training FFHQ(Some hyperparameters setting)?
In addition, my face image is 256x256, does it affect the model?

Hi @ljiqy!
To make sure I undestand correctly - you wish to train the e4e encoder based on your own dataset, with the pretrained FFHQ StyleGAN2?
If so, i suggest using the training script in the README: https://github.com/omertov/encoder4editing#training-the-e4e-encoder,
with the following hyper parameters: --max_steps 300000 --id_lambda 0.1.
You should also specify --stylegan_size 1024 (this is so the StyleGAN model will load properly), and change the dataset_type according to your dataset configuration (see https://github.com/omertov/encoder4editing#training-on-your-own-dataset).
It is fine that your images are 256x256, as we downscale the output 1024x1024 stylegan images into 256x256 resolution before calculating the loss.
In case you meant for training the StyleGAN itself, you should head over to rosinality's stylegan2-pytorch repo and train the StyleGAN there before importing the trained checkpoint to be used in this repo.
Hope it helps!
Omer

Thank you for your reply quickly!
I have finish the stylegan2 training according rosinality's stylegan2-pytorch, now I use this script to train e4e:
python scripts/train.py
--dataset_type my_ffhqencode
--exp_dir ./my_model/mye4e
--start_from_latent_avg
--use_w_pool
--w_discriminator_lambda 0.1
--progressive_start 20000
--id_lambda 0.5
--val_interval 10000
--max_steps 250000
--stylegan_size 1024
--stylegan_weights ./pretrained_models/my-trained-stylegan2-config-f.pt
--workers 8
--batch_size 8
--test_batch_size 4
--test_workers 4
after using this script training, it doesn't work very well, If I understand you correctly, I need to change: --max_steps 300000 --id_lambda 0.1?

Yes, this is the configuration we ran for the FFHQ StyleGAN.
Can you elaborate on what's wrong with your results? Is it the reconstruction quality or the editing?

the reconstruction quality problem，I use twenty thousand images to train, my result(above is the original image, below is the reconstruction):

Hi @ljiqy,
Our encoder is known for favoring editing properties by giving up on slight reconstruction (distortion) quality, so perhaps this is where the reconstruction degradation comes from.
Other possibilities for such degradation might come from the GAN itself - perhaps the "well-behaved" regions in the GAN's latent space are not expressive enough for generating the source image, so it opts to the given result as the closest image (I wonder what happens when you generate a random image and then encode it back into the latent space).
Another way of testing it is by training a regular pSp network (removing the --w_discriminator_lambda --progressive_start keys and adding --encoder_type GradualStyleEncoder) to see if the resulting image is better.
Also it is possible that more training steps will help.
Best,
Omer


I wonder if there is an maintained version of Audiogram

It looks like it hasn't been maintained for a long time. I have managed to install it and it works for short soundbites but I need to do a long (1hr) audiogram and it fails. Not sure what's wrong and as this version is not under active development I wonder if someone uses a newer version with an active developer?

Hi @foxy123 - I had an issue with longer podcasts and made some changes to waveform.js that seems to have helped. I submitted a pull request https://github.com/nypublicradio/audiogram/pull/127. If you drop in this waveform.js file, it may help you.


move types to devDependencies

"@types/mysql": "^2.15.2",
"@types/bluebird": "^3.5.26"
they are currently under dependencies, but should be under devDependencies

We export those types as part of our own types. See discussion on here: https://github.com/lukeb-uk/node-promise-mysql/pull/108


Expose a command line interface

So that we make sure users don't try to use the templateflow resources directly on their own.

to add: we should make it super simple for people to update their templateflow skeletons
templateflow update

or something along those lines


Always include coerce= model argument in check_response

In some cases we are not yet doing it.
E.g. in workflow_show the block
    res = await client.get(f"{settings.BASE_URL}/workflow/{workflow_id}")
    workflow = check_response(res, expected_status_code=200)
    return RichJsonInterface(retcode=0, data=workflow)

should rather look like
    res = await client.get(f"{settings.BASE_URL}/workflow/{workflow_id}")
    workflow = check_response(res, expected_status_code=200, coerce=WorkflowRead)
    return RichJsonInterface(retcode=0, data=workflow.dict())

Note: https://github.com/fractal-analytics-platform/fractal-server/issues/489 will not be an issue here, but we still have nested objects. Let's check carefully that the .dict() method woks as expected.

Side comment: this would not work out-of-the-box for lists (e.g. List[ApplyWorkflow]), and therefore those cases can still be handled as in
    res = await client.get(f"{settings.BASE_URL}/project/{project_id}/jobs/")
    jobs = check_response(res, expected_status_code=200)

    # Coerce to a list of ApplyWorkflowRead objects
    jobs = [ApplyWorkflowRead(**job) for job in jobs]

Note: fractal-analytics-platform/fractal-server#489 will not be an issue here, but we still have nested objects. Let's check carefully that the .dict() method woks as expected.

This block in test_workflow_add_task, for instance, works as expected
    # Check that the WorkflowTask's in Workflow.task_list have the correct IDs
    cmd = f"workflow show {wf.id}"
    res = await invoke(cmd)
    assert res.retcode == 0
    workflow = res.data
    debug(workflow)
    list_IDs = [wftask["id"] for wftask in workflow["task_list"]]
    assert list_IDs == [workflow_task_id_1, workflow_task_id_2]

Output
    workflow: {
        'name': 'name',
        'project_id': 1,
        'id': 1,
        'task_list': [
            {
                'workflow_id': 1,
                'task_id': 1,
                'order': 0,
                'meta': {
                    'executor': 'some-executor',
                },
                'args': {
                    'arg': 'arg_value',
                },
                'id': 1,
                'task': {
                    'name': 'test_task',
                    'command': 'cmd',
                    'source': 'source',
                    'input_type': 'Any',
                    'output_type': 'Any',
                    'default_args': {},
                    'meta': {},
                    'id': 1,
                },
            },
            {
                'workflow_id': 1,
                'task_id': 1,
                'order': 1,
                'meta': None,
                'args': None,
                'id': 2,
                'task': {
                    'name': 'test_task',
                    'command': 'cmd',
                    'source': 'source',
                    'input_type': 'Any',
                    'output_type': 'Any',
                    'default_args': {},
                    'meta': {},
                    'id': 1,
                },
            },
        ],
    } (dict) len=4


clean up browser console warnings and some naming

no significant changes here, nothing in the UI and no logic
did change:
naming of some props, vars, components
key and justify is deprecated prop browser console warnings

Still works on my end!


Teams Page Added

Teams Page Added

@Anushka-shukla If you need assets and names for this checkout the website repo please

Okay will do the necessary changes

@Anushka-shukla  what is the progress on this?


[Feature]: tab completion page results

What would you like to see added?
the tab completion menu is hardcoded to output 10 entries. first should add configuration to adjust how many are shown, and should also display more entries if the user navigates down

configuration is complete, but no paging support if overflow


[WIP] Support sidelining large events/payloads to an external event store

Description
EventBridge's PutEvents size limit is currently 256KB. Often, Kafka records are larger (default 1MB) and the connector drops (or DLQs, if configured) those records with a warning. By adding a new configuration property, the connector could send (sideline) those records to a configured event store, such as S3 e.g., by using the claim-check pattern, replacing the event payload with an S3 link (reference).
Test Steps

Checklist:

[ ] I have tested my changes. No regression in existing tests.
[ ] I have modified and/or added unit-tests to cover the code changes in this Pull Request.

Related Issue
#110
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

FYI @embano1 I replaced JSON Pointer by JSON Path in the implementation by using Jayway JsonPath

@baldawar ICYMI, we're going to use https://github.com/json-path/JsonPath to mimic EventBridge's JSON path expression syntax (with restrictions that only $.<child> path syntax can be used to find/extract a key).
Also, WDYT about the way we replace values in the object to keep the existing structure in tact (for filtering), but providing a $.dataref pointer at the root for consumers to find the replaced value, if any? This will obviously only work for consumers w/out strict schema validation (or optional fields).

@embano1 I added two test cases for SinkRecords with string value and started S3 exception handling. S3 operations have a request limit? How should be handle these cases?

S3 operations have a request limit? How should be handle these cases?

You mean payload (object) or complete request size limit? What's the limits (also key length) we should know of?

I mean the frequency putting S3 objects. Is it possible to restrict the (payload) size of S3 objects by a policy?

I mean the frequency putting S3 objects. Is it possible to restrict the (payload) size of S3 objects by a policy?

Not that I know of. Only way is through https://stackoverflow.com/questions/31412335/can-i-limit-the-size-of-an-object-put-into-s3-via-the-javascript-api which we don't use here.

> 3500 PUT/POST req/s => https://docs.aws.amazon.com/AmazonS3/latest/userguide/optimizing-performance.html, yield in an 503 (Slow Down)

Regarding response/exception handling for put S3 object.
Any non successful put operation throws an S3Exception which contains the HTTP status code. The ErrorCodeList contains the S3 error codes.
How should the 4xx errors be handled?
Solution (a) each 4xx should yield in Error.panic to shut down the connector.
Solution (b) skip S3 upload and keep value as is and handle EventBridge event size via batching.
...
The 500 and 503 (w/ Error Code ServiceUnavailable) is clear, try again. But how should we handle 504 w/ Error Code SlowDown?
Is there any case we can/should use Error.reportOnly?

@baldawar ICYMI, we're going to use https://github.com/json-path/JsonPath to mimic EventBridge's JSON path expression syntax (with restrictions that only $.<child> path syntax can be used to find/extract a key).
Also, WDYT about the way we replace values in the object to keep the existing structure in tact (for filtering), but providing a $.dataref pointer at the root for consumers to find the replaced value, if any? This will obviously only work for consumers w/out strict schema validation (or optional fields).

It sounds a fair starting point to me. I'd be worried about the security implications and correctness, so would recommend having some means to sanity check the data prior to reading on the consumer side (and adding as much guidance you can get away with before you we turn off our readers)

As discussed w/ @embano1, I have implemented in addition to $.dataref the attribute $.datarefPath which is always present when S3 sidelining is activated and contains the configured JSON Path.
To summarize:
When Kafka SinkRecord value is null

$.detail.value == null
$.dataref is absent
$.datarefPath present w/ configured JSON Path

When Kafka SinkRecord value is valid JSON and:
JSON Path matches:

$.detail.value w/ transformed SinkRecord value
$.dataref w/ S3 ARN
$.datarefPath present w/ configured JSON Path

JSON Path does not match

$.detail.value w/ transformed SinkRecord value
$.dataref is absent
$.datarefPath present w/ configured JSON Path

As discussed w/ @embano1, I have implemented in addition to $.dataref the attribute $.datarefPath which is always present when S3 sidelining is activated and contains the configured JSON Path.
To summarize:
When Kafka SinkRecord value is null

$.detail.value == null
$.dataref is absent
$.datarefPath present w/ configured JSON Path

When Kafka SinkRecord value is valid JSON and:
JSON Path matches:

$.detail.value w/ transformed SinkRecord value
$.dataref w/ S3 ARN
$.datarefPath present w/ configured JSON Path

JSON Path does not match

$.detail.value w/ transformed SinkRecord value
$.dataref is absent
$.datarefPath present w/ configured JSON Path

@agebhar1 I thought about the above proposal and IMHO we need to introduce a slight change to get this behavior:

If $.datarefPath exists in $.detail.value and is not null $.dataref links to S3 target
If $.datarefPath exists in $.detail.value and is null $.dataref is null
If $.datarefPath does not exist in $.detail.value $.dataref is absent

Reason is to give the consumer a way to know whether $.datarefPath actually matched a null key so they know whether to add this key back to their event with null (since it can only be null in that case) or leave it out.
Do you see any issues with that? IMHO we're already good with $.datarefPath being {} or [] as we're treating this as a normal offload situation, right? Would be great to get some unit tests for this.

@embano1 implement all hints from the review and updated the commit messages.
@embano1, @maschnetwork I re-use the EventBridgeMappingResult for now. It is from the package ...mapping, should we move it on level up?

@agebhar1 I think we can keep it in the mapping package for now

Thx @agebhar1 ! Will do some real-world testing this week so we can close this one! Great work!

@agebhar1 verified against a live environment, works like a charm:

[x] fails with permanent exception (non-retryable) when credentials don't have S3 upload permissions
[x] offloads to S3 when JSON path matches
[x] does not upload same file again if in cache
[x] ignores upload to S3 if no JSON path match

I have two minor things I'd like to discuss before merging this one:

I feel datarefPath might still be ambiguous and would recommend a small change to datarefJsonPath or similar to make it clear that this is the JSON pointer (yes, this binds our implementation to JSON/JSONPath but I don't see this going away and we're coupled to JSON in many places today)
we should log (TRACE) when we upload to S3 or have a cache hit (similarly what we do with the starting/completing putevents call with some nice duration stats for correlation)

@agebhar1 verified against a live environment, works like a charm:

Awesome!

I feel datarefPath might still be ambiguous and would recommend a small change to datarefJsonPath or similar to make it clear that this is the JSON pointer (yes, this binds our implementation to JSON/JSONPath but I don't see this going away and we're coupled to JSON in many places today)

Done.

we should log (TRACE) when we upload to S3 or have a cache hit (similarly what we do with the starting/completing putevents call with some nice duration stats for correlation)

Done.

Updated the log (trace) messages, e.g.:
TRACE [@2721950-dirty] uploading object to s3 with key=arn:aws:s3:::test-bucket/9a9e905a-b601-45d7-a1d7-df6bbd75da62
TRACE [@2721950-dirty] s3 offloading completed: jsonPath='$.detail.value.message' key=arn:aws:s3:::test-bucket/9a9e905a-b601-45d7-a1d7-df6bbd75da62 cache=MISS durationMillis=112

This should be enough, if datarefJsonPath is present, when dataref with the concrete S3 key is present too (and vice versa). If dataref is not present, then nothing matched or was 'null'.


Added variable for cloudwatch Full Request and Response Logs: data_trace_enabled

Repeat of PR 36 with requested updates from main and to readme.
what

Added variable data_trace_enabled to the aws_api_gateway_method_settings resource
This change allows for configuration of CloudWatch logging setting "Full Request and Response Logs" available in the AWS UI. [2]

why
The variable logging_level controls CloudWatch log setting in the AWS UI for OFF, INFO,  and ERROR, but doesn't include an option for "Full Request and Response Logs". In the AWS UI for API GW, there's an additional option, "Full Request and Response Logs", as shown in the screenshot:

According to terraform documentation, the variable data_trace_enabled = true is required in conjunction with logging_level  = "INFO" to enable "Full Request and Response Logs". This is added to the aws_api_gateway_method_settings resource in the settings code block [2]:
settings {
    logging_level      = "INFO"
    metrics_enabled    = true
    data_trace_enabled = true
  }

references
[1] Terraform resource:
https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/api_gateway_method_settings
[2] Closed Issue with Terraform citing solution
https://github.com/hashicorp/terraform-provider-aws/issues/35863#issuecomment-1954633612

/terratest

@sheilman16 good stuff -- Thanks for the contribution + circling back around to this work! 👍


Remote Write "queue_config" Options Exposure

Would it be possible to expose some of the queue_config knobs available for remote write in the RemoteWriteSpec object? We have a use case where we need to tweak these settings and we would like to use the prometheus-operator to do so.

https://prometheus.io/docs/prometheus/latest/configuration/configuration/#remote_write

Thanks for opening this issue! Yes I think we should expose these options. Would you like to shoot us a PR to add these? :slightly_smiling_face:

Sure, let me see if I can get one out today. :D

Verified with the following prometheus remoteWrite config:
  remoteWrite:
    - url: http://prometheus/prometheus
      queueConfig:
        capacity: 1
        maxShards: 10000
        maxSamplesPerSend: 1000
        batchSendDeadline: 1s
        maxRetries: 1
        minBackoff: 10ms
        maxBackoff: 30ms

+1 on this, need to tweak parameters on my end to try to hunt down an OOM issue we're experiencing

This issue is stale, closing.  :smile:


Update zio, zio-test, zio-test-junit, ... to 2.1.2

About this PR
📦 Updates

dev.zio:zio
dev.zio:zio-test
dev.zio:zio-test-junit
dev.zio:zio-test-magnolia
dev.zio:zio-test-sbt

from 2.0.22 to 2.1.2
📜 GitHub Release Notes - Version Diff
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "dev.zio" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "dev.zio" }
}]

labels: library-update, early-semver-minor, semver-spec-minor, commit-count:1

Superseded by #168.


ClassCastException when using Async Client

I created a test project to check the functionality of the asynchronous client as described in the HTTP Async Transport section. I created a server and a client for testing. Here is the link to the project on GitHub:
https://github.com/dabkhazi/quarkus-soap-async-test
However, I am unable to use the asynchronous client due to the following error:
java.lang.ClassCastException: class org.apache.cxf.jaxws.JaxwsResponseCallback cannot be cast to class io.netty.util.concurrent.Future
(org.apache.cxf.jaxws.JaxwsResponseCallback and io.netty.util.concurrent.Future are in unnamed module of loader io.quarkus.bootstrap.classloading.QuarkusClassLoader)
The test project uses Java 17 and Quarkus 3.15.1. Do you have any ideas on what might be going wrong?

Thanks for the report and reproducer, @dabkhazi. I guess the import around here should be java.util.concurrent.Future rather than io.netty.util.concurrent.Future https://github.com/dabkhazi/quarkus-soap-async-test/blob/main/client/src/main/java/io/github/dabkhazi/wscalculator/boundary/CalculatorResource.java#L5

@ppalaga Thank you very much, I fixed the Future class and now everything works. But I saw that there was no need to implement asynchronous processing in a special way for the server, because according to the logs it is not called and it can be safely removed from the server implementation. Also frustrating is the fact that the asynchronous call comes from the eventloop thread, and the response returns to the worker thread.

Also frustrating is the fact that the asynchronous call comes from the eventloop thread, and the response returns to the worker thread.

Yeah, the execution and result on a worker thread is given by CXF design. The only thing we customize in that area on Quarkus is using a worker thread pool managed by Quarkus.
Actually, the lazy client initialization on first use may block if you set a WSDL URL on the client via quarkus.cxf.client."client-name".wsdl. In that case the even loop would be blocked and the best practice is to suspend to a worker thread strainght away, e.g. by annotating the REST endpoint by @io.smallrye.common.annotation.Blocking.


[Request] Minion Charges

Would really like to see an option somewhere to see what effect frenzy, power, and endurance charges have on all minion skills (Spectres, Golems, Zombies, Animate Weapon, Raging Spirit...).
Keystone passive "Necromantic Aegis" with Victario's Charity Shield gives minions full frenzy and power charges almost 100% of the time. Can't decide if I want to use a clay shaper over a shield or not due to the the power bonuses minions get from charges.
Would be much appreciated if you could implement! Thanks for all your hard work, this program is amazing!

Minion charge options are already present; they're under the player charge options, but they only appear when you have a minion skill as your main skill.


Updated README for clarity

There was a typo in the line that has been changed. The sentence itself has been altered slightly to accommodate the fix for the typo and maintain clarity.

Checklist

[ ] I've looked at the contribution guidelines.
[ ] This pull request is complete and ready for review.

Description

👍


CFP Remainders in SM

Remainder Posts for CFP in SM
@anistark @wavicles @WatchfulHitman
**

[x] Putting up remainder post in the SM for CFP this week
[ ] Count Down messages in SM after 3rd Week of this month
**

Guess this is done. @muralinmc  feel free to reopen if anything pending.