Proper SSE parsing

Specs: https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events#event_stream_format
As you can see, the specifications are rigidly defined, eliminating the necessity to use LPEG and iterative trial-and-error JSON decoding.
In addition, the current implementation is unable to parse valid cases, for instance, when the response contains comments (lines commencing with a colon). This is something you could encounter.
Specifically, below is a sample response from https://openrouter.ai/api/v1/chat/completions:

: OPENROUTER PROCESSING

data: {"id":"gen-Hh4bFjJS0U0YgPav7S9tMpfXrNHA","model":"mistralai/mistral-7b-instruct:free","object":"chat.completion.chunk","created":1713306744,"choices":[{"index":0,"delta":{"role":"assistant","content":"The"},"finish_reason":null}]}

data: {"id":"gen-Hh4bFjJS0U0YgPav7S9tMpfXrNHA","model":"mistralai/mistral-7b-instruct:free","object":"chat.completion.chunk","created":1713306744,"choices":[{"index":0,"delta":{"role":"assistant","content":" next"},"finish_reason":null}]}

data: {"id":"gen-Hh4bFjJS0U0YgPav7S9tMpfXrNHA","model":"mistralai/mistral-7b-instruct:free","object":"chat.completion.chunk","created":1713306744,"choices":[{"index":0,"delta":{"role":"assistant","content":" day"},"finish_reason":null}]}

data: {"id":"gen-Hh4bFjJS0U0YgPav7S9tMpfXrNHA","model":"mistralai/mistral-7b-instruct:free","object":"chat.completion.chunk","created":1713306744,"choices":[{"index":0,"delta":{"role":"assistant","content":","},"finish_reason":null}]}

data: {"id":"gen-Hh4bFjJS0U0YgPav7S9tMpfXrNHA","model":"mistralai/mistral-7b-instruct:free","object":"chat.completion.chunk","created":1713306744,"choices":[{"index":0,"delta":{"role":"assistant","content":" I"},"finish_reason":null}]}

data: {"id":"gen-Hh4bFjJS0U0YgPav7S9tMpfXrNHA","model":"mistralai/mistral-7b-instruct:free","object":"chat.completion.chunk","created":1713306744,"choices":[{"index":0,"delta":{"role":"assistant","content":" wand"},"finish_reason":null}]}

data: {"id":"gen-Hh4bFjJS0U0YgPav7S9tMpfXrNHA","model":"mistralai/mistral-7b-instruct:free","object":"chat.completion.chunk","created":1713306744,"choices":[{"index":0,"delta":{"role":"assistant","content":"ered"},"finish_reason":null}]}

data: {"id":"gen-Hh4bFjJS0U0YgPav7S9tMpfXrNHA","model":"mistralai/mistral-7b-instruct:free","object":"chat.completion.chunk","created":1713306744,"choices":[{"index":0,"delta":{"role":"assistant","content":" through"},"finish_reason":null}]}

data: {"id":"gen-Hh4bFjJS0U0YgPav7S9tMpfXrNHA","model":"mistralai/mistral-7b-instruct:free","object":"chat.completion.chunk","created":1713306744,"choices":[{"index":0,"delta":{"role":"assistant","content":" the"},"finish_reason":null}]}

data: {"id":"gen-Hh4bFjJS0U0YgPav7S9tMpfXrNHA","model":"mistralai/mistral-7b-instruct:free","object":"chat.completion.chunk","created":1713306744,"choices":[{"index":0,"delta":{"role":"assistant","content":" streets"},"finish_reason":null}]}

data: {"id":"gen-Hh4bFjJS0U0YgPav7S9tMpfXrNHA","model":"mistralai/mistral-7b-instruct:free","object":"chat.completion.chunk","created":1713306744,"choices":[{"index":0,"delta":{"role":"assistant","content":" of"},"finish_reason":null}]}

data: {"id":"gen-Hh4bFjJS0U0YgPav7S9tMpfXrNHA","model":"mistralai/mistral-7b-instruct:free","object":"chat.completion.chunk","created":1713306744,"choices":[{"index":0,"delta":{"role":"assistant","content":" the"},"finish_reason":null}]}

data: {"id":"gen-Hh4bFjJS0U0YgPav7S9tMpfXrNHA","model":"mistralai/mistral-7b-instruct:free","object":"chat.completion.chunk","created":1713306744,"choices":[{"index":0,"delta":{"role":"assistant","content":" city"},"finish_reason":null}]}

data: {"id":"gen-Hh4bFjJS0U0YgPav7S9tMpfXrNHA","model":"mistralai/mistral-7b-instruct:free","object":"chat.completion.chunk","created":1713306744,"choices":[{"index":0,"delta":{"role":"assistant","content":","},"finish_reason":null}]}

data: {"id":"gen-Hh4bFjJS0U0YgPav7S9tMpfXrNHA","model":"mistralai/mistral-7b-instruct:free","object":"chat.completion.chunk","created":1713306744,"choices":[{"index":0,"delta":{"role":"assistant","content":" feeling"},"finish_reason":null}]}

data: {"id":"gen-Hh4bFjJS0U0YgPav7S9tMpfXrNHA","model":"mistralai/mistral-7b-instruct:free","object":"chat.completion.chunk","created":1713306744,"choices":[{"index":0,"delta":{"role":"assistant","content":" lost"},"finish_reason":null}]}

data: {"id":"gen-Hh4bFjJS0U0YgPav7S9tMpfXrNHA","model":"mistralai/mistral-7b-instruct:free","object":"chat.completion.chunk","created":1713306744,"choices":[{"index":0,"delta":{"role":"assistant","content":" and"},"finish_reason":null}]}

data: {"id":"gen-Hh4bFjJS0U0YgPav7S9tMpfXrNHA","model":"mistralai/mistral-7b-instruct:free","object":"chat.completion.chunk","created":1713306744,"choices":[{"index":0,"delta":{"role":"assistant","content":" alone"},"finish_reason":null}]}

data: {"id":"gen-Hh4bFjJS0U0YgPav7S9tMpfXrNHA","model":"mistralai/mistral-7b-instruct:free","object":"chat.completion.chunk","created":1713306744,"choices":[{"index":0,"delta":{"role":"assistant","content":"."},"finish_reason":null}]}

data: {"id":"gen-Hh4bFjJS0U0YgPav7S9tMpfXrNHA","model":"mistralai/mistral-7b-instruct:free","object":"chat.completion.chunk","created":1713306744,"choices":[{"index":0,"delta":{"role":"assistant","content":" I"},"finish_reason":null}]}

data: {"id":"gen-Hh4bFjJS0U0YgPav7S9tMpfXrNHA","model":"mistralai/mistral-7b-instruct:free","object":"chat.completion.chunk","created":1713306744,"choices":[{"index":0,"delta":{"role":"assistant","content":" had"},"finish_reason":null}]}

data: {"id":"gen-Hh4bFjJS0U0YgPav7S9tMpfXrNHA","model":"mistralai/mistral-7b-instruct:free","object":"chat.completion.chunk","created":1713306744,"choices":[{"index":0,"delta":{"role":"assistant","content":" no"},"finish_reason":null}]}

data: {"id":"gen-Hh4bFjJS0U0YgPav7S9tMpfXrNHA","model":"mistralai/mistral-7b-instruct:free","object":"chat.completion.chunk","created":1713306744,"choices":[{"index":0,"delta":{"role":"assistant","content":" idea"},"finish_reason":null}]}

data: {"id":"gen-Hh4bFjJS0U0YgPav7S9tMpfXrNHA","model":"mistralai/mistral-7b-instruct:free","object":"chat.completion.chunk","created":1713306744,"choices":[{"index":0,"delta":{"role":"assistant","content":" where"},"finish_reason":null}]}

data: {"id":"gen-Hh4bFjJS0U0YgPav7S9tMpfXrNHA","model":"mistralai/mistral-7b-instruct:free","object":"chat.completion.chunk","created":1713306744,"choices":[{"index":0,"delta":{"role":"assistant","content":" I"},"finish_reason":null}]}

data: {"id":"gen-Hh4bFjJS0U0YgPav7S9tMpfXrNHA","model":"mistralai/mistral-7b-instruct:free","object":"chat.completion.chunk","created":1713306744,"choices":[{"index":0,"delta":{"role":"assistant","content":" was"},"finish_reason":null}]}

: the rest is trimmed, to keep the issue message length in limits (maximum is 65536 characters).

data: [DONE]

As you can see, the specifications are rigidly defined, eliminating the necessity to use LPEG and iterative trial-and-error JSON decoding.

The specification is unrelated to the reason for the way the parsing is done. Output is being streamed from the http client, there is no guarantee that a full message will be put into the buffer as output is written from the server. cjson is not a streaming parser, so repeated parsing is done to emulate a streaming parser.
If you're concerned about performance, a potential optimization would be to start from the full length of the string and shrink by 1, instead of starting from length 1 and increase to the end. As it's more likely the correct position is located near the end of the string, not the beginning.

Specifically, below is a sample response from https://openrouter.ai/api/v1/chat/completions:

I've only tested this with the OpenAI API, but I have no issue with having comments be ignored if other "compatible" APIs output them.

Output is being streamed from the http client, there is no guarantee that a full message will be put into the buffer as output is written from the server.

Yes, but if we adhere to the specifications, we can still depend on the subsequent message being separated by two new lines. Instead of trying to simulate streaming JSON parsing, we may simply accumulate the response until '\n\n' is hit, and then JSON-parse the message in one go. This is the method employed in certain other libraries.

If you're concerned about performance.

Well, not so much, but while we still need to consider the comments-case, why not implement everything in the most optimal way?
I could undertake the implementation.


storage: Remove Mvcc from storage::Error

Signed-off-by: Little-Wallace<EMAIL_ADDRESS>
What problem does this PR solve?
Problem Summary:
storage::mvcc::Error has been included in storage::txn::Error::Mvcc, it is not necessary to define it in storage::Error. And  as we all know, MVCC (Multi-Version Concurrency Control) is just one kind of transaction implement. Not all databases implement transaction with MVCC, so we shall not define such a kind of error type in storage::Error.
What is changed and how it works?
Proposal: xxx
What's Changed:
Related changes

PR to update pingcap/docs/pingcap/docs-cn:
PR to update pingcap/tidb-ansible:
Need to cherry-pick to the release branch

Check List
Tests

No code

Side effects

Performance regression

Consumes more CPU
Consumes more MEM

Breaking backward compatibility

Release note

No release note.

/test

Please fix the conflict, thanks!


The strict mode for types inference

In Langium 0.2.0 types are inferred from rules names implicitly. For example, the following 2 examples will infer a type MyRule with the attribute name.
MyRule:
    name=ID;

MyRule returns MyRule:
    name=ID;

When we add types, type inference like described above will still be allowed by default.
This feature issue proposes to add the strict mode, when implicit type inference is forbidden. So, the way to specify a type of MyRule in the strict mode is:
type MyRule = {
    name: string
}

MyRule returns MyRule:
    name=ID;

We can generate "a type system template" automatically in the strict mode. So, we just infer types like in the "default" mode and put them in separate .langium file. As the result:

The strict mode works out of box
A user develops the type system by adding and modifying types instead of creating everything from zero

@danieldietrich proposed to add that functionality to the CLI.

This will be considerably easier to implement with #438 and #440.


fix edit menu items

Steps to reproduce bug:

Start application
Click any item in the Edit menu
Application will throw ReferenceError: mainWindow is not defined

Looks like I broke this in PR #140 when I removed the mainWindow variable from the global scope but forgot to update these menu items. I think this is more correct anyway as previously these operations were probably being applied to whatever window was most recently created instead of the currently active window.

Hi @chris48s, thanks for all your contributions. The ODI Australian Network has big plans for Comma Chameleon.
https://ask.theodi.org.au/t/a-vision-for-comma-chameleon/473
@pezholio pointed me in your direction.
Contact me via
Twitter @SGatesHQ
Email: StephenG @ ODIQueensland.org.au
to see the draft spec.

Hi @chris48s - thanks for working on this. I've confirmed that this patch fixes the majority of commands in the edit menu. The one exception is copy, for which I've opened a separate ticket #164 as I suspect its not related to the fix you've made here.
I'd also like to echo Stephen's sentiment - thanks so much for the consistent contributions to this repo! I wanted to let you know that you have push access on this repository, so you don't have to work in forks for any future contributions you'd like to make to this repository.


Chore: Import the WebUI into the main Datapane Repository

This PR contains 2 main components;

a rewritten history of datapane-next to be under the webui/ directory (we don't allow merge commits)
Minor alterations for building standalone from our build harness

Details for replicating the subtree import (without a merge commit):
Command for generating the prefixed tree:
git clone git@github.com:datapane/datapane-next.git -b master dp-next-scratch
cd dp-next-scratch
git filter-repo --to-subdirectory-filter webui
git remote add origin git@github.com:datapane/datapane-next.git
git push origin master:webui-master

The above is available here: https://github.com/datapane/datapane-next/tree/webui-master
Then from the datapane repo:
git remote add dpnext git@github.com:datapane/datapane-next.git
git checkout -b chore/new-branch-name dpnext/webui-master
git pull --rebase origin master
git push -u origin chore/new-branch-name

for open PRs in datapane-next, Ping me, and I'll work with you to rewrite the history ready

General structure and commits all LGTM!
Just got a Q before we merge -- will there be a follow-up PR in the datapane-hosted repo to point to datapane/webui/? As the codebase is still using the old datapane-next submodule.

Ah ha, sorry I thought I had the draft PR up already: https://github.com/datapane/datapane-hosted/pull/929

@Buroni I've just had to do a rebase, as we disallow merge commits.
Updating generation instructions atm

We can enable merge commits temporarily to merge this if makes things easier

Too late, done the history rewrite!
It's more reflective of what's actually happened anyway; and the original repo will be archived, not deleted.


[file_selector] getSavePath doesn't return the selected extension group

https://github.com/flutter/plugins/blob/main/packages/file_selector/file_selector/lib/file_selector.dart#L89
Currently call getSavePath, set multiple acceptedTypeGroups, and when user changes suggested name, it will not take the selected extension with it.

it will not take the selected extension with it.

I'm not sure what this means exactly. Could you provide a specific example with details (what exactly is typed, what is selected, and what the return value is vs what you expect)?

But according the api provided by file_selector package, we can not known which XTypeGroup has been selected by user.

Why do you need to know what the filter was set to? You have the full path they chose, including the filename.

This would require a new API with a different return type; the current API assumes that the type groups are just a selection filter (e.g., so that summertime doesn't accidentally select an existing file of the wrong type to save over), not an input to the logic of deciding what file to write.
It'll take some thought to decide how to handle macOS in that API since macOS doesn't work this way.

Hi, we were reviewing this request and found that the path returned for saving a file contains the complete file name even the extension if any.
What is not happening is a reactive behavior when the user changes the type.
This is because the IFileDialog doesn't change the file name in the text box when the save type is modified.
For Windows, we can implement this behavior by adding an event handler to the IFileDialog using advise implementing the IFileDialogEvents::OnTypeChange method to update the file name with the extension.
This won't modify the API and will be transparent to the user.
What do you think?

We also reviewed the macOS functionality and, when passing a single allowed extension when opening the dialog, the user doesn't need to indicate the extension in the file name, it is saved directly with that extension.
With more than one allowed extension the element is saved using the first one in the list.
const allowedTypes= XTypeGroup(label: 'text', extensions: <String>[
      'txt',
      'json',
    ]);

This will save the file in txt unless the user indicates .json in the file name.

This request isn't relevant to macOS because there's no UI related to type groups on macOS. It's specific to Windows and Linux.
Updating the file name directly doesn't necessarily work in all cases:

If the group has multiple extensions, there's no way to do so, but a post-selection option might have internal, app-specific logic about what to use.
If the user changes the group first, then changes the name, the group selection is effectively lost.

I think the most flexible option would be to add a save method that returns a result object containing both a String? and an XTypeGroup?, where the latter is non-null only on platforms that provide a type selection UI. It'll be more work to use in the simpler cases though, so we may want to keep both versions indefinitely, rather than treating it as a replacement.

This is now fixed at the API level via the new getSaveLocation, and is implemented for Windows. The Linux implementation still needs to be updated to return the selected group; it's doable, but will require some restructuring of the C implementation, in order to keep the list of filter objects around for comparison (since the relevant GTK API returns the selected object, not index, so we need to be able to map back to the index from that).


Best API for having the Merchant sheet NPC create/update/delete/find items

Hi I'm the developer of the https://github.com/whelan/fvtt-merchant-sheet-npc and have been asked by more than one GURPS GM if I could add support for GURPS.
I started out finding how the money and item cost and weight.
So now I have the basic view and calculations in order. But I'm struggling with how to transfer the items from merchant to player and the other way.
Normally I use createEmbeddedDocuments
The class I'm working on for fixing this is:
https://github.com/whelan/fvtt-merchant-sheet-npc/blob/gurps-integration/Source/merchant/systems/GurpsCurrencyCalculator.ts
Many methods have not been ported yet, so it has a lot of the stuff from dnd5e which is the most complete system integration I have.
Another thing when you drop the item from a compendium to the actor what happens which methods are called?
Because that item I can not find either in the equipment or items?

As for the money aspect... I don't know how it is currently set up, but as I mentioned before... GURPS is so configurable, we don't have a standard definition for money.
What you could do, and this is just a wild suggestion, is provide a setting for a string (or multiple strings) that can be searched for in the equipment list (using the findEquipmentByName() that you found).
The resultant equipment objects will have a count and a cost.   Dungeon Fantasy RPG (GURPS specifically for fantasy) uses the conversion 1 gold = 20 silver = 400 copper (and copper is the base currency, denoted with "$").   So an equipment entry for "gold" would have a count of 1, but a cost of 400.
This is the hardest currency to work with, because you would need to remove from Gold first, then silver, then copper... which you could tell by sorting on the cost.
By making this a setting, the GM could define the equipment name(s) as "wallet" or "galactic credits"), with cost 1.
NOTE:   It is allowed to have equipment with count == 0.   Especially for something like "wallet" or "galactic credits".    The user can delete the "zero" qty equipment line if they want... but more likely, they will be getting some in the future, so there is no need to remove the line.
Ok, just musing on some ideas.

For the money I think I will keep it like a free for all string, but only one currency.
If I was to support the fantasy, I need to also have the conversion rates defined as well. So you add a currency and the rate to the next one.

I have now got it to insert the item, but now I'm working on the update item. What is the best way to update the count on the item?

We currently don't have a single API call to update the count.    You have to find the equipment (and the key into the data structure), and then call updateEqtCount().   To subtract 1 from the current count, you could execute:
[eqt, key] = actor.findEquipmentByName("some item name")
actor.updateEqtCount(key, eqt.count - 1)

If this "equipment" is provided by a Foundry Item, it will update the item's data.eqt.count as well.

I got this error and I have no idea why
Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'split')
at GurpsActor.updateParentOf (actor.js:4094:24)
at GurpsActor.updateEqtCount (actor.js:4076:16)

Found the error.
Another problem. After updating the currency, the count stays the same in the sheet, but the count with updateEqtCount is changed. And is stored on the actor. Could this be a missing link between equipment and the item?

Any idea why the sheet information about item count is not updated, but the updateEqtCount updates the count and is correct. Should I export the actor for you to have a look? @crnormand

Sorry... I haven't had the time to check... but yes.   I can fork your build (just let me know which branch has the new code on it).
Is your process creating a Foundry Item (in GURPS), then added it to the NPC merchant, and then transferring from the merchant to the PC?

Oh fudge.... I have no idea how to build your module.   Sorry, I am not a JS developer (or I wasn't, before I started the GURPS Game Aid), so I don't know a lot of things.
If I have my fork downloaded, I assume I need to rename it to "/merchantsheetnpc'?    And then how do I build it to create the "scripts/merchant.js"?

Hehe.
Yes, I also started by building it in js, but then I wanted to try out typescript so I was closer to the Java I normally develop in.
To build the project you can run
npm install
npm run build
and if you want to have it watched as I do. So I have the dist directory soft-linked into foundry modules. This way every time I change the code I can just reload and see if the change works.
This is done by
npm run build:watch

And the branch I work on is gurps-integration.
I also found out why the count was not updated on the actor. I actually was watching the non linked token to the actor. I missed that the character was not linked to the token.
Now I'm missing the item lookup on the merchant for the sell to merchant. But I hope I will soon get that fixed also.

The last lines of the log after executing npm install:
56 verbose stack Error: ENOENT: no such file or directory, open 'C:\Users\crnor\Documents\FoundryVTT\data\modules\merchantsheetnpc\package.json'
57 verbose cwd C:\Users\crnor\Documents\FoundryVTT\data\modules\merchantsheetnpc
58 verbose Windows_NT 10.0.19044
59 verbose argv "C:\Program Files\nodejs\node.exe" "C:\Users\crnor\AppData\Roaming\npm\node_modules\npm\bin\npm-cli.js" "install"
60 verbose node v16.13.2
61 verbose npm  v8.5.2
62 error code ENOENT
63 error syscall open
64 error path C:\Users\crnor\Documents\FoundryVTT\data\modules\merchantsheetnpc/package.json
65 error errno -4058
66 error enoent ENOENT: no such file or directory, open 'C:\Users\crnor\Documents\FoundryVTT\data\modules\merchantsheetnpc\package.json'
67 error enoent This is related to npm not being able to find a file.
67 error enoent
68 verbose exit -4058
69 timing npm Completed in 5182ms

Ahh I think you have not got the full project checkout.
I normally have the project not directly in the foundry modules. And checkout the project there. there should be a package.json in the root of the merchantsheetnpc folder. Be in the folder and run npm install.
I must admit that I have not tried to build it on a windows. :)

merchantsheetnpc.zip
I have here the newest compiled version, I have nearly fixed it, but the problem is the items found with the findEquipment is not a type that is compatible with the foundry item.


chore: add .editorconfig

As I discussed Prettier handles formatting for us. I am closing this for now.


page 2 cannot be triggered.

After page 1 loaded, if the page still no scrollbar.
Cannot Automatically triggers  page2 ?
html code:
<div infinite-scroll="vuls.vulList.nextPage()"
  infinite-scroll-distance="1"
  nfinite-scroll-disabled='vuls.vulList.busy'
  ng-show="vuls.vulList.items.length>0"
  infinite-scroll-listen-for-event="list:loadmore">
</div>

So I need to add these code in my controller:
// If page 1 loaded still no scorllbar. Trigger loadmore by hand.

if(($(window).height() >= $(document).height()) && !loadedFlag) {
 $scope.$emit('list:loadmore');
}

Your method will ddos, if we assume elements  will have 0 height.
Of course, it can be added as an option, but it is better to use the button ' Load more </ div>' :-)

@uzen With a button will not be an infinite scroll, will be?


Fix Publish Ability

If creating a new event without any attributes attached to it, the API blocks the creation of the event and it will never make its way into MISP. Forcing published=False at this stage fixes the issue.  Once the event has been populated the final publishing will be handled by ln 435.

Does this PR include everything in PR #33?


MBIT_IR not showing up after latest upgrade

Since the yupgrade to makecode, new projects do not bring in the mbit_ir functions when adding this extension.

UPDATE: I got it to work by "RESET" in makecode to delete all projects.

You can also use the makecode version that is on Yahboom's website, attached link. If it doesn't work for you, check out Yahboom's FAQ, attached link. (You need to go down)

You can also use the makecode version that is on Yahboom's website, attached link.

If it doesn't work for you, check out Yahboom's FAQ, attached [link.]((http://www.yahboom.net/study/Building_bit) (You need to go down)


byteswap instructions

It would be nice to have byteswap instructions. They are simple to implement in hardware and ZFS would benefit from it.
When ZFS imports a pool from a system with opposite endianness, it must byteswap data when verifying checksums. The ZFS code has many versions of checksumming functions, which are native endian vs byteswapped, scalar vs simd vs superscalar, and arm vs Intel vs etcetera. The superscalar one is a scalar implementation of the algorithm that enables us to do SIMD. Intel did a write up explaining the math behind how it works:
https://www.intel.com/content/www/us/en/developer/articles/technical/fast-computation-of-fletcher-checksums.html
When SIMD is not available, the superscalar version is always the fastest, but on architectures without byteswap instructions, performance of the byteswap version of the checksum function can be expected to suffer severely.
We generate so many versions of checksumming functions that I have been experimenting with using high level GNU C vector code to generate them all. The following has LLVM/Ciang compile the high level GNU C vector code for RV64GC:
https://gcc.godbolt.org/z/xdedG1Tre
The first function is the function for calculating partial sums for native endian checksums while the second function is the byteswap version. The native endian checksum’s loop only uses 22 instructions. The byteswap version’s loop uses 70 instructions. It will likely operate around 1/3 of the speed at best. Byteswap instructions would restore most of the speed of the native endian version.

I had missed rev8. Nevermind.


Fix subscription name base on topic name

ATTENTION
Please file all new issues in our new repo: https://github.com/GoogleCloudPlatform/spring-cloud-gcp/issues/new/choose.
Issued filed in this repo should only be bug reports for Spring Cloud GCP 1.x (corresponding to Spring Cloud Hoxton and below).

Describe the bug
Please provide details of the problem, including the version of Spring Cloud that you
are using.
Topic in project B
Subscription on project A
App running on project A using Subscription on project A.
using full topic name
projects//topics/
I'm not able to use the Subscription on project A .
// Use <topicName>.<group> as subscription name
			subscriptionName = topicName + "." + group;

projects/<project-id>/topics/<topic-name>.<group> is not a valid subscription name to be created
Sample
If possible, please provide a test case or sample application that reproduces
the problem. This makes it much easier for us to diagnose the problem and to verify that
we have fixed it.
@Override
	public ConsumerDestination provisionConsumerDestination(String topicName, String group,
			ExtendedConsumerProperties<PubSubConsumerProperties> properties)
			throws ProvisioningException {
                // this will work with a topicName like projects/<project-id>/topics/<topic-name>
		Topic topic = makeSureTopicExists(topicName, properties.getExtension().isAutoCreateResources());

		String subscriptionName;
		Subscription subscription;
		if (StringUtils.hasText(group)) {
			// Use <topicName>.<group> as subscription name
                        // FIX sugestion
                        // subscriptionName = (TopicName.isParsableFrom(topicName) ?  TopicName.parse(topicName).getTopic() : topicName) + "." + group;
			subscriptionName = topicName  + "." + group;
			subscription = this.pubSubAdmin.getSubscription(subscriptionName);
		}
		else {
			// Generate anonymous random group since one wasn't provided
			subscriptionName = "anonymous." + topicName + "." + UUID.randomUUID().toString();
			subscription = this.pubSubAdmin.createSubscription(subscriptionName, topicName);
			this.anonymousGroupSubscriptionNames.add(subscriptionName);
		}

https://github.com/spring-cloud/spring-cloud-gcp/blob/71ef73ada647939e7f2cfc51b53fc0f07f31fb43/spring-cloud-gcp-pubsub-stream-binder/src/main/java/org/springframework/cloud/gcp/stream/binder/pubsub/provisioning/PubSubChannelProvisioner.java#L80

Is this the same issue as https://github.com/GoogleCloudPlatform/spring-cloud-gcp/issues/232?
@ttomsu I think we just need to backport that fix.

It looks pretty similar.
@abelmatos Would you be willing to upgrade to 2.0? If not, we can backport the fix, but you'd have to wait for the next Hoxton service release. (the good news is we control the release schedule/cadence in the 2.0+ world)

thanks for the info about the 2.0.
We will try it.
meanwhile we find a temporary workaround.

@ttomsu Any reason not to backport it anyway?

I don't think so - opened https://github.com/spring-cloud/spring-cloud-gcp/pull/2647


useEffectOnce may render two times with React.StrictMode

https://stackoverflow.com/questions/48846289/why-is-my-react-component-is-rendering-twice
Maybe a solution:
import {EffectCallback, useEffect, useRef} from 'react';

function useEffectOnce(effect: EffectCallback) {
  const destroyFunc = useRef<void | any>();
  const calledOnce = useRef(false);
  const renderAfterCalled = useRef(false);

  if (calledOnce.current) {
    renderAfterCalled.current = true;
  }

  useEffect(() => {
    if (calledOnce.current) {
      return;
    }

    calledOnce.current = true;
    destroyFunc.current = effect();

    return () => {
      if (!renderAfterCalled.current) {
        return;
      }

      if (destroyFunc.current) {
        destroyFunc.current();
      }
    };
  }, []);
}

export default useEffectOnce;

I believe that this will only happen in development, not production which is by design with React 18 StrictMode changes.

I believe that this will only happen in development, not production which is by design with React 18 StrictMode changes.

Yes, definitely but this being handled can avoid some problems for use cases of StrictMode in development (even not present in production). If the hook name is useEffectOnce and there is a way to "ignore this strictmode problem" why not?

This is affecting useThrottle too in #2343.

The very reason it happens in dev mode using  in React 18 is because it can happen in production mode. One way to look at it is that without upgrading useEffectOnce, react-use becomes a React 17 library.

Any chance we can get this fix into the library? I know there haven't been updates in a year or so but would love to see it work.
This simple implementation is what I'm using in my code:
  const isLoaded = useRef(false);

  useEffectOnce(() => {
    console.log('(from the hooks lib, this does run twice in strict mode)!');
  });

  useEffect(() => {
    if (isLoaded.current === false) {
      isLoaded.current = true;
      console.log('this only happens once, truly!');
    }


Harmonize community contributions

The main fork has some good/valuable contributions sitting in PR/closed status, or spread across forks.
Make an attempt at actually putting the community contributions together into a single place (ideally: without breaking anything).

Change default time support to be utc instead of local:
https://github.com/raceresult/odbc/commit/a68ad22504f8759f6e77150fd082a5c26640e3fc

Last insert id: https://github.com/What-Color/odbc/commit/433e9c49e8e40be701b422ec1699d7f8a5b31bfb


ROX-24712: Include Add in autocomplete options

Description
This PR adds the "Add" option to autocomplete, allowing you to search for a value even if no suggestions are available.
Screenshots

Screen recording
https://github.com/stackrox/stackrox/assets/4805485/946a633e-fe68-4cc6-81ea-000af6f9661d

/retest


Feature request: make it work in IE

Since such a tool is useful to compare a piece of code in different browsers, I would like to have the ability, that it works in older browsers, too - including IE6 (no, it doesn't have to be beautiful...).
I have huge problems in IE8 (certificate error, "null" in all editable fields (on startup), "content was blocked because it was not signed by a valud security certificate" on run) and it doesn't show anything in IE<=7.
Even IE9 has it's problems (when I don't add tinker.io to trusted sites, it also has the "null" as startup values in all fields - but it works though).
Don't know, maybe a non-https-version can help to fix at least some of these problems?!

It's weird that you get certificate errors though, since I bought two certificates for the domains involved. :/
The idea is that it should work well in IE8+ at some point, but it's not a huge priority for me right now, since most people who visit the site will be developers using a somewhat modern browser. That said, yes, it's super-handy to be able to test code snippets in older browsers so it should definitely do something, but as I said, it's not a huge priority for me, yet. :)

I use a so-called "somewhat modern browser, too - but my customers say, I have to make everything work in older browsers (even IE6), too...
So yes - tinker.io is purely developer-centric (and for them (including me), I think you should not even support IE8).
I guess I'm not alone if I say these developers need to use it for older browsers. But ok, I'm looking forward it.
IE8: I get the following error.
This certificate cannot be verified up to a trusted certification authority
Issued to: tinker.io
Issued by: RapidSSL CA
Valid from 2/7/2012 to 2/9/2013
Thumbprint c3 c6 e7 35 85 74 2b f2 ad fa f1 57 03 8e 07 da b2 c5 24 c0
I'm using IE8 (and all other versions) in a virtual pc session.
The results-iframe shows "Content was blocked because it was not signed by a valid security certificate.". I can see the results if I choose "Allow blocked content" from this yellow-line-information-bar at the top of the content (don't know the name of it). When I choose it, the page reloads, so my content is gone (if not saved yet). But after the reload, it works in IE8, too (somewhat edgily, but without errors).

That's really weird. At some point I'll look into this. Clearly just IE being an asshat.
Also, if your clients want it to work in IE6.. you should probably shoot them in the face. :p

Probably.. Never thought about this reasonable and understated reaction ;)


Accessibility: Keyboard Shortcuts Menu Accessibility and I18N Improvements

Context
Improves translations and accessibility in keyboard shortcuts menu.
Summary
The translation setup in keyboard shortcuts wasn't fully working because of scenarios involving keys in a statement like hold ⌘. Not every language here would put hold before ⌘.
This PR is to create more context around the translations and markup so that statements involving keys are better translated. Also adds kbd element for semantic purposes which helps accessibility.
Relevant Technical Choices
Decreased level of abstraction around rendering and declaring keyboard shortcut definitions. Makes declaring a keyboard shortcut a little more verbose, but much more explicit which removes a lot of logic when rendering. This also benefits translations by giving the translator a higher amount of context like: hold ⌘ instead of hold and ⌘
To-do
NA

User-facing changes
Should be no visual changes, just better translations into other languages and improved accessibility.

Testing Instructions
Verify that there are no visual or functional changes to the keyboard shortcuts popup menu. Beyond that can try out other translations of try a screenreader on it if you so please.

QA

[x] This is a non-user-facing change and requires no QA

This PR can be tested by following these steps:

UAT

[x] UAT should use the same steps as above.

This PR can be tested by following these steps:

Reviews
Does this PR have a security-related impact?

Does this PR change what data or activity we track or use?

Does this PR have a legal-related impact?

Checklist

[x] This PR addresses an existing issue and I have linked this PR to it in ZenHub
[x] I have tested this code to the best of my abilities
[x] I have verified accessibility to the best of my abilities (docs)
[x] I have verified i18n and l10n (translation, right-to-left layout) to the best of my abilities
[ ] This PR contains automated tests (unit, integration, and/or e2e) to verify the code works as intended (docs)
[x] I have added documentation where necessary
[x] I have added a matching Type: XYZ label to the PR

Fixes #7478

When navigating with left/right arrow keys through this, voiceover reads each character. But the ⌘ key (also others) are read, they read a group and then the key. Should it be ignoring the command group?

Okay so looking into this a bit, it seems to happen if there's an aria-label on the key. We can remove the aria-label to remove the group announcement from voiceover, but might lose some of the translation benefits 🤷 . Not sure of other ways to escape that behavior beyond removing the aria-label?
Anyway, happy to go either way with it. Thoughts @samwhale @swissspidy ?

When navigating with left/right arrow keys through this, voiceover reads each character. But the ⌘ key (also others) are read, they read a group and then the key. Should it be ignoring the command group?

Okay so looking into this a bit, it seems to happen if there's an aria-label on the key. We can remove the aria-label to remove the group announcement from voiceover, but might lose some of the translation benefits 🤷 . Not sure of other ways to escape that behavior beyond removing the aria-label?
Anyway, happy to go either way with it. Thoughts @samwhale @swissspidy ?

(Pascal's on PTO right now)
Given we have a follow up for the ability to navigate the popup with a keyboard I'm not too worried about this right now. Voice over is a good first check but almost anyone relying on a screen reader isn't using voice over :)

When navigating with left/right arrow keys through this, voiceover reads each character. But the ⌘ key (also others) are read, they read a group and then the key. Should it be ignoring the command group?

Okay so looking into this a bit, it seems to happen if there's an aria-label on the key. We can remove the aria-label to remove the group announcement from voiceover, but might lose some of the translation benefits 🤷 . Not sure of other ways to escape that behavior beyond removing the aria-label?
Anyway, happy to go either way with it. Thoughts @samwhale @swissspidy ?

Hmm yeah if it would remove the translation benefits then let's keep it. It's okay if it's announced twice.

@littlemilkstudio This now leads to translators having to translate strings like
<lkbd aria-label="%1$s"><cmdOrCtrlSymbol /></lkbd><shiftSymbol /><upSymbol /> or <downSymbol />

Which obviously is super hard to read and prone to making mistakes.
Something like %1$s%2$s or $3$s seems easier, or probably even translateToExclusiveList() could be used.
There's also
Hold <lkbd aria-label="%s"><cmdOrCtrlSymbol /></lkbd>

Which could be turned into
Hold %s

I'll try to whip up a PR.


Update private-clusters.md

Work in progress. There's still more work to be done to fix all of the issues in #1367.

This change is

/assign
/ok-to-test

Preview: https://deploy-preview-1403--competent-brattain-de2d6d.netlify.com/docs/gke/private-clusters/

Thanks for the great updates @sxbfundi!
/lgtm
/approve


Map height inference doesn't work inside VSCode

When viewing inside vscode, I don't see any map unless I manually set _height = 800 😕. I expect this is a side effect from #220 cc @vgeorge

Repro for me is just
from lonboard import Map
m = Map(layers=[])
m

I don't see anything unless I then set
m._height = 800

We might need to do some environment sniffing to determine when to apply certain defaults.


Koan per process

This PR is necessary to build good Agents and Supervisior koans.
It changes the way we run koans from single process for all modules, to a individual process per koan.
That way, anything process that is started with a link will be torn down after the koan.
It also splits executing a koan from running all koans. The former has no dependencies, the latter depends on Display and Options.
A large chunk of credit goes to @marosluuce :+1:
@ukutaht
@Maikon

whoa


Remove libdparse from imports printing functionality

We should probably add a CI tester since now we cannot know whether this patch passes any tests or not.

@lucica28 please squash the commits so that we can merge this.

@edi33416 does this need a rebase to use the latest github actions? @lucica28 you need to squash these anyway, so please re-push.

Awesome, all tests pass!

The first brick has been laid.


adding serdeproperties

Adding serdeproperties support.
Hive SERDEPROPERTIES -- https://spark.apache.org/docs/3.0.1/sql-ref-syntax-ddl-create-table-hiveformat.html

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

12 of 15 (80.0%)  changed or added relevant lines in 3 files are covered.
84 unchanged lines in 2 files lost coverage.
Overall coverage increased (+0.03%) to 87.818%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/parser/mod.rs
7
8
87.5%

src/ast/mod.rs
3
5
60.0%

Files with Coverage Reduction
New Missed Lines
%

src/parser/mod.rs
40
83.34%

src/ast/mod.rs
44
79.7%

Totals

Change from base Build<PHONE_NUMBER>:
0.03%

Covered Lines:
20098

Relevant Lines:
22886

💛  - Coveralls

Thank you @alamb !


Extracting adresses from free text ?

Hi!

Here's how I'm using libpostal
I am wondering if  libpostal can be used in some way to extract adresses from free text. My goal is to make name entity recognition:

parse_address('I am going to the Book Club 100-106 Leonard\
St Shoreditch London EC2A 4RH, United Kingdom to have a dring')
 {"the Book Club 100-106 Leonard St Shoreditch London EC2A 4RH, United Kingdom"}

Thanks a lot

Hey @parisni, I've written about this a few times before, but no, it's not a goal of libpostal to extract addresses. The parser is designed for strings that are already known to be addresses/geographic text (geocoder input, free-text address fields in a database, etc.) and will label every token as an address component. I know a lot of users who are writing scrapers, bots, etc. want this feature, but it's a very different machine learning problem and it would be more difficult to obtain a large, suitable training set in multiple languages (or even in one language for that matter). Libpostal makes use of the fact that there are many high-quality sources of free, openly-licensed addresses with house number/street, etc. split out into components, as well as lat/lon so the other components like city/country, etc. can be inferred through reverse geocoding. We then take the component values (the desired output of our model) and reconstruct the input address, generating several variations, sometimes with random secondary units, different formats, etc.
The major difference in address extraction is the need to model the transition from non-address text to address text and back again (the "I am going to", etc.) As mentioned in #22, it might be possible to generate examples of this using Wikipedia links for formal third-person text, or possibly from something like Facebook posts that are tagged with places for informal text. However, I can easily see that being a 6 month project to get something as high-quality/robust as libpostal's parser, and I don't personally have a use case to justify that kind of time commitment.

Hi @parisni , did ou find a solution to address extraction from raw text ?

@revolunet  yes. we annotated a corpus of 2k discharge summaries. And then trained a neural nerwork. https://github.com/strayMat/tag_serve
the f-mesure is .97 on adresses.
Then libpostal can be used to extract meaning-full information

Thanks for sharing ;)
So you manually tagged addresses in 2k documents  ? Why not use openaddresses or similar ?
Is the result sharable ?

As a noob in NER i'd love to understand how to achieve this starting with a manually crafted dataset. would you mind sharing the process ?


Upgrade to podman breaks --docker feature

There may be a problem with how podman interprets the --user argument to docker run. The tempdir being mapped in (as in /tmp/publish-kBywA9 or whatever) isn't readable by the child docker process, resulting in latexmk failing.

[this is as encountered on upgrading a user's workstation from Fedora 30 to Fedora 31; since they upgraded to CGroups 2 the docker binary from Docker, Inc is no longer supported. podman is intended to be a command-line compatible drop-in replacement but as it runs as user (not as root) there appear to be some permission issues]


[IOTDB-1350]FIx log Compression time

Logs are compressed every hour. There are many compressed packages in a day, and the size of each compressed package is small. Therefore, logs are compressed every day.

@neuyilan @mychaow @qiaojialin @jixuan1989   hi , could you please review?
Thanks.


random segfault when running with luajit

Hi,
We have been having this issue for awhile. 90% of the time, it segfaults at the end of the test, inside luajit's lua_close function (most of the time it dies at lj_gc_freeall).  Does busted do multi-threading at all?

Reduced testsuite reproducing the issue: put this into spec/1_spec.lua and spec/2_spec.lua, run busted spec.
for i = 1, 1000 do
  loadstring([[
    for j = 1, 20 do
      local x = {0LL}
    end
  ]])()
end

Interesting, It could be a bug in LuaJIT. But we have been running the same code on many real devices for a long time any never ran into this issue. It only happens quiet frequently when running with busted.

we have seen this as well on Travis and CircleCI, where tests crashed at the end (busted with LuaJit). Probably warrants a bug report...

Sometimes the tests even crash during the execution. But this happens way less frequent.

:+1: I am experiencing this issue as well. In my case it occurs maybe 20% of the time.

did someone report this on the LuaJit mailing list?

I can report it to the list. @Tieske , just to confirm, busted is not doing multi-threading right?

No, it isn't. Please do report, otherwise I will, just wanted to make sure we don't report it multiple times.

I have sent the report to the mailing list.

:+1:
For the record, the reported issue; http://www.freelists.org/post/luajit/random-segfaults-when-running-tests-with-busted

Actually, given that I can reproduce the issue with busted 2.0.rc7 but not 2.0.rc6 this could be an issue in lua-term, a C dependency added in rc7.

Nevermind, segfault is introduced in @b3993e6.

@mpeterv what version of luajit are you using?

@houqp I reproduced the issue using busted with testcase I posted above with luajit 2.0.0, 2.0.4, head of master branch, head of v2.1 branch. I've also reduced the testcase (the whole blob with busted, luarocks, penlight and other dependencies) to one file. I'll finish some more testing with different compilation flags and post it to luajit list.

So, as expected, this is a duplicate of #369. Some workarounds are posted there.

Hm... I think the problem now is the discussion in #369 is not documented. @Tieske is it possible to get this documented on the official documentation page?
For us, the workaround does not work. Most of our modules requires ffi, and it's not easier to put all of them into a helper.lua script. A cleaner way to fix it is to run each test in its own process...

@houqp it should be enough to require just the FFI module in helper.lua, ignoring any other modules depending on it.

@mpeterv I tried that, but got a bunch of redefined error in the cdef call 1, which is imported by some of the tests.

@houqp so busted drops and reloads that module, too. So, yes, you'll need to add all modules that can't be safely reloaded in helper.lua.

That's why I said this workaround is not practical for us ;) Basically all our modules depends on FFI, that means we will have to put all of them in helper.lua...

As a workaround, I've had good results running busted with the option --no-auto-insulate, e.g.
busted --lua=luajit --no-auto-insulate tests

This does sacrifice the test insulation feature of busted, but it seems to prevent the crash caused by the ffi module from being unanchored between tests.

@vsergeev thanks! I can also confirm adding --no-auto-insulate fixes all our random segfaults :)

I suggest we add this to the documentation.

Alongside the documentation, I suggest we add a list of default preloaded libraries. We add common things that don't like to be reloaded, like ffi, by default, and allow users to add modules either in helper.lua or at the command line.

yep, that will work too, as long as the behavior is documented :)

for reference, a possible fix: https://github.com/Olivine-Labs/luassert/pull/150

closing this, reloading ffi has been resolved


Do we support UnixSocket forwarding?

Currently, "Clusters" -> "Address" should be a URL like "http://www.example.com".
So I want to ask that could this filled with something like "path/to/unix_socket"?

Not currently. @MihaZupan could they do this themselves using the SocketsHttpHandler.ConnectCallback?
https://github.com/dotnet/runtime/blob/26c045b556010bb68fc21722ed6bf76683028d79/src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/SocketsHttpHandler.cs#L398

Yes, it should work with the ConnectCallback
Some (untested) code for example:
services.AddReverseProxy()
    .LoadFromConfig(_configuration.GetSection("ReverseProxy"))
    .ConfigureHttpClient((context, handler) =>
    {
        if (context.NewMetadata.TryGetValue("UnixSocket", out var unixSocket))
        {
            handler.ConnectCallback = async (context, cancellation) =>
            {
                var socket = new Socket(AddressFamily.Unix, SocketType.Stream, ProtocolType.IP) { NoDelay = true };
                try
                {
                    await socket.ConnectAsync(new UnixDomainSocketEndPoint(unixSocket), cancellation);
                    return new NetworkStream(socket, ownsSocket: true);
                }
                catch
                {
                    socket.Dispose();
                    throw;
                }
            };
        }
    });

And add the UnixSocket to the cluster's metadata:
"MetaData": {
  "UnixSocket": "path/to/unix_socket"
}

Nice.

And add the UnixSocket to the cluster's metadata:

You mean the destination's metadata?

@MihaZupan
I am thinking since YARP can be configured HttpClient, that WebHost can not be configured HttpClient?
Because I see that ConnectCallback is required to return a Stream, so can a MemoryStream be passed from Kestrel in the same process?
My application scenario is a program with multiple Kestrel are responsible for different Web, the same program’s YARP in front of the responsible forwarding. UnixSocket is a good solution but not the best, so I asked.

My application scenario is a program with multiple Kestrel are responsible for different Web, the same program’s YARP in front of the responsible forwarding. UnixSocket is a good solution but not the best, so I asked.

If YARP is in the same process as the web server, then YARP routes will combine with ASP.NET routes that are also configured against the same endpoint, so they don't need to be routed through YARP. AFAIK you could also delegate a request to ASP.NET to handle it if you need it to.

Maybe there is something wrong with my description.
In my case, there are multiple IHosts listening to different endpoints in a .NET VM. This is a bit like Web Hosting (cPanel).
So this obviously needs a proxy.
I'll try to find a better solution, thanks, guys.


Add hint in Docs to activate the YouTube API before usage

🚨 Problem:
Running the application I encountered a HTTP 403 Error saying the YouTube API hasn't been activated:

YouTube Data API v3 has not been used in project xxxxxxxxxx before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/youtube.googleapis.com/overview

📝 Context:
As a user unfamiliar with Google Cloud Services, it was not immediately apparent that YouTube API activation is a prerequisite.
📌 Proposed Documentation Update:
In the README, add a point 1.1 "Activate YouTube API" with the link below.
📦 Additional Information:

Google Cloud - Activate YouTube API: https://console.cloud.google.com/apis/library/youtube.googleapis.com

Thanks for your feedback. Will add the YouTube API activation link.


⚠ Bofrost.lu has degraded performance

In c7fe731, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 881 ms

Resolved: Bofrost.lu performance has improved in 42ea135.


Expression-based interaction between RASSF4 & Chronic childhood arthritis

Laura from Expression Atlas got in touch: "I can't find any  evidence in Open Targets between RASSF4 gene and that chronic childhood arthritis disease. I was expecting to see it as we have an experiment in Expression Atlas  showing a high up-regulation of that gene in that disease.
Do you know why this evidence is missing in Open Targets?"

Summary: The evidence string for this association could not be validated in the 18.08 (and previous) release(s) because the disease had an Orphanet ID (Orphanet_92). This association should appear in the 18.10 release because the disease now has an EFO ID (EFO_0002609).
Investigating this using the Expression Atlas evidence file that was used for 18.08:
gsutil cat gs://otar010-atlas/cttv010-2018-02-14.json.gz\* | gunzip | grep http://identifiers.org/ensembl/ENSG00000107551 >atlas_RASSF4_18.08.txt

The association is given in the evidence string below between:
http://identifiers.org/ensembl/ENSG00000107551 = RASSF4
&
http://www.orpha.net/ORDO/Orphanet_92 = juvenile idiopathic arthritis
{"type":"rna_expression","validated_against_schema_version":"1.2.8","literature":{"references":[{"lit_id":"http://europepmc.org/abstract/MED/26387944"}]},"access_level":"public","unique_association_fields":{"geneID":"http://identifiers.org/ensembl/ENSG00000107551","study_id":"http://identifiers.org/gxa.expt/E-GEOD-71595","comparison_name":"'juvenile idiopathic arthritis' vs 'normal' in 'none'"},"evidence":{"confidence_level":"medium","log2_fold_change":{"percentile_rank":99,"value":4.5},"reference_replicates_n":4,"evidence_codes":["http://purl.obolibrary.org/obo/ECO_0000357"],"date_asserted":"2016-04-22T14:34:07Z","reference_sample":"none; normal","organism_part":"http://purl.obolibrary.org/obo/UBERON_0001090","resource_score":{"type":"pvalue","value":4.1e-13,"method":{"description":"DESeq2 version: 1.10.1"}},"unique_experiment_reference":"STUDYID_E-GEOD-71595","test_replicates_n":3,"is_associated":true,"urls":[{"nice_name":"ArrayExpress Experiment overview","url":"http://identifiers.org/arrayexpress/E-GEOD-71595"},{"url":"http://www.ebi.ac.uk/gxa/experiments/E-GEOD-71595?geneQuery=ENSG00000107551","nice_name":"Gene expression in Expression Atlas"},{"url":"http://www.ebi.ac.uk/gxa/genes/ENSG00000107551","nice_name":"Baseline gene expression in Expression Atlas"}],"experiment_overview":"RNA-sequencing of cells derived from the site of inflammation of Juvenile Idiopathic Arthritis patients","comparison_name":"'juvenile idiopathic arthritis' vs 'normal' in 'none'","provenance_type":{"database":{"id":"Expression_Atlas","version":"prod.22"}},"test_sample":"none; juvenile idiopathic arthritis"},"sourceID":"expression_atlas","disease":{"biosample":{"id":"http://purl.obolibrary.org/obo/UBERON_0001090","name":"synovial fluid"},"id":"http://www.orpha.net/ORDO/Orphanet_92"},"target":{"id":"http://identifiers.org/ensembl/ENSG00000107551","activity":"http://identifiers.org/cttv.activity/unknown","target_type":"http://identifiers.org/cttv.target/transcript_evidence"}}
http://www.orpha.net/ORDO/Orphanet_92
This evidence string fails validation because the disease could not be mapped:
          "explanation": {
            "invalid_disease": "http://www.orpha.net/ORDO/Orphanet_92",
            "gene_error": false,
            "disease_error": true,
            "gene_mapping_failed": false
          },

Investigating this using the latest Expression Atlas evidence file:
gsutil cat gs://otar010-atlas/cttv010-2018-09-27\* | gunzip | grep http://identifiers.org/ensembl/ENSG00000107551 >atlas_RASSF4.txt

This reveals the evidence string below among the 48 evidence strings for RASSF4 describing an association between:
http://identifiers.org/ensembl/ENSG00000107551 = RASSF4
&
http://www.ebi.ac.uk/efo/EFO_0002609 = juvenile idiopathic arthritis
{"unique_association_fields":{"study_id":"http://identifiers.org/gxa.expt/E-GEOD-71595","geneID":"http://identifiers.org/ensembl/ENSG00000107551","comparison_name":"'juvenile idiopathic arthritis' vs 'normal' in 'none'"},"type":"rna_expression","sourceID":"expression_atlas","validated_against_schema_version":"1.2.8","access_level":"public","target":{"activity":"http://identifiers.org/cttv.activity/increased_transcript_level","id":"http://identifiers.org/ensembl/ENSG00000107551","target_type":"http://identifiers.org/cttv.target/transcript_evidence"},"literature":{"references":[{"lit_id":"http://europepmc.org/abstract/MED/26387944"}]},"evidence":{"resource_score":{"type":"pvalue","method":{"description":"DESeq2 version: 1.10.1"},"value":4.1e-13},"evidence_codes":["http://purl.obolibrary.org/obo/ECO_0000357"],"organism_part":"http://purl.obolibrary.org/obo/UBERON_0001090","reference_sample":"none; normal","urls":[{"url":"http://identifiers.org/arrayexpress/E-GEOD-71595","nice_name":"ArrayExpress Experiment overview"},{"url":"http://www.ebi.ac.uk/gxa/experiments/E-GEOD-71595?geneQuery=ENSG00000107551","nice_name":"Gene expression in Expression Atlas"},{"nice_name":"Baseline gene expression in Expression Atlas","url":"http://www.ebi.ac.uk/gxa/genes/ENSG00000107551"}],"test_sample":"none; juvenile idiopathic arthritis","provenance_type":{"database":{"version":"prod.23","id":"Expression_Atlas"}},"test_replicates_n":3,"unique_experiment_reference":"STUDYID_E-GEOD-71595","reference_replicates_n":4,"comparison_name":"'juvenile idiopathic arthritis' vs 'normal' in 'none'","experiment_overview":"RNA-sequencing of cells derived from the site of inflammation of Juvenile Idiopathic Arthritis patients","log2_fold_change":{"percentile_rank":99,"value":4.5},"is_associated":true,"confidence_level":"medium","date_asserted":"2016-04-22T14:34:07Z"},"disease":{"id":"http://www.ebi.ac.uk/efo/EFO_0002609","biosample":{"name":"synovial fluid","id":"http://purl.obolibrary.org/obo/UBERON_0001090"}}}
This evidence string should pass validation for the 18.10 release.


Fix typo (change 'n' to 'k' in get_dummies documentation)

Just changing an n to a k.

Current coverage is 85.25% (diff: 100%)

Merging #14153 into master will not change coverage

@@             master     #14153   diff @@
==========================================
  Files           139        139
  Lines         50502      50502
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
  Hits          43058      43058
  Misses         7444       7444
  Partials          0          0

Powered by Codecov. Last update 900ae6b...d725192

Thanks!


Update JAX's custom call shim's ABI to use the new FFI interface

Currently the external functions don't use the new FFI interface. Once they are ported to use the FFI interface, the custom call

@vmurali Stale issue here with no priority - can you update?

I have moved to working on cublas support before I come back to this


Upsert for POST /sounds and POST /photos

Currently POST /observation_photos and POST /observation_sounds upsert records based on the provided UUID attribute, i.e. if a client posts an obs photo with a given UUID and the request suceeds but the client never receives a response, when the client tries to post that same obs photo again with the same UUID, the server will not create a duplicate record and will instead update the existing record with the same UUID. This helps avoid the creation of duplicate media/observation associations.
However, POST /photos and POST /sounds do not support this functionality, presumably because our own mobile client applications have not made much use of those endpoints. iNaturalistReactNative uploads media does use these endpoints like the web does, i.e. it posts photos and sounds before posting observation photo and observation sound associations. Since iNaturalistReactNative is a mobile app and will be making a lot of requests in less stable network conditions, upsert / dedupe functionality seems desirable.

[x] POST /photos should find and update an existing photo matching the photo[uuid] attribute instead of creating a new record if such a photo exists, otherwise it should create a new record
[x] POST /sounds should find and update an existing sound matching the sound[uuid] attribute instead of creating a new record if such a sound exists, otherwise it should create a new record
[x] The equivalent endpoints in the API should behave the same way

should be working in production now


Meet 'FileNotFoundError' on git-bash/Windows 10

Environment
Windows 10 64-bit
Python 3.7
Git for Windows 2.25.0 64-bit
manifest-tool 1.5.2
Description
I am running Pelion E2E tests. For non-update test, it is passed:
pytest TESTS/pelion-e2e-python-test-library/tests/dev-client-tests.py \
-k test_01_get_device_id

For update test, I meet FileNotFoundError.
pytest TESTS/pelion-e2e-python-test-library/tests/dev-client-tests.py \
-k test_06_update_device \
--update_bin=/C/CCLi8/IoT/mbed/mbed-examples/mbed-os-example-pelion/BUILD/NUMAKER_PFM_NUC472/ARMC6/mbed-os-example-pelion_update.bin \
--manifest_tool=/C/CCLi8/IoT/mbed/mbed-examples/mbed-os-example-pelion

Per preliminary check, the update image upload step has succeeded. The failure point is in manifest-tool create. I check my manifest-tool installation directory and find there's no manifest-tool.exe (only manifest-tool python script) there. I am not sure if this is the cause on Windows environment.
Log
=================================== ERRORS ====================================
___________________ ERROR at setup of test_06_update_device ___________________

cloud = <pelion_test_lib.cloud.cloud.PelionCloud object at 0x000001701AAC6C50>
client = <pelion_test_lib.tools.client_runner.Client object at 0x000001701B3BDC50>
request = <SubRequest 'update_device' for <Function test_06_update_device>>

    @pytest.fixture(scope='function')
    def update_device(cloud, client, request):
        """
        Fixture for updating device.
        This uploads firmware image, creates manifest, uploads manifest, and creates and starts the update campaign.
        :param cloud: Cloud fixture
        :param client: Client fixture
        :param request: Requests fixture
        :return: Campaign ID
        """
        binary_path = request.config.getoption('update_bin', None)
        manifest_tool_path = request.config.getoption('manifest_tool', os.getcwd())
        no_cleanup = request.config.getoption('no_cleanup', False)
        delta_manifest = request.config.getoption('delta_manifest', None)
        log.info('Update image: "{}"'.format(binary_path))
        log.info('Path for manifest-tool init: "{}"'.format(manifest_tool_path))
        if not binary_path or not manifest_tool_path:
            skip_msg = 'Provide missing binary image in startup arguments to run update case\n' \
                       '--update_bin={}'.format(binary_path if binary_path else 'MISSING!')
            log.warning(skip_msg)
            pytest.skip(skip_msg)

        sleep(5)
        resp = cloud.device_directory.get_device(client.endpoint_id(), expected_status_code=200).json()
        assert resp['state'] == 'registered'

        fw_image = cloud.update.upload_firmware_image(binary_path, expected_status_code=201).json()
        fw_image_id = fw_image['id']
        log.info('Firmware image uploaded!

        if not isinstance(args, str):
            args = list2cmdline(args)

        # Process startup details
        if startupinfo is None:
            startupinfo = STARTUPINFO()
        else:
            # bpo-34044: Copy STARTUPINFO since it is modified above,
            # so the caller can reuse it multiple times.
            startupinfo = startupinfo._copy()

        use_std_handles = -1 not in (p2cread, c2pwrite, errwrite)
        if use_std_handles:
            startupinfo.dwFlags |= _winapi.STARTF_USESTDHANDLES
            startupinfo.hStdInput = p2cread
            startupinfo.hStdOutput = c2pwrite
            startupinfo.hStdError = errwrite

        attribute_list = startupinfo.lpAttributeList
        have_handle_list = bool(attribute_list and
                                "handle_list" in attribute_list and
                                attribute_list["handle_list"])

        # If we were given an handle_list or need to create one
        if have_handle_list or (use_std_handles and close_fds):
            if attribute_list is None:
                attribute_list = startupinfo.lpAttributeList = {}
            handle_list = attribute_list["handle_list"] = \
                list(attribute_list.get("handle_list", []))

            if use_std_handles:
                handle_list += [int(p2cread), int(c2pwrite), int(errwrite)]

            handle_list[:] = self._filter_handle_list(handle_list)

            if handle_list:
                if not close_fds:
                    warnings.warn("startupinfo.lpAttributeList['handle_list'] "
                                  "overriding close_fds", RuntimeWarning)

                # When using the handle_list we always request to inherit
                # handles but the only handles that will be inherited are
                # the ones in the handle_list
                close_fds = False

        if shell:
            startupinfo.dwFlags |= _winapi.STARTF_USESHOWWINDOW
            startupinfo.wShowWindow = _winapi.SW_HIDE
            comspec = os.environ.get("COMSPEC", "cmd.exe")
            args = '{} /c "{}"'.format (comspec, args)

        # Start the process
        try:
            hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
                                     # no special security
                                     None, None,
                                     int(not close_fds),
                                     creationflags,
                                     env,
                                     os.fspath(cwd) if cwd is not None else None,
>                                    startupinfo)
E                                    FileNotFoundError: [WinError 2] The system cannot find the file specified

c:\python37\lib\subprocess.py:1178: FileNotFoundError

Hi @ccli8!
Were you able to run manifest-tool init before starting the pytest?

@janisimonen , on my git-bash/Windows 10, I can run manifest-tool init separately and successfully. Actually, before the pytest, I've run mbed dev-mgmt init, which includes manifest-tool init.

Okey, interesting. I'll try to find a Win machine and try to reproduce this situation here.

pelion_e2e_test.log
same issue observed when running this test for DISCO_L496AG + STMod BG96

Hi @ccli8 and @soleilplanet!
Looks like the (pip) installed manifest-tool is missing from the Window' systems PATH - if it's not found by the pytest. So, if you can check where the manifest-tool is installed and then add that folder to env PATH - e.g like this: https://www.architectryan.com/2018/03/17/add-to-the-path-on-windows-10/

@janisimonen I am sure manifest-tool has installed in my Windows. Checking my python installation path C:\Python37, I find the file manifest-tool is placed under C:\Python37\Scripts, but no manifest-tool.exe. Actually, I find no manifest-tool.exe in my Windows.

@ccli8 Looks like manifest-tool has issue on pip installation - I just noted the same behaviour. So, you need to clone the manifest-tool repo and install the local copy on Windows environment to have executable where it should be.
https://github.com/ARMmbed/manifest-tool/issues/11

@janisimonen The workaround works. I find manifest-tool.exe under C:\Python37\Scripts folder now. Thanks for your information.

@ccli8 Good to hear - thanks!


Theta Tech

Add
Theta Tech
We are an IT company based in Pokhara, offering graphic design, web design, and digital marketing services. Our team works hard to create designs, websites, and strategies that help businesses grow.
Soon, we’ll be adding more services to serve you even better. Let’s work together to bring your ideas to life!

@th3ta-tech Please open a pull request instead of an issue. Happy to merge!


Session file driver error

A PHP Error was encountered
Severity: Warning
Message: touch() [function.touch]: Unable to create file session\ci_session9f38398e905f372fd39b7e267bbfbb5b8ebd75ba because No such file or directory
Filename: drivers/Session_files_driver.php
Line Number: 234
Backtrace:

#3664


unable to test new component locally within typescript example application

@akashyeole Could you please provide more information in the description, such as how steps to reproduce, working environment (where you are running it, and how), and any other relevant information.

@tishoyanchev I guess react-typescript example is simply outdated.
When I run npm run dev in my console I have Uncaught SyntaxError: ambiguous indirect export: IfxNavbarItem
And if you open Navbar.tsx you see all those errors highlighted.

@tishoyanchev
This issue can be marked as resolved as the typescript applications are working fine now.

Resolved


Consider making JibContainerBuilder interface more fluent.

Continuing from #881...
Considering that we are modeling JibContainerBuilder's interface off of the Stream API, and that JibContainerBuilder begins with Jib.from, we should probably make the rest of the API more fluent as well.
addLayer(List<Path>, String) -> layer(List<Path>, String)
addLayer(LayerConfiguration) -> layer(LayerConfiguration)
setLayers(List<LayerConfiguration>) -> layer(List<LayerConfiguration>)
setEntrypoint(List<String>/String...) -> entrypoint(List<String>/String...)
setProgramArguments(List<String>/String...) -> programArguments(List<String>/String...)
setEnvironment(Map<String, String>) -> environment(Map<String, String>)
setEnvironmentVariable(String, String) -> environmentVariable(String, String)
setExposedPorts(List<Port>/Port...) -> expose(List<Port>/Port...)
addExposedPort(Port) -> expose(Port)
setLabels(Map<String, String>) -> label(Map<String, String>)
addLabel(String, String) -> label(String, String)
Also, currently, the collection setters (like setExposedPorts) replace previously-set ports. If converting to use this more fluent API, these collection setters should append, not replace.

We could provide exposeOnly(Map<>) variants that take a Map?

Should we close this? Haven't had any discussions about it in a while (personally I like the explicit API over the fluent one).

@loosebazooka recently turned appengine-plugins-core to the more fluent approach.  I think it looks better, but I don't feel strongly.

I just made everything a builder: ¯_(ツ)_/¯

You also did setX() → x()

Closing and sticking with current design since it is clearer in the functions' intentions, especially for methods like addExposedPort vs setExposedPort (expose vs exposeOnly is less clear and still involves extra Only portion).


[FEATURE] Implementation of Language model estimator

Description
Implementation of word language model estimator and large rnn language model estimator
Checklist
Essentials

[x] PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
[x] Changes are complete (i.e. I finished coding on this PR)
[ ] All changes have test coverage
[ ] Code is well-documented

Changes

[ ] Feature1, tests, (and when applicable, API doc)
[ ] Feature2, tests, (and when applicable, API doc)

Comments

If this change is a backward incompatible change, why must this change be made.
Interesting edge cases to note here

cc @dmlc/gluon-nlp-team

Job PR-1155/1 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-1155/1/index.html

Job PR-1155/2 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-1155/2/index.html

Job PR-1155/4 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-1155/4/index.html

Job PR-1155/5 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-1155/5/index.html

Job PR-1155/6 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-1155/6/index.html

Job PR-1155/7 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-1155/7/index.html

Job PR-1155/8 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-1155/8/index.html

Job PR-1155/9 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-1155/9/index.html

Job PR-1155/10 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-1155/10/index.html


Copy edits, markdown formatting

SUMMARY

Some copy changes, making it a little more open to new plugins. Also lots of markdown auto-formatting because ¯\_(ツ)_/¯
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TEST PLAN

ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Changes UI
[ ] Requires DB Migration.
[ ] Confirm DB Migration upgrade and downgrade tested.
[ ] Introduces new feature or API
[ ] Removes existing feature or API

Codecov Report

Merging #251 into readme_plugins will decrease coverage by 10.67%.
The diff coverage is n/a.

@@                 Coverage Diff                 @@
##           readme_plugins     #251       +/-   ##
===================================================
- Coverage           70.15%   59.47%   -10.68%
===================================================
  Files                 598      404      -194
  Lines               31934    13112    -18822
  Branches             3223     3223
===================================================
- Hits                22402     7799    -14603
+ Misses               9425     5132     -4293
- Partials              107      181       +74

Flag
Coverage Δ

#cypress
?

#javascript
59.47% <ø> (ø)

#python
?

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c95374e...a99418a. Read the comment docs.


Forever Oculus Loading, But Steam VR Launched Automatically

I use a Quest 2, and I've just gotten started with trying to get killer to work out. No success: I'm seeing a loading screen that lasts forever. Although steam VR does launch, but it does so with an error saying that "Something's Wrong" with an error type 303.
Hope this helps Kaitlyn!

I'll investigate this, thank you!


Refactor AppHeaderComponent to ClueAppHeaderComponent

This is the master side of the refactor done in the dataflow branch in #602.

@mklewandowski Merging pre-review in the interest of moving things along. Feel free to review/comment post-merge.


Fits: Produced datasets table is not prefilled with uploaded files

Describe the bug

The CSV_example.csv was uploaded via the file upload option, but it has no datatype.

When exported, the format and type are not prefilled.
Is this a regression?
No.
Steps to Reproduce

Create a new DMP + project.
In step 3, choose Upload sample data and uplaod a csv file.
Export the dmp.

Expected behavior
A clear and concise description of what you expected to happen.
Environment

Chrome Version 120.0.6099.217
Damap version 3.0

Fits works as intended - issue is closed since it is not needed.


feature: add FMOD

FMOD can play some tracker formats
https://www.fmod.com/core

I'm not sure it's worth support fmod which is primary (these days) targeting other thing than tracker formats.

I just thought for hearing differences between all the libraries.
I hear a lot of differences.  But you decide if you add it
Read here for example :)
https://hydrogenaud.io/index.php?topic=119449.0

Tracker formats in case of xm, mod, etc doesn't include (for most cases that I know of at least) any soundchip emulation. In the case of adplug that just deals with that then it totally makes sense. It's the same for SID (C64) and and other soundchip based emulation.

Don't think it makes a lot of sense to do this so will close it for now


Refactor module imports and file structure

This pull request includes refactoring of module imports and file structure to improve code organization and maintainability. The changes ensure that the correct types are used for path parameters and allow for saving the diagram with a string path. Additionally, tests have been added to verify the saving functionality with both Path and string paths.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 98.15%. Comparing base (1194440) to head (11f75da).
Report is 5 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #163      +/-   ##
==========================================
- Coverage   98.78%   98.15%   -0.64%
==========================================
  Files          45       32      -13
  Lines        1893      867    -1026
==========================================
- Hits         1870      851    -1019
+ Misses         23       16       -7

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


npm run preview generate the following error

error when starting dev server:
Error: The following dependencies are imported but could not be resolved:
@/hooks/useFocusIdx (imported by  /Email-Builder/example/pages/Editor/components/CustomBlocks/ProductRecommendation/Panel/index.tsx)
Are they installed?
at optimizeDeps (\Email-Builder\node_modules\vite\dist\node\chunks\dep-c1a9de64.js:73153:15)
at processTicksAndRejections (internal/process/task_queues.js:93:5)
at async runOptimize (Email-Builder\node_modules\vite\dist\node\chunks\dep-c1a9de64.js:73827:48)
at async Server.httpServer.listen (Email-Builder\node_modules\vite\dist\node\chunks\dep-c1a9de64.js:73843:21)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>preview: npm run lib && vite --config vite.preview.config.ts
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>preview script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

The cause of the bug is the wrong import, fixed now.


Source control  panel turned extremly slow

Type: Bug
When click on Source control, panel take too long to load. Specially Commit button.

Did you try the setting "scm.showHistoryGraph": false that the release notes mention?

You just saved me at least 90 seconds of wait this week, and the hassle of reading documentation!

/duplicate

Did you try the setting "scm.showHistoryGraph": false that the release notes mention?

You just saved me at least 90 seconds of wait this week, and the hassle of reading documentation!

Where do i find this setting?

@5starcoders it was removed in 1.93 when the graph got its own view, which you can collapse by clicking its > icon or hide completely from its titlebar context menu.


Closes #10 Adding vars as a template_field to DbtBaseOperator

Adding vars to the list of template_fields on the DbtBaseOperator.
This will allow passing variables to the operator from the dag_run.conf

Looks good to me, let's wait and see what the tests say before merging. I'll probably push this out in version 0.12, and leave the custom backend code for 0.13 (as that is a much bigger and riskier change).
Important to note that this probably doesn't work with Airflow 1.x or 2.0 as support for Jinja's NativeEnvironmentwas added in 2.1, and we expect vars to already be a dict. Eventually, we can work that in a different issue.

Thanks @tomasfarias!


Fix CLI inconsistencies #1320

Description
Fix CLI inconsistencies:

[ ] Remove data-dir option when not necessary
[x] Fix the display order of some commands (1. config, 2. index, 3. source/split, 4. other options)
[x] Make required options such as config or index actually required
[ ] Set default values with default_value
[x] Add aliases (ls, rm, ...) whenever possible

How was this PR tested?
Try local CLI + run tests
Content

Fix the display order of some commands (1. config, 2. index, 3. source/split, 4. other options)
Add aliases (ls for list, del for delete, desc for describe)
Remove data-dir when not necessary

close #1320

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@kalil-pelissier thank you Kalil.
I made the PR "draft" since it seems like it is a work in progress.
If you want to have it merged independently, that's even better. Just switch it to "Ready for Review", and edit the PR description with the actual content of the PR.
We require contributors to sign a CLA electronically. The process is rather fast.
You work is still under the AGPL license, but it gives Quickwit the right to also emit commercial licenses.

data_dir is only necessary for commands such as index ingest, index merge, index  search that write intermediates data structure (index files, caches) `.
Let's no worry about setting default values for now.

@guilload I think I'm missing something. I search where data_dir is not used but each time this option is given to the load_quickwit_config function to generate a config:
    let quickwit_config = load_quickwit_config(&args.config_uri, args.data_dir).await?;

I don't think that I should remove the option in the subcommand and let a default value be set?

load_quickwit_config(&args.config_uri, None) is fine. We already do that for a bunch of commands.

Ok so for example the index create command don't need data-dir option?

Merged, congrats on your first contribution @kalil-pelissier 🎉


WIP: Segregation model example

Did some small refactoring:

Simulation is now an interface with two implementations to use.
Provide some utility classes like Grid class and identifier generators
Observers now get to run after initialization but before the simulation starts

(Oh, and WIP means "work in progress" so please do not merge)

Pull Request Test Coverage Report for Build 16

49 of 79 (62.03%) changed or added relevant lines in 5 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage increased (+1.5%) to 58.989%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

dworp/observer.py
7
10
70.0%

dworp/agent.py
6
13
46.15%

dworp/environment.py
21
28
75.0%

dworp/simulation.py
9
22
40.91%

Files with Coverage Reduction
New Missed Lines
%

dworp/simulation.py
1
28.57%

Totals

Change from base Build 15:
1.5%

Covered Lines:
105

Relevant Lines:
178

💛  - Coveralls


Another openjdk has been released

I found a new openjdk when searching to already compiled versions of the original openjdk, and then i found the AdoptOpenJDK. Maybe now the boardwalk-2.0 project can go forward:
https://adoptopenjdk.net/releases.html
https://github.com/AdoptOpenJDK/openjdk8-binaries/releases
(in the case of the android for those who dont know,it would be the arm32 or aarch64 builds)
i will try this tomorrow :D

No android


Migration versions and naming

One more from me and @cosgroveb. ;)
Rails allows us to version migrations outside of the 14-digit version name convention, so we want our Cassandra migrations to have the same flexibility.  This pull requests adds that flexibility, and also provides helpful error messages in the case of a mis-named migration file.  We have also added some testing surrounding the parsing and execution of migration names.
Thanks again, and cheers!

:shipit:

Thanks! Looks good.


Consider transitioning windows-implement and windows-interface to use syn@2

Hi! I noticed that windows-implement and windows-interface (and possibly other parts of this repository) depend on syn@1. Perhaps you could consider updating to use syn@2 wherever possible?
Some things worth noting for anyone who intends to take on this suggestion:

syn@2's MSRV is Rust 1.56, up from syn@1's Rust 1.31. This shouldn't be a problem, since both windows-implement and windows-interface specify a MSRV of 1.64 in their respective Cargo.toml files.
Updating to syn@2 may not be as simple as swapping the version number - there are a number of breaking changes in syn@2, listed at https://github.com/dtolnay/syn/releases/tag/2.0.0.

If you do consider to transition to syn@2, you have my thanks! :)

I'd like to make the switch but yes, I took a quick look and there are quite a few breaking changes that I'll have to figure out first.


Shrink DNSMap if DNS not enabled to save memory

shrink dnsmap to 1 entry when the feature is not enabled to optimize memory usage

Note: I have to move code out of FlowFetcher function because lint hit complexity limit in that function

/ok-to-test

/approve


143 move weekly dose inside knowledge cafe

#143 Moved weekly dose module inside knowledge cafe
updated Home view and Knowledge Cafe view
moved weekly doses controller, model, view inside knowledge cafe
updated namespaces accordingly
updated the weekly dose api

@rathorevaibhav please have a look at this PR in review.

@rathorevaibhav corrections done. please reviews.


Delay

The Delay performance metric is the amount of additional time spent by the vehicles on a section of road due to congestion. This is the difference between the travel time at a non-congestion speed and the current speed. The congestion, or threshold, speed is usually 35MPH but in PeMS we compute the delay for a number of different thresholds so that we can accommodate different definitions of delay. The threshold values used are 35, 40, 45, 50, 55, and 60 MPH. The formula for this is simply D = F * (TT - TT_t) = F * (L/V - L/V_t). In these formulas, D is delay, TT is the travel time, TT_t is the travel time at the threshold speed, F is the flow, L is the length of the segment, V is the current speed and V_t is the threshold speed. Note that by definition, delay can never be negative.
We compute Delay for the various threshold values at the 5-minute level for each loop detector location (lane) as well as across all lanes at a given station. The measure of Delay can be summed up across time and space. This makes it quite easy for us to aggregate these measures to larger geographical regions and longer temporal periods.

D = F * (TT - TT_t) = F * (L/V - L/V_t)

Is this one formula or two? @kengodleskidot
I see this was copied and pasted from here. I am just confused because the following text says these formulas but it look slike a long one that equals the other??


Add requirements.txt installation step

https://github.com/Eeysirhc/sigmabot/commit/c72cd7fd23b54a7b543fb0eac9467064906014e7


Migrates to GH action matrix for E2E runs, preferring actions-setup-docker

This moves special-casing to rust+macOS cell in a normal build matrix.
Issue #152 still needs investigation on how to get around the slowness.
Fixes #153

re-kicking build as the rust+macOS-latest matrix died

rebased on #158

This shows the special-casing where >2year old docker is only used for the rust+macOS cell in the matrix

rebased back to master to decouple this from other various PRs

FYI: this all passes, the only reason it isn't showing that is because via the matrix, the action names drifted. So, it is expecting feedback from the old names. This is expected

realized that TinyGo also experiences the increase of e2e running time, but this is vs running on the latest macos docker instead of 2yo one. Thanks for the work, I'm going to merge this.

Workflow name Run end-to-end (e2e) tests (macos-latest, rust, -v ${RUNNER_TEMP}:/tmp/cargohome -e CARGO_HOME=/t... looks confusing, so I'm going to try -v ${RUNNER_TEMP}:/tmp/cargohome -e CARGO_HOME=/t... both for TinyGo and Rust so that it clears job names

changed the job name according to https://futurestud.io/tutorials/github-actions-customize-the-job-name

:raised_hands:

good idea on rename and thanks for sharing!


Travis CI support

@vieux, it would be wonderful with an official release so I took the liberty of automating the process by adding a build file, .travis.yml. Travis CI will build the code on every commit and deploy the binary to the GitHub repository's release page for every tagged commit.
What's left is that you create a Travis CI account, a GitHub token, and add a GITHUB_TOKEN environmental variable in Travis CI for this repository.

@kongslund I added travis. Thanks.


PHK need D-bus to communicate with avahi-daemon ?

if i change in avahi-daemon.conf "enable-dbus=yes" to "enable-dbus=no" the IPhone don't find the PHK
and there are no error message
i need to now if the PHK use d-bus to communicate with avahi-daemon

For the Bonjour broadcast, I access you will need dbus. Because judging from what I can find from Google, it's used for the software to the daemon, which actually broadcast the service.
Sent from my iPhone

On Apr 2, 2015, at 11:45 PM, GharsallaouiNader<EMAIL_ADDRESS>wrote:
if i change in avahi-daemon.conf "enable-dbus=yes" to "enable-dbus=no" the IPhone don't find the PHK
and there are no error message
i need to now if the PHK use d-bus to communicate with avahi-daemon
—
Reply to this email directly or view it on GitHub.

the broadcast MDNS of bonjour work fine the problem is that the service added by the PHK DNSServiceRegister() don't work. i use wireshark to verify

Yeah. You need service register to attach service to the broadcast.
Sent from my iPhone

On Apr 3, 2015, at 12:07 AM, GharsallaouiNader<EMAIL_ADDRESS>wrote:
the broadcast MDNS of bonjour work fine the problem is that the service added by the PHK DNSServiceRegister() don't work. i use wireshark to verify
—
Reply to this email directly or view it on GitHub.


How to generate CRD with metadata label

The CRD generated by the operator-SDK is controlled by GO annotations.  Example:
+k8s:openapi-gen=true
+kubebuilder:subresource:status
+kubebuilder:resource:shortName=sk

If we wanted to create the CRD with some metadata label such as the following:
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: somename
  labels:
    department: finance
spec:
  group: somegroup
  names:
    kind: SomeKind
    shortNames:
    - sk

Is there something we can put in the go type such that the department: finance would be generated into the CRD?

@tony-clarke-amdocs There isn't a way to generate object metadata like labels from CRD annotations defined in the *_types.go file.
Since object metadata isn't specific to CRDs it doesn't make sense to generate that from the API definition in *_types.go.
Adding common labels, annotations or other object metadata to the various operator manifests would need to be done after the manifests are generated/re-generated by the CRD generator.
So either a kubectl patch or kustomize patch would be the way to do it.
Unfortunately the SDK doesn't provide support for this at the moment, so you'll have to add this step yourself for now.
With some redesign in the near future the SDK would most likely support patching manifests via kustomize.

@hasbro17 Thanks for the info.


Initial project setup tweaks.

Task linked: CU-863gg7bz9 Create GitHub action component for triggering instanced pipelines