Unable to use autocomplete for unusual words using square brackets "[]" in scientific paper

Hi there,
I'm writing a scientific paper that's about some chemicals that use "[]" in their names. For example, "benzo[a]pyrene" or "B[a]P".  I'd like to be able to not have to continually type this word over and over but the brackets in the word seem to be preventing autocompletion to work. When I go to type benzo[a]pyrene it will only autocomplete to benzo, then I have to type "[a]" myself, and then when I start typing pyrene it will autofill that (see images below).

Otherwise autocompletion works fine with other common and uncommon words I'm using.

That is an interesting issue.
I don't think this is a language-markdown issue, but I might be able to help you. In Settings > Editor is an option called Non Word Characters. These characters define word boundaries and [ and ] are among them. These are probably also used when indexing words for autocompletion. Try removing [ and ] from this list, then restart Atom, and see if it helps.
Word of warning: this will have an impact on programming languages if you also use Atom for programming. It probably won't break anything, but it will likely be (mildly) inconvenient.

Hi @burodepeper,
Thanks for the quick reply. Instead of going to general settings > editor, I went into the language-markdown settings and deleted the "[]" from the Non Word Characters like you suggested and it worked (see pic below)! I tried replicating this inside a python file and it didn't work, so that appears to be the best solution for getting around the Non Word Character issue in markdown files and not have it affect other file formats!
This will save me so much time! Is there anyway I can make a little donation to you for your help/work on this awesome package?

 I didn't think of changing it only for language-markdown; an even better solution! What is your paper about, if I may ask?
If you'd like, you can send a donation to my PayPal (see my profile for my email), but you'd make me equally happy by donating to something environmental. I haven't set up anything specific for it, because you would be my first donateur!  I didn't think of changing it only for language-markdown; an even better solution! What is your paper about, if I may ask?

Sure! Thanks for asking :) I'm studying the gut microbiome (microbes that live in our intestines) to see if it digests pollutants (e.g. benzo[a]pyrene, found in smoke or burnt food) and in turn how those digested pollutants might influence how our brain grows in early life. It appears that the microbiome does have an influence on neurological development of young zebrafish, but we need to do more studies to understand exactly why.

If you'd like, you can send a donation to my PayPal (see my profile for my email), but you'd make me equally happy by donating to something environmental. I haven't set up anything specific for it, because you would be my first donateur! :)

Great idea, I'll do both ;)


Rename delete to disable WRT accounts

This follows https://github.com/many-protocol/specification/pull/43

Fixed in #124


Track last NTP server update and display if unable to sync

Give some notification that time sync is broken so the user might be aware that the time is wrong.

I dont think this is required now that the API calls will timeout and give notification.. how long is not as important.


investigate tsup

Look into implementing tsup for our build.
Evaluate if it streamlines our build process, changes package size, and keeps the same functionality.
Things to check:
[ ] Types are usable in dependent projects
[ ] Package is usable in CJS and ESM projects
[ ] Package size doesn't go up a ton
[ ] Package is tree-shakeable (if it already is)

done


Add a missing index

It's missing, we need it, therefore, add it.

I didn't realize this file existed. How do we know it has all the current indexes?
Do we have a plan to deploy indexes exclusively from this file?

I've been using this file as a way to repeatedly deploy indexes to custom instances.
Ideally, this file (and also firestore.rules) would be the source of truth for our Firestore config, and we wouldn't edit things via CLI or web GUI. It would be a bit of a pain to set up probably, but until then this represents a risk of random things failing via missing index or incorrect rules (plus no version control to represent changes in state)

Haha I guess I know why the file keeps not having indexes now!


Different size for different devices

Images show different size in different phones.

You need to provide more details if you expect any hints on what's wrong with your setup.


Any reason to use .format() instead of fstrings?

Hi,
I've noticed most (if not all) string interpolation in your package uses the .format() style, instead of fstrings. I don't think it's for backwards compatibility, since your package already requires Python>=3.6 and fstrings were implemented in 3.6, so is it a question of personal preference using .format()? I ask because, even though there isn't a clear advantage of one over the other, I find fstrings more concise and easier to read.
If you plan on changing them, the output of grep -R "[\"|'].format(" .  on the root directory would show all occurrences.

I also prefer fstrings, and began the project using them. But then I decided to start using .format() to try to produce a 3.5 compatible version. However some fstrings remain in the code and I've largely gave up on this.
If you want to refactor .format() occurrences to fstrings, go ahead, it would be a welcome addition.

Oh I see! I'll send a PR refactoring them then. Thanks!


Add property based tests on diff-sequences

Summary
When I added the first property based tests in Jest codebase - see #8012 - @pedrottimark asked me:

Your thoughts also welcome about tests for diff-sequences package in Jest repo.

This PR adds property based tests to check diff-sequences.
I think we should find better namings for some of the tests I added 🤔
Test plan
PR only adds tests.

Nicolas, thank you for this practical tutorial on property-based testing!
Beyond the scope of this pull request, is there a generic name for a test to compare the package to a short but slow implementation of diff for random inputs?

Beyond the scope of this pull request, is there a generic name for a test that compares the package to a short but slow implementation of diff for random inputs?

Such test would be pretty useful - testing the optimal implementation against a dummy one that just does the job. It might even replace my hacky test: should find at the least the maximal known subset.
I have no generic naming for it. But we can name it something like:
should produce the same output as a non optimal reference *reference or implementation

Super, I will find my quadratic implementation in JavaScript and convert to TypeScript
Do you recommend that I add it to this test file or separate test file in future pull request?

Do you recommend that I add it to this test file or separate test file in future pull request?

As you like ;) But really good idea to add this one ;)
If we want to go further we can even go further by adding yet another test:

generate a and b fully random arrays of chars
compute common items
count the number of each char in common, same for a (and b)
assert that for any character in common the number of times this character appear in common is less or equal to the number of times it appears in a

Can be seen as a variant of should have at most the length of its inputs

I added the test I was talking about in my previous comment here: https://github.com/facebook/jest/pull/9072/commits/5c2e72567c4775a70e89ce819c07b918ea640aa3

After some time in my mental slow cooker, I remembered some pieces that might interest you:

index.test.ts has an alternative algorithm as countDifferences because, as you commented in the symmetric test, you can test for the number but not the items themselves
for randomized sequences, you can test that common items are subsequence of a and b

const isSubsequenceOf = (
  subsequence: Array<string>,
  sequence: Array<string>,
): boolean => {
  let iSub = 0;
  let iSeq = 0;

  while (iSub !== subsequence.length && iSeq !== sequence.length) {
    if (subsequence[iSub] === sequence[iSeq]) {
      iSub += 1;
    }
    iSeq += 1;
  }

  return iSub === subsequence.length;
};

Would you like to add a test of random sequences which asserts

common items have expected length computed by alternative algorithm
common items are indeed a subsequence of both sequences

Because the last test constructs a subsequence of a sequence, it can replace assertion about number of items with assertion about the items themselves:
expect(commonItems).toEqual(partialData);

Codecov Report

Merging #9072 into master will decrease coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #9072      +/-   ##
==========================================
- Coverage   64.07%   64.05%   -0.03%
==========================================
  Files         277      277
  Lines       11681    11676       -5
  Branches     2863     2862       -1
==========================================
- Hits         7485     7479       -6
  Misses       3572     3572
- Partials      624      625       +1

Impacted Files
Coverage Δ

packages/jest-reporters/src/get_result_header.ts
33.33% <0%> (-14.29%)
:arrow_down:

packages/expect/src/utils.ts
94.87% <0%> (-0.07%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bd48ccc...03d6b64. Read the comment docs.

I'm happy with it, we can go ahead


Why is cs locale removed?

According to the release version 1.59.0 the cs locale is added but currently its not avaibale in the newest release and main github repository.

Hi @alexxsexotic,
That's right, my mistake.
I apparently lost this commit in one of the merges from master into the V2 branch back when the V2 was still in development.
Anyway, it is now reintegrated in version 2.3.1 👌


Translate ordinals (1st, 2nd...) into several languages [de, es, fr, it, ja, ko, nb, nl, pt, ru, sv, uk, zh]

https://github.com/korlibs/klock/blob/ca25e8a8b7d8ed2acefda96678331471b546b32b/klock/src/commonMain/kotlin/com/soywiz/klock/KlockLocale.kt#L20-L33
https://github.com/korlibs/klock/issues/150

[x] English - Initial - https://github.com/korlibs/klock/pull/152
[ ] German
[ ] Spanish
[ ] French
[ ] Italian
[ ] Japanese
[ ] Korean
[ ] Norwegian
[ ] Dutch
[ ] Portuguese
[ ] Russian
[ ] Swedish
[ ] Ukrainian
[ ] Chinese

Hey @soywiz, it's not quite easy for example for Russian because the ending of an ordinal depends on the gender of the word after it.
For example: let's take two words with different gender:

language
boy
girl

Eng
1st
1st

Eng
20th
20th

Rus
1-й
1-я

Rus
20-й
20-я

Reference: https://en.wikipedia.org/wiki/Ordinal_indicator#Russian
Maybe for some other languages the same is true. So how we should solve it?

I see. I was not aware of that! We could provide something like LocaleContext, and then populate it with extra information, like gender or other stuff, so it is extensible to support several languages and we don't have to hardcode them for parameters.
But first of all, this library is just for dates, and there is no Locale concept in the Kotlin standard library just yet:
Do date numbers 1-31 in Russian have gender? Boy/girl? If so, are they static depending on the specific day, or could both be supported.
In the end the current signature: open fun getOrdinalByDay(day: Int): String could work?

Do date numbers 1-31 in Russian have gender?

Which exactly "date numbers"? I've just come up with two variants and remembered that we also use the suffix of middle gender sometimes for ordinals in Russian 😅

language
day of month (like "boy", example: 1st day of month)
of MONTH_NAME (middle gender, example: the 1st of January)

Eng
1st
1st

Eng
20th
20th

Rus
1-й
1-е

Rus
20-й
20-е

So I need to know in which sentence the result of getOrdinalByDay is used... This method can't cover all the cases in the current state. However, if it's meant to cover only the specific case, then it's possible, please give me an example of this specific case.

The third variant uses feminine gender:

language
date (like "girl", example: 1st date)

Eng
1st

Eng
20th

Rus
1-я

Rus
20-я

So all genders of Russian language are possible for "date numbers", unless you define them more strictly.

Would this work for you?
class KlockLocalContext(...)
fun KlockLocalContext.withGender(gender: KlockLocaleGender): KlockLocalContext
enum class KlockLocaleGender {} // What about non-binary, etc.? Are used in any supported language?

open fun getOrdinalByDay(day: Int, context: KlockLocalContext = EmptyKlockLocalContext): String

Edit: so in the end is it required for Klock?
Still this discussion could help if someone is making or willing to make a Locale library, or if the Kotlin team plans to add Locale support to the std library at some point

Edit: so in the end is it required for Klock?

Yep, for Russian two genders of ordinals seem to be used in "date numbers": masculine and neuter.

Would this work for you?

Seems like it will work for Russian. I can try opening a PR for KlockLocaleGender, KlockLocalContext, and Russian ordinals. OK?

What about non-binary, etc.? Are used in any supported language?

Just a wild guess: dates appeared in languages quite long ago so there are no dates with non-binary gender in different languages. Of course I can be wrong, but I think other genders can be added later if needed

Sure. Feel free to open a PR. Additionally, could you ask someone from Russia to review the PR too when available?

So korlibs/korge-next#191 is merged, a tick can be placed here 😉
- [x] Russian - korlibs/korge-next#191

Done! It will be available on next version once I integrate changes from korge-next


start time cmake configure doesn't respect cmake.buildDirectory settings.

when starting vscode in project dir it uses default build/ directory and creates some files in build/ directory i don't want.
when configuring manually in CMAKE: PROJCT OUTILE it works correctly.
my settings for build directory:
    "cmake.buildDirectory": "${workspaceFolder}/build/${buildKitTargetArch}",

Sorry if this question seems obvious, but are you using CMakePresets.json to configure your workspace, or CMake Tools Kits?  I didn't reproduce this problem on my first few attempts, so there could be a race condition or something else different about your configuration that is triggering this problem.

thanks @bobbrow,
i am using CMake Tools Kits.

Are you able to share some more information about your environment and/or your repository?

VS Code version
CMake Tools version
OS
Enable the "cmake.loggingLevel": "debug" setting and reload your window - then share the logs in the OUTPUT window


你的提交信息

Hey @Shixuan-Ma just wanted to confirm that you are not planning on submitting to the Shared Task at COLING. If you want to be on the main leaderboard you will be disqualified from the shared task.
If that's okay then I'll approve your PR and add your model.

Thank you for your reply. In fact, this is a test submission, and I do not intend to submit the results of this method to COLING 2025. You can submit it to the leaderboard. An additional question: if I want to submit to COLING 2025, when should I start submitting?

Thank you for your reply. In fact, this was a test submission. I do indeed want to participate in COLING 2025. You don't need to approve this pull request. By the way, if I want to submit to COLING 2025, when should I start the submission process?

@Shixuan-Ma You can start the submission process now. We just released an update to the shared task repository with more detailed submission instructions here. It should be as simple as opening the same pull request but on the other repository with your current predictions.json and metadata.json.

Thanks!


Native Events (Replacement for NWNX events)

Please post which events you are currently using so we can focus on migrating them first :)
The full list:

[ ] AssociateEvents
[ ] BarterEvents
[ ] CalenderEvents
[ ] ClientEvents
[ ] CombatEvents
[ ] DebugEvents
[ ] DMActionEvents
[ ] EffectEvents
[ ] ExamineEvents
[ ] FactionEvents
[ ] FeatEvents
[ ] HealingEvents
[ ] InputEvents
[ ] InventoryEvents
[ ] ItemEvents
[ ] JournalEvents
[ ] LevelEvents
[ ] MapEvents
[ ] MaterialChangeEvents
[ ] ObjectEvents
[ ] PartyEvents
[ ] PlymorphEvents
[ ] PVPEvents
[ ] QuickbarEvents
[ ] QuickChatEvents
[ ] ResourceEvents
[ ] SkillEvents
[ ] SpellEvents
[ ] StealthEvents
[ ] StoreEvents
[ ] TimingBarEvents
[ ] TrapEvents
[ ] UUIDEvents

FeatUseEvents
ExamineEvents
SpellEvents
PlaceableEvents
BarterEvents
StealthEvents
StoreEvents
CombatEvents

Those are Events I'm already using.

Attack and Damage events (DamagePlugin)
Chat events (ChatPlugin)
CombatEvents (on_start_combat_round and on_combat_mode)
DmActionsEvents
DetectionEvents (on_do_listen_detection)
ELCEvents
ExamineEvent
FeatEvents
ItemEvents (including item scroll learn)
InputEvents (on_input_toogle_pause_before and on_input_emote)
JournalEvents
LevelEvents
MapPinEvents
PartyEvents
SkillEvents
SpellEvents
StealthEvents
StoreEvents
TimingBarEvents

(What "PvPEvents" and "ObjectEvents" are you refering to?)

ELC and Examine events currently

StoreEvents
PartyEvents

Since the system for adding native events has been implemented, as well as a fair chunk of the existing NWNX events, I'm closing this issue. Please make individual issues for any events that are still missing, but you would like to use in your plugin/project.


Draggable feature with natural (parent div direction)or image direction

@drozhzhin-n-e  on top of using ngx-pinch-zoom, I added a rotate feature to the image. But after rotating, the drag feature of image works not the natural way. I have to drag in opposite direction to go the required area. May be we can have a feature for this to keep the dragging natural as per parent div or as per image co ordinates (which is current behaviour). :)

I realised I was rotating the image wrong way. Fixed it. Closing this issue. :)


Add Trash Volume

Hi, This project is excellent. I think it will be great if trash volume function added in this project

Sounds good. I'll look at that when I have the time.
Do you have any idea how that works?


Loans query/facet - pending loans with no copies available

In the same way that we need to extract pending loan requests where an item is immediately available (issue reported already in https://github.com/CERNDocumentServer/cds-migrator-kit/issues/106)
we need to be able to filter pending loan requests where:

there is no item at all or,
all items have a status different from "Can Circulate" (i.e. Missing, Scanning, In Binding, For Reference Only) and "Not on loan".

Which search query /facet can we do to achieve this?

in the loans search page you can try:
state:PENDING and available_items_for_loan_count:0


[OneNote] ...

Extension
https://www.raycast.com/alewandowski/onenote
Description
It would be great if there would be a global search over all sections. If you don't know the section you have to go to OneNote and execute the search in there :-(
Who will benefit from this feature?
No response
Anything else?
No response

Hello,
Actually, the extension already searches over all sections globally. You just have to type some keywords in the search bar and corresponding notes (or sections, or notebooks) will appear, wherever they are located.

@raycastbot close this issue

Thanks for your reply.
If I enter some search phrase I get the following error:
Error: Command failed with exit code 1: osascript -e
tell application "System Events"
keystroke "a" using command down
keystroke "c" using command down
end tell
tell application "Microsoft OneNote" to activate
54:86: execution error: System Events got an error: osascript is not allowed to send keystrokes. (1002)
    tell application "System Events"
        keystroke "a" using command down
        keystroke "c" using command down
    end tell

How could I activate OneNote or what Do I have to activate?

When the search gives no result, the extension creates a new note with the search string as title.
To do so, it needs some access rights.
Go to System Settings > Privacy & Security > Accessibility,   and add Raycast to the list.

Ok, Raycast added to Security Settings, thanks.
But how do I make a global search (equivalent to OneNote|Find|Search All Notebooks)? Looks like I only can search in titles?! A global search in "All Notebooks" would really be great!

Not implemented for now.
But it could be interesting, you're right. I put it on the feature request list 👍

Ok, that would be very nice, for me this would be a key feature within Raycast itself.
Also the Mac|OneNote|Find|Search All Notebooks search is a little poor, you only get the page-names where the search string occurs. A perfect search would be to list all lines (clickable to go into the position of that orrurance in that note) in all notebooks with that search phrase.
Not even Microsoft is able to implement such a search :-(


Joint_trajectory action not implemented

Hi,
I'm trying to use the code I have written with real robot with QiBullet. With QiBullet, I cannot use /joint_trajectory action like I did with real robot (with the same ros package /naoqi_driver). I can move the robot with /joint_angles but not with /joint_trajectory. Why?
Thank you for your help!

Hi,
It's normal because it's not linked in the qibullet ros_wrapper. The ros_wrapper contained the basic topics linked as the /joint_angles topic (see _jointAnglesCallback in ros_wrapper.py). If you want, you can implement the function to link  /joint_trajectory and request a PR.

Ok, thank you for you answer. I have used /joint_angles but now I prefer to use the action joint_trajectory to define the entire movement. When there are a lot of positions for a single movement, the movement is more smooth with an action (avoiding publishing multiple positions) espacially with real robot.


crowdsec/http-crawl-non_statics blocking nextcloud

CrowdSec http-crawl-non_statics recently blocked my nextcloud instance.
Versions:

crowdsec  v1.3.0
crowdsecurity/http-crawl-non_statics   enabled  0.2

I fed the nextcloud nginx access log into crowdsec and it seems that CS http-crawl-non_statics scenario ran mad on nextcloud's PROPFIND requests when i was renaming a folder within nextcloud.
Heres the generated (but obfuscated) data:
Alert intel
 - ID         :  XXXX
 - Date       : 2022-02-02T03:45:09Z
 - Machine    : obfuscated
 - Simulation : false
 - Reason     : crowdsecurity/http-crawl-non_statics
 - Events Count : 44
 - Scope:Value: Ip: obfuscated
 - Country    : DE
 - AS         : Deutsche Telekom AG
 - Begin      : 2022-02-02 03:45:05.299144812 +0000 UTC
 - End        : 2022-02-02 03:45:06.831214124 +0000 UTC

Decision
  - Active Decisions  :
+---------+-----------------+--------+---------------+----------------------+
|   ID    |   SCOPE:VALUE   | ACTION |  EXPIRATION   |      CREATED AT      |
+---------+-----------------+--------+---------------+----------------------+
| 1303017 | Ip:  obfuscated | ban    | 26m7.4717213s | 2022-02-02T03:45:09Z |
+---------+-----------------+--------+---------------+----------------------+

Example:
 | datasource_path | /var/www/my-nextcloud/log/nginx-access.log                                   |
| datasource_type | file                                                                          |
| http_args_len   |                                                                             0 |
| http_path       | /remote.php/dav/files/user/dir1/dir2/dir3 |
| http_status     |                                                                           207 |
| http_user_agent | Mozilla/5.0 (Macintosh)                                                       |
|                 | mirall/3.4.1git (build                                                        |
|                 | 8596) (Nextcloud, osx-21.2.0                                                  |
|                 | ClientArchitecture: x86_64                                                    |
|                 | OsArchitecture: x86_64)                                                       |
| http_verb       | PROPFIND

Log Excerpt

Those all have been proper (authenticated) user requests using the PROPFIND method. 33 of them happened within the same second.

Hello,
Can you confirm this has been fixed by #368 ?

Yes, from my perspective the problem is solved.


LIBCLOUD-960: Expand GCE Firewall options coverage

Expand GCE Firewall options coverage
Description
Expanding coverage for full range of GCP firewall rule options currently in GA:

Ingress or Egress direction
Allow or Deny action
Set Description and Priority values
Define source and/or targets by service account

I've preserved the existing defaults.
Issue Link: https://issues.apache.org/jira/browse/LIBCLOUD-960
Status

done, ready for review

Checklist (tick everything that applies)

[x] Code linting (required, can be done after the PR checks)
[x] Documentation
[x] Tests
[ ] ICLA (required for bigger changes)

Codecov Report

Merging #1144 into trunk will decrease coverage by 0.01%.
The diff coverage is 61.53%.

@@            Coverage Diff             @@
##            trunk    #1144      +/-   ##
==========================================
- Coverage   85.49%   85.48%   -0.02%
==========================================
  Files         348      348
  Lines       66579    66613      +34
  Branches     5925     5933       +8
==========================================
+ Hits        56924    56944      +20
- Misses       7247     7254       +7
- Partials     2408     2415       +7

Impacted Files
Coverage Δ

libcloud/test/compute/test_gce.py
97.6% <100%> (ø)
:arrow_up:

libcloud/compute/drivers/gce.py
76% <58.33%> (-0.16%)
:arrow_down:

libcloud/test/compute/test_ec2.py
97.77% <0%> (-0.16%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dcb0cef...cdf4a73. Read the comment docs.

I think these are Travis or other issues - I didn't touch what's failing. But I added tests!

@maxlip Thanks. However you do have lint errors:
libcloud/test/compute/test_gce.py:847:29: E126 continuation line over-indented for hanging indent
libcloud/test/compute/test_gce.py:864:29: E126 continuation line over-indented for hanging indent
libcloud/test/compute/test_gce.py:881:30: E126 continuation line over-indented for hanging indent

Rebasing on top of trunk would solve the paramiko issue and would allow me to see the coverage easily. Do you think you could do that? Thanks!

Sorry, I realized that the paramiko failure is not your fault.  Thanks.

Just making sure it's clear that linting looks fixed - CI is now all good.
Thanks for your patience with my misreading of CI output :) I can't ever seem to get this tox suite to work on my local, so it can be a challenge.

Thank you @maxlip! Merged in trunk! ✨
I've credited you as "maxlip". Would you like me to use something else?

Thanks for fixing that Quentin - "maxlip" is good for me.


gn: 20190403 -> 2020-03-09

This updates gn to the required version for chromiumDev (the recommended
version for the stable release of Chromium isn't sufficient [0]).
[0]: The Chromium build fails during the configuration phase:
ERROR at //mojo/public/tools/bindings/mojom.gni:393:16: Undefined identifier
"cpp_typemaps",
^-------------
Testing
I only verified it with the configuration phase of chromium and chromiumDev (which both work),e.g.:
./content/renderer/PRESUBMIT_test.py: interpreter directive changed from "/usr/bin/env python" to "/nix/store/vplvkrddlj3fia6m79x0czgnchag5mbs-python-2.7.17/bin/python"
configuring
Done. Made 11223 targets from 2053 files in 27168ms
building
ninja: Entering directory `out/Release'
[1/1378] STAMP obj/build/win/default_exe_manifest.stamp

Update: I've verified additional configuration phases:
v8
configuring
gn flags: use_custom_libcxx=false is_clang=false use_sysroot=false is_component_build=false v8_use_external_startup_data=false v8_monolithic=true is_debug=true is_official_build=false treat_warnings_as_errors=false v8_enable_i18n_support=true use_gold=false use_system_xcode=true host_toolchain="//build/toolchain/linux/unbundle:default" v8_snapshot_toolchain="//build/toolchain/linux/unbundle:default"
WARNING at the command-line "--args":1:266: Build argument has no effect.
use_custom_libcxx=false is_clang=false use_sysroot=false is_component_build=false v8_use_external_startup_data=false v8_monolithic=true is_debug=true is_official_build=false treat_warnings_as_errors=false v8_enable_i18n_support=true use_gold=false use_system_xcode=true host_toolchain="//build/toolchain/linux/unbundle:default" v8_snapshot_toolchain="//build/toolchain/linux/unbundle:default"

                                                     ^---
The variable "use_system_xcode" was set as a build argument
but never appeared in a declare_args() block in any buildfile.

To view all possible args, run "gn args --list <out_dir>"

The build continued as if that argument was unspecified.

Done. Made 145 targets from 77 files in 4901ms

qt5.qtwebengine
( test -e Makefile.gn_run || /nix/store/ra0c9zj9arbp4cw9d8g0bixnh4hqqrdf-qtbase-5.12.7-dev/bin/qmake -o Makefile.gn_run /build/qtwebengine-everywhere-src-5.12.7/src/core/gn_run.pro PREFIX=/nix/store/1k7zrr72cvmwwy7wcmbphifdjjybz3a2-qtwebengine-5.12.7 NIX_OUTPUT_OUT=/nix/store/1k7zrr72cvmwwy7wcmbphifdjjybz3a2-qtwebengine-5.12.7 NIX_OUTPUT_DEV=/nix/store/cgg01ghmalpc1scw2sx0jy65vr43xb7h-qtwebengine-5.12.7-dev NIX_OUTPUT_BIN=/nix/store/l9kagvcayl3yf0ybfqfzlq8cz96xrmiq-qtwebengine-5.12.7-bin NIX_OUTPUT_DOC=/nix/store/cgg01ghmalpc1scw2sx0jy65vr43xb7h-qtwebengine-5.12.7-dev/share/doc/qt-5.12.7 NIX_OUTPUT_QML=/nix/store/l9kagvcayl3yf0ybfqfzlq8cz96xrmiq-qtwebengine-5.12.7-bin/lib/qt-5.12.7/qml NIX_OUTPUT_PLUGIN=/nix/store/l9kagvcayl3yf0ybfqfzlq8cz96xrmiq-qtwebengine-5.12.7-bin/lib/qt-5.12.7/plugins CONFIG+=release ) && make -f Makefile.gn_run
Project MESSAGE: Running: /build/qtwebengine-everywhere-src-5.12.7/src/3rdparty/gn/out/Release/gn gen /build/qtwebengine-everywhere-src-5.12.7/src/core/release --script-executable=/nix/store/vplvkrddlj3fia6m79x0czgnchag5mbs-python-2.7.17/bin/python2 --args='use_qt=true is_component_build=false is_shared=true enable_message_center=false enable_mus=false enable_nacl=false enable_remoting=false enable_reporting=false enable_web_speech=false enable_widevine=true has_native_accessibility=false use_allocator_shim=false use_allocator="none" v8_use_external_startup_data=false treat_warnings_as_errors=false enable_swiftshader=false use_custom_libcxx=false use_jumbo_build=true jumbo_file_merge_limit=8 jumbo_build_excluded=["browser"] enable_basic_printing=true enable_print_preview=true enable_pdf=true enable_plugins=true enable_spellcheck=true enable_webrtc=true proprietary_codecs=true ffmpeg_branding="Chrome" enable_precompiled_headers=true is_official_build=true is_debug=false symbol_level=0 remove_webcore_debug_symbols=true remove_v8base_debug_symbols=true v8_use_snapshot=true use_kerberos=false enable_iterator_debugging=false use_cups=false use_gio=false use_gnome_keyring=false linux_use_bundled_binutils=false use_udev=true use_bundled_fontconfig=false use_sysroot=false enable_session_service=false is_cfi=false toolkit_views=false use_ozone=true ozone_auto_platforms=false ozone_platform_headless=false ozone_platform_external=true ozone_platform="qt" is_clang=false custom_toolchain="/build/qtwebengine-everywhere-src-5.12.7/src/toolchain:target" host_toolchain="/build/qtwebengine-everywhere-src-5.12.7/src/toolchain:host" host_cpu="x64" pkg_config="pkg-config" host_pkg_config="/nix/store/f6dakip9hyywfvwd23wnrh41fxl9p4yi-pkg-config-0.29.2/bin/pkg-config" use_system_zlib=true use_system_minizip=true pdfium_use_system_zlib=true use_system_libpng=true pdfium_use_system_libpng=true use_system_libjpeg=true use_system_freetype=true use_system_harfbuzz=true use_pulseaudio=false use_alsa=true ozone_platform_x11=true use_xscrnsaver=true use_system_libevent=true use_system_libwebp=true use_system_opus=true use_system_snappy=true qtwebengine_target="/build/qtwebengine-everywhere-src-5.12.7/src/core/release:QtWebEngineCore"' --root=/build/qtwebengine-everywhere-src-5.12.7/src/3rdparty/chromium
Done. Made 9697 targets from 1582 files in 21653ms

aseprite.skia
configuring
Done. Made 42 targets from 28 files in 121ms
building

Motivation
This fixes the build of chromiumDev and is possibly required for other builds too (or will be soon).
I've also added some extra notes and did a minor refactoring.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

cc maintainers: @stesie @matthewbauer

I did some additional manual testing and fixed one regression. Assuming I didn't miss anything with my minimal testing approach, the rest of the rebuilds should be fine.

@NixOS/backports I just realized that I'll need this change on 20.03 as well to update Chromium (version 83 should be released on May 19). Should I backport this PR or write a specific patch for 20.03 to override gn for chromium?

We shouldn't use ${pkgs.gn} to build chromium.
Each chromium needs own version of gn (and nodejs,btw), and the task to keep pkgs.gn version so it can build pkgs.chromium, pkgs.chromiumBeta, pkgs.chromiumDev does not seem meaningful.
First, it breaks other software, and second, it is not always possible: one day pkgs.chromium and pkgs.chromiumBeta will require different gn
So it would be better to download gn binary from chromium website (it is a single file - statically linked ELF binary), exact version which needed for each chromium.
cc @ivan

We shouldn't use ${pkgs.gn} to build chromium.

I don't necessarily disagree (would e.g. be easier to maintain), but why does this apply to Chromium specifically? There aren't any tagged releases for gn so IMO this applies to any package that depends on gn. And due to the lack of any tagged release it would IMO make the most sense to base the gn version on the current Chromium release (stable channel). Otherwise, how would we decided when to update gn and to which version?

First, it breaks other software

Did this PR break anything?

and second, it is not always possible: one day pkgs.chromium and pkgs.chromiumBeta will require different gn

This is currently the case due to breaking gn changes. We therefore simply override gn for the beta and dev channels.

but why does this apply to Chromium specifically?

Because chromium sources have explicit info on how to download (older version) or how to build (newer) gn.
They never rely on gn (or node) which might be installed systemwide.
So to say, building proper gn is part of building chromium

First, it breaks other software
Did this PR break anything?

For me: older chromiums (which I keep to test that websites does support them).
It could be argued with "it is removed from nixpkgs, so we shouldn't care", so ok.

I've opened #85457 to further track this discussion (as this PR is already merged) ;)


Event occurs after unwatching file

Hi, I forked a new repository and run a test case. Then this error appeared:
  Tail
    ✓ should read a file with Windows line ending
    ✓ should read a file with Linux line ending
    ✓ should handle null separator option to not split chunks
    ✓ should respect fromBeginning flag (3003ms)
    ✓ should respect fromBeginning from even the first appended line
    ✓ should send error event on deletion of file while watching
    1) should send error event on deletion of file while watching

  6 passing (3s)
  1 failing

  1) Tail
       should send error event on deletion of file while watching:
     Uncaught Error: ENOENT: no such file or directory, stat '/media/anhhao/Data1/Projects/Anh Hao/node-tail/test/example.txt'
      at Object.statSync (fs.js:1086:3)
      at Tail.latestPosition (src/tail.js:74:23)
      at Tail.change (src/tail.js:118:22)
      at Tail.watchEvent (src/tail.js:183:18)
      at FSWatcher.<anonymous> (src/tail.js:143:101)
      at FSEvent.FSWatcher._handle.onchange (internal/fs/watchers.js:186:12)

After a few minutes of research, I detected the problem occur when a change event still emit after unwatching the file.
I'm using Ubuntu 20.04 and saw more problems than this one. Hope to see your fixes in the future soon.
Thanks.

hi @anhhao-me,
Thanks for reporting this issue.
Two questions:

is this a problem purely on running the tests, or have you seen these issue also when running the library in production?
2)"I  saw more problems than this one": which other issues have you seen?

Currently tail emits specific error events after unwatch. There's no additional line event after unwatch.


Release cert-manager 1.16

Patch Release v1.16.1

Release thread: https://kubernetes.slack.com/archives/CDEQJ0Q8M/p1728477663574319
Helm chart: https://github.com/jetstack/jetstack-charts/pull/362
Release notes: https://github.com/cert-manager/website/pull/1584
Release: https://github.com/cert-manager/cert-manager/releases/tag/v1.16.1
OLM release: https://github.com/cert-manager/cert-manager-olm/pull/111

Release

Release thread: https://kubernetes.slack.com/archives/CDEQJ0Q8M/p1727957184327449
Helm Chart: https://github.com/jetstack/jetstack-charts/pull/360
Release: https://github.com/cert-manager/cert-manager/releases/tag/v1.16.0
ArtifactHub: https://artifacthub.io/packages/helm/cert-manager/cert-manager/1.16.0
Mailing List Announcement: https://groups.google.com/g/cert-manager-dev/c/fWs5JaCvOcE/m/jaEY6vZAAgAJ
Tweet: https://x.com/CertManager/status/1842139744777498942
Mastadon: https://infosec.exchange/@CertManager/113248523073785650
https://github.com/endoflife-date/endoflife.date/pull/5946

Documentation

https://github.com/cert-manager/website/pull/1520
https://release-next--cert-manager.netlify.app/docs/releases/release-notes/release-notes-1.16

Pre-releases

https://github.com/cert-manager/cert-manager-olm/pull/110
https://github.com/k8s-operatorhub/community-operators/pull/5096
https://github.com/redhat-openshift-ecosystem/community-operators-prod/pull/5276
https://github.com/cert-manager/cert-manager/releases/tag/v1.16.0-beta.0
https://github.com/cert-manager/cert-manager/releases/tag/v1.16.0-alpha.1
https://github.com/cert-manager/cert-manager/releases/tag/v1.16.0-alpha.0

Testing
A record of the pre-release testing.

[x] With Venafi TLSPK components
[x] https://github.com/cert-manager/makefile-modules/pull/199
[x] https://github.com/cert-manager/approver-policy/pull/508
[x] https://github.com/cert-manager/approver-policy/pull/509
[x] https://github.com/cert-manager/trust-manager/pull/447
[x] https://github.com/cert-manager/csi-driver/pull/309
[x] https://github.com/cert-manager/csi-driver-spiffe/pull/196
[x] https://github.com/cert-manager/istio-csr/pull/416
[x] https://github.com/cert-manager/openshift-routes/pull/104
[x] https://github.com/cert-manager/sample-external-issuer/pull/50
[x] https://github.com/cert-manager/google-cas-issuer/pull/212
[x] https://github.com/cert-manager/aws-privateca-issuer/pull/341
[x] https://testgrid.k8s.io/cert-manager-periodics-release-1.16
[x] https://github.com/redhat-openshift-ecosystem/community-operators-prod/pull/5276#issuecomment-2389166491
[x] https://github.com/k8s-operatorhub/community-operators/pull/5096#issuecomment-2390865589
[x] https://github.com/cert-manager/cert-manager/actions/runs/11161647273

Thank you for releasing 1.16. Based on your task list, I think this is now completed.


[5.5] Future-proof the layout of AsyncTask

5.5 version of #37530

@swift-ci Please test

@swift-ci Please test

Build failed
Swift Test Linux Platform
Git Sha - 4095a7fe0b27bf47bfab3849b335ec79a01267fa

Build failed
Swift Test OS X Platform
Git Sha - 4095a7fe0b27bf47bfab3849b335ec79a01267fa

@swift-ci Please test

@ktoso You approved the main version of this; mind reviewing this version as well so I can merge it?


Badge of notifications count does not increasing when receive notification from Batch.com

This is know issue of Batch.com and it is in a roadmap.


Column overlaps in Datagrid when using expandable rows

Describe the bug
A Datagrid with horizontal scrollbar and long text values can produce overlapping text when expandable rows are used.

How to reproduce
_https://stackblitz.com/edit/clarity-light-theme-v2-qkuu4k_
Steps to reproduce the behavior:

Go to 'https://clarity-light-theme-v2-qkuu4k.stackblitz.io/'
See overlapping text in columns

Expected behavior
Column text should not overlap as is the case when no expandable rows are used.
Versions
App

Angular: [8]
Node: [8.9.4]
Clarity: [2.0.0]

Device:

Type: [ThinkPad]
OS: [Windows 10]
Browser [chrome]
Version [78.0.3904.97]

Thanks for the report, something is indeed strange here. Updated to Clarity 2.3.1 and still see it happening, but there seems to be some bizarre calculation problem only for some of the columns. Will need to investigate.

It looks as if the calculated widths for the headers are shifted by 1 column. The width of header x should be the width of header x+1, etc...

Any progress on this reported bug?

No progress yet, please review our labels to get a sense of what is being worked on or not.

One workaround for now is to put an explicit width if you have an issue with a column like this.

Another workaround
clr-dg-cell {
overflow: hidden;
text-overflow: ellipsis;
}
or
clr-dg-cell element-around-your-text {
overflow: hidden;
text-overflow: ellipsis;
}


Configurable template directory

Thanks for the extension, it works well and saves a lot of time 👍
I'd like to be able to configure the directory where the extension looks for templates. This would help me keep my root directory organized.

Hey @deasems. Sorry, I somehow missed this issue.
Thank you for the suggestion. Makes alot of sense. I will implement this soonish.

Hey @deasems , thanks again for the idea.
I just published a new version. Feel free to take a look at this new configuration option: https://github.com/Huuums/vscode-folder-templates#foldertemplatestemplatefolderpath


Unit test support in GAX

Add support for unit test in GAX

Add MockStubTrait and MockUnaryCall
Update existing tests

Current coverage is 88.76% (diff: 43.33%)

Merging #52 into master will decrease coverage by 1.01%

@@             master        #52   diff @@
==========================================
  Files            18         20     +2
  Lines          1341       1371    +30
  Methods         147        154     +7
  Messages          0          0
  Branches          0          0
==========================================
+ Hits           1204       1217    +13
- Misses          137        154    +17
  Partials          0          0

Powered by Codecov. Last update 0189f6f...05219ba


Feature Request: thread_id

It would be great if the thread_id could be printed out, which could be helpful to learn the context in multi-threaded applications.

I believe this should be possible through the use of the FnValue struct.
The following code should create a new child logger. The new child will have a thread_id key, computed by the "FnValue". The FnValue will debug-display the ThreadId
let child = parent.new(o!(
    "thread_id" => slog::FnValue(|| {
        format!("{:?}", std::thread::current().id())
    })
));
// any use of the `child` logger will have `thread_id` as context.
play(&child);

You should also be able to modify this to work with the root loger (just move it to the initial o!() instead of creating a new child).
Please let me know if this doesn't work


Re-check TLS versions and cipher-suites across the project.

We generally use Mozilla Modern has a reference for our internal communications as we generally control clients and servers (https://wiki.mozilla.org/Security/Server_Side_TLS) but we're behind and more in the intermediate range these days.
It's probably time to go through and update the configuration across all the gravity components.
We can also consider going to Ed25519 certificates because we control both clients and servers, but may need some thought.

The current etcd that we use doesn't support TLS v1.3 cipher suites. It fails to start with error:
unexpected TLS cipher suite

We will go with Mozilla intermediate compatibility until etcd implements support for v13 ciphers.

The current version of kube-apiserver that we use doesn't support TLS v13 suites. It fails to start with error:
kube-apiserver[16629]: Error: Cipher suite TLS_AES_128_GCM_SHA256 not supported or doesn't exist

The latest version of Kubernetes supports v13. We will go on with Mozilla intermediate compatibility until we upgrade k8s and after that we will upgrade the cipher suites as well.

Our current kubelet also doesn't support v13 Mozilla modern level ciphers:
kubelet[13373]: F1215 20:48:25.086905   13373 server.go:253] Cipher suite TLS_AES_128_GCM_SHA256 not supported or doesn't exist

Kube-scheduler:
kube-scheduler[13393]: Cipher suite TLS_AES_128_GCM_SHA256 not supported or doesn't exist

monitoring    kube-state-metrics-7975db7f8c-j6fs2                              1/3     CrashLoopBackOff   36         87m

Container: kube-rbac-proxy-main
main.go:233] Failed to convert TLS cipher suite name to ID: Cipher suite TLS_AES_128_GCM_SHA256 not supported or doesn't exist

Work on this story is postponed due to the version of Kubernetes that we use not supporting the new ciphers. We may revisit after we upgrade K8s to another version.


Get Mapped Result from Hilbert Curve

Hi, Good work! I am wondering how I can get the mapped result from n-dimensional data point alongside Hilbert Curve. As far as I know, if I use distance_from_point, I only get the index of the data instead of mapped value. If you have any advice, that will be very helpful.
Looking forward to your early reply!

Can you rephrase the question? I'm not sure exactly what you're asking for. The package converts back and forth between distance along a curve and the discrete coordinates in n-dim space. It sounds like maybe you want to take an arbitrary point in n-dim space and find the closest discrete coordinates? If so, then you can bound each dimension and then do a check on the remaining 2^n discrete points.

Can you rephrase the question? I'm not sure exactly what you're asking for. The package converts back and forth between distance along a curve and the discrete coordinates in n-dim space. It sounds like maybe you want to take an arbitrary point in n-dim space and find the closest discrete coordinates? If so, then you can bound each dimension and then do a check on the remaining 2^n discrete points.

So kind of you to give a reply. I have found that i misunderstood the function. Now I have solved the issue. Thank you so much.


changed-ACE-settings-to-enforce-mods

With these settigns ACE checks all of the PBOs (i.e. mods) and if they don't match up with what is on the server the player gets kicked. Whitelist is currently blank. Needs testing

Personnaly never used that ACE settings, I fear that it would require to put some client-side mods on server
If you want to enforce mods, better way is to use verifySignatures in server.cfg, with this setting only .bikey are required on the server for client-side mods, not the whole pbo

https://ace3.acemod.org/wiki/framework/checkpbos-framework


Exposed KVCollectionValue

When doing some data extraction, the operation like
List<KVObject> collection = ((KVCollectionValue)app.Data["config"]["launch"]).children;
was not allowed due to the Internal Access Modifiers.
In some situation, KVCollectionValue is much handy than abstract KVValue

Publicly exposing the inner List ties us to that particular implementation forever, and allows consumers to bypass any validation on the members (currently that is just nullability).

I'm analyzing appinfo.vdf. With SteamAppInfoParser
There are many games with multiple start options.
When I'm trying to get index of each start option.
I Must use Indexer like var options = AppData["config"]["launch"] that return a KVValue (actually KVCollectionValue)
This KVValue Contains multiple children with diffirient name (int index number).
The index number was not always increment by 1.
I don't know how many items insides options.
So I can't keep using indexer like options["index"]
The only way is to iterate the options.children to exract infomations that I'm intrerested in.
I can't do that without exposed iterable properties.

Casting it to IEnumerable<KVObject> should let you enumerate the children, giving you access to both the names and values.

See #21

See #21

yea, casting IEnumerable<T> makes APIs more complicated.
There is another confused method public override int ToInt32(IFormatProvider provider)
Although I can call value.ToInt32(null).
But I think adding a value.ToInt32() method is much more convenient and clear.
Handy to consumers, without digging into source code.
btw, I'm using self-modified lib now.

You should be able to simply cast the object to int if you don't have a particular culture to use etc.:
https://github.com/SteamDatabase/ValveKeyValue/blob/master/ValveKeyValue/ValveKeyValue.Test/KVObjectCastTestCase.cs#L103

Closing in favour of #30 to provide a more wholistic API redesign.


文档样式问题

Have you read the Contributing Guidelines on issues?

[X] I have read the Contributing Guidelines on issues.

Description
滑动的时候下面的内容溢出了
https://github.com/JDFED/drip-form/assets/2911620/3c9b1bd3-ef3a-45ec-b1a0-a6684161236e
Self-service

[x] I'd be willing to address this documentation request myself.

@mengshang918 还会修复么？


[Question]: Using vector index with Firestore return empty response

Question Validation

[X] I have searched both the documentation and discord for an answer.

Question
I am following the FirestoreDemo with vector index (https://github.com/run-llama/llama_index/blob/main/docs/examples/docstore/FirestoreDemo.ipynb). I have touched up my code as below
import nest_asyncio

from llama_index import (
    SimpleDirectoryReader,
    ServiceContext,
    StorageContext,
)
from llama_index import VectorStoreIndex, SummaryIndex, SimpleKeywordTableIndex
from llama_index.composability import ComposableGraph
from llama_index.llms import OpenAI
from llama_index.response.notebook_utils import display_response

import logging
import sys

from llama_index.storage.kvstore.firestore_kvstore import FirestoreKVStore
from llama_index.storage.docstore.firestore_docstore import (
    FirestoreDocumentStore,
)
from llama_index.storage.index_store.firestore_indexstore import (
    FirestoreIndexStore,
)

from llama_index.node_parser import SentenceSplitter

<EMAIL_ADDRESS>chatroomid="000001"
namespace=email+"_"+chatroomid

nest_asyncio.apply()
logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))

folder=r"data/paul_graham"
data_path = os.path.join(os.getcwd(), folder)
reader = SimpleDirectoryReader(data_path)
documents = reader.load_data()
nodes = SentenceSplitter().get_nodes_from_documents(documents)

kvstore = FirestoreKVStore()

storage_context = StorageContext.from_defaults(
    docstore=FirestoreDocumentStore(kvstore,namespace=namespace),
    index_store=FirestoreIndexStore(kvstore,namespace=namespace),
)

storage_context.docstore.add_documents(documents)
vector_index = VectorStoreIndex.from_documents(documents, storage_context=storage_context)
#%%
vector_id = vector_index.index_id

storage_context.persist()

from llama_index.indices.loading import load_index_from_storage

kvstore = FirestoreKVStore()

re-create storage context
storage_context = StorageContext.from_defaults(
    docstore=FirestoreDocumentStore(kvstore,namespace=namespace),
    index_store=FirestoreIndexStore(kvstore,namespace=namespace)
)

load indices
vector_index = load_index_from_storage(
    storage_context=storage_context, index_id=vector_id
)
query_engine = vector_index.as_query_engine()
vector_response = query_engine.query("What did the author do growing up?")
display_response(vector_response)```

However, I only get empty response. I have tried with Summary index and keyword index and they did work.
May I know how to make vector index work?

@dosu-bot I notice everything work fine until I use the
vector_index = load_index_from_storage(     storage_context=storage_context, index_id=vector_id )
It does return
<llama_index.indices.vector_store.base.VectorStoreIndex object at 0x7fd4a9bab1f0>
However, it still gives empty response
If i am not separating the query part and not using
load_index_from_storage(     storage_context=storage_context, index_id=vector_id )
It gives me a response. Is there anything I miss?

🤖
Hello again @Tomly-Hola-Lee! It's good to see you diving further into the LlamaIndex library. Let's see what we can do about your question.
Based on the information you've provided and the context from the LlamaIndex repository, it seems like the issue might be related to how the vector index data is being stored and retrieved in Firestore.
The FirestoreKVStore class in the LlamaIndex framework is responsible for storing and retrieving data from Firestore. It uses methods like put, aput, put_all, and aput_all to store data, and get, aget, get_all, and aget_all to retrieve data. These methods use the replace_field_name_set and replace_field_name_get functions to replace certain field names before storing and after retrieving the data.
If the vector index data contains the field names "data" or "type", they will be replaced with "data" and "type" when stored, and replaced back when retrieved. If the vector index data does not contain these field names, or if they are not replaced correctly, this could result in an empty response when retrieving the data.
Here are some steps you can take to troubleshoot this issue:

Check if the vector index data is being stored correctly in Firestore. You can do this by checking the Firestore database directly.
Verify that the vector index data contains the field names "data" and "type". If it does not, you may need to modify the data or the replace_field_name_set and replace_field_name_get functions to handle the correct field names.
Review the code that is using the FirestoreKVStore class to store and retrieve the vector index data. Ensure that it is using the correct collection name and key when calling the put and get methods.
If the issue persists, you may need to debug the FirestoreKVStore class to ensure that the put, get, replace_field_name_set, and replace_field_name_get methods are working correctly.

You can find more information about the FirestoreKVStore class in the firestore_kvstore.py file in the LlamaIndex repository.
As for the VectorStoreIndex.from_documents function, I wasn't able to find specific information about it in the LlamaIndex repository. It would be helpful if you could provide more details about the data you're passing to this function and the output you're expecting. This could help in identifying any potential issues with how this function is being used.
Sources

llama_index/storage/kvstore/firestore_kvstore.py

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.
Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.
If you want to continue the conversation, start your reply with @dosu-bot.
@dosu-bot I notice everything work fine until I use the
vector_index = load_index_from_storage( storage_context=storage_context, index_id=vector_id )
It does return
<llama_index.indices.vector_store.base.VectorStoreIndex object at 0x7fd4a9bab1f0>
However, it still gives empty response

If i am not separating the query part and not using
load_index_from_storage( storage_context=storage_context, index_id=vector_id )
It gives me a response. Is there anything I miss?

can you try using https://docs.llamaindex.ai/en/stable/understanding/tracing_and_debugging/tracing_and_debugging.html for debugging which part yield empty results

@hatianzhang

can you try using https://docs.llamaindex.ai/en/stable/understanding/tracing_and_debugging/tracing_and_debugging.html for debugging which part yield empty results

Thanks for suggestion
I have open the debug mode and this is the response when I don't recreate storage context i.e. make query after indexing
vector_index = VectorStoreIndex(nodes, storage_context=storage_context)

query_engine = vector_index.as_query_engine()
vector_response = query_engine.query("Who is the author?")
print(vector_response)

The debug panel shows:
DEBUG:openai._base_client:Request options: {'method': 'post', 'url': '/embeddings', 'files': None, 'post_parser': <function Embeddings.create..parser at 0x7fd778273160>, 'json_data': {'input': ['file_path: /Users/Tomly/Documents/Python/LlamaIndex/data/paul_graham/paul_graham_essay.txt  What I Worked On  February 2021  Before college the two ma...T_EMBED_ADA_002: 'text-embedding-ada-002'>, 'encoding_format': 'base64'}}
Request options: {'method': 'post', 'url': '/embeddings', 'files': None, 'post_parser': <function Embeddings.create..parser at 0x7fd778273160>, 'json_data': {'input': ['file_path: /Users/Tomly/Documents/Python/LlamaIndex/data/paul_graham/paul_graham_essay.txt  What I Worked On  ...T_EMBED_ADA_002: 'text-embedding-ada-002'>, 'encoding_format': 'base64'}}
DEBUG:httpx._client:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
DEBUG:openai._base_client:HTTP Request: POST https://api.openai.com/v1/embeddings "200 OK"
HTTP Request: POST https://api.openai.com/v1/embeddings "200 OK"
DEBUG:openai._base_client:Request options: {'method': 'post', 'url': '/embeddings', 'files': None, 'post_parser': <function Embeddings.create..parser at 0x7fd759e73550>, 'json_data': {'input': ['Who is the author?'], 'model': <OpenAIEmbeddingModeModel.TEXT_EMBED_ADA_002: 'text-embedding-ada-002'>, 'encoding_format': 'base64'}}
Request options: {'method': 'post', 'url': '/embeddings', 'files': None, 'post_parser': <function Embeddings.create..parser at 0x7fd759e73550>, 'json_data': {'input': ['Who is the author?'], 'model': <OpenAIEmbeddingModeModel.TEXT_EMBED_ADA_002: 'text-embedding-ada-002'>, 'encoding_format': 'base64'}}
DEBUG:httpx._client:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
DEBUG:openai._base_client:HTTP Request: POST https://api.openai.com/v1/embeddings "200 OK"
HTTP Request: POST https://api.openai.com/v1/embeddings "200 OK"
DEBUG:llama_index.indices.utils:> Top 2 nodes:

[Node 0f9ec477-71c6-4d02-a12a-facf05a1ef45] [Similarity score:             0.773803] All that seemed left for philosophy were edge cases that people in other fields felt could safely...
[Node 73561cee-5ff5-49d7-9c4f-c048bd65d0ad] [Similarity score:             0.773635] I remember taking the boys to the coast on a sunny day in 2015 and figuring out how to deal with ...
Top 2 nodes:
[Node 0f9ec477-71c6-4d02-a12a-facf05a1ef45] [Similarity score:             0.773803] All that seemed left for philosophy were edge cases that people in other fields felt could safely...
[Node 73561cee-5ff5-49d7-9c4f-c048bd65d0ad] [Similarity score:             0.773635] I remember taking the boys to the coast on a sunny day in 2015 and figuring out how to deal with ...
DEBUG:openai._base_client:Request options: {'method': 'post', 'url': '/chat/completions', 'files': None, 'json_data': {'messages': [{'role': 'system', 'content': "You are an expert Q&A system that is trusted around the world.\nAlways answer the query using the provided context information, and not prior knowledge.\nSome rules to follow:\n1. Never directly reference the given context in your answer.\n2. Avoid statements like 'Based on the context, ...' or 'The context information ...' or anything along those lines."}, {'role': 'user', 'content':...nformation and not prior knowledge, answer the query.\nQuery: Who is the author?\nAnswer: '}], 'model': 'gpt-3.5-turbo', 'stream': False, 'temperature': 0.1}}
Request options: {'method': 'post', 'url': '/chat/completions', 'files': None, 'json_data': {'messages': [{'role': 'system', 'content': "You are an expert Q&A system that is trusted around the world.\nAlways answer the query using the provided context information, and not prior knowledge.\nSome rules to follow:\n1. Never directl...answer the query.\nQuery: Who is the author?\nAnswer: '}], 'model': 'gpt-3.5-turbo', 'stream': False, 'temperature': 0.1}}
DEBUG:httpx._client:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
DEBUG:openai._base_client:HTTP Request: POST https://api.openai.com/v1/chat/completions "200 OK"
HTTP Request: POST https://api.openai.com/v1/chat/completions "200 OK"
The author of the text is Paul Graham.

However, if I restore storage context with vector id as below
kvstore = FirestoreKVStore()

storage_context = StorageContext.from_defaults(
    docstore=FirestoreDocumentStore(kvstore),
    index_store=FirestoreIndexStore(kvstore)
)

vector_id='28ddc079-bc01-4947-a543-455dbdb8700a'
vector_index = load_index_from_storage(
    vector_id=vector_id,storage_context=storage_context
)

query_engine = vector_index.as_query_engine()
vector_response = query_engine.query("Who is the author?")
print(vector_response)

It gives the following
DEBUG:google.auth._default:Checking None for explicit credentials as part of auth process...
Checking None for explicit credentials as part of auth process...
DEBUG:google.auth._default:Checking Cloud SDK credentials as part of auth process...
Checking Cloud SDK credentials as part of auth process...
DEBUG:google.auth._default:Checking None for explicit credentials as part of auth process...
Checking None for explicit credentials as part of auth process...
DEBUG:google.auth._default:Checking Cloud SDK credentials as part of auth process...
Checking Cloud SDK credentials as part of auth process...
DEBUG:google.auth._default:Checking None for explicit credentials as part of auth process...
Checking None for explicit credentials as part of auth process...
DEBUG:google.auth._default:Checking Cloud SDK credentials as part of auth process...
Checking Cloud SDK credentials as part of auth process...
DEBUG:google.auth._default:Checking None for explicit credentials as part of auth process...
Checking None for explicit credentials as part of auth process...
DEBUG:google.auth._default:Checking Cloud SDK credentials as part of auth process...
Checking Cloud SDK credentials as part of auth process...
INFO:llama_index.indices.loading:Loading all indices.
Loading all indices.
Loading all indices.
DEBUG:google.auth.transport.requests:Making request: POST https://oauth2.googleapis.com/token
Making request: POST https://oauth2.googleapis.com/token
Making request: POST https://oauth2.googleapis.com/token
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): oauth2.googleapis.com:443
Starting new HTTPS connection (1): oauth2.googleapis.com:443
Starting new HTTPS connection (1): oauth2.googleapis.com:443
DEBUG:urllib3.connectionpool:https://oauth2.googleapis.com:443 "POST /token HTTP/1.1" 200 None
https://oauth2.googleapis.com:443 "POST /token HTTP/1.1" 200 None
https://oauth2.googleapis.com:443 "POST /token HTTP/1.1" 200 None
DEBUG:openai._base_client:Request options: {'method': 'post', 'url': '/embeddings', 'files': None, 'post_parser': <function Embeddings.create..parser at 0x7fd7891ffb80>, 'json_data': {'input': ['Who is the author?'], 'model': <OpenAIEmbeddingModeModel.TEXT_EMBED_ADA_002: 'text-embedding-ada-002'>, 'encoding_format': 'base64'}}
Request options: {'method': 'post', 'url': '/embeddings', 'files': None, 'post_parser': <function Embeddings.create..parser at 0x7fd7891ffb80>, 'json_data': {'input': ['Who is the author?'], 'model': <OpenAIEmbeddingModeModel.TEXT_EMBED_ADA_002: 'text-embedding-ada-002'>, 'encoding_format': 'base64'}}
Request options: {'method': 'post', 'url': '/embeddings', 'files': None, 'post_parser': <function Embeddings.create..parser at 0x7fd7891ffb80>, 'json_data': {'input': ['Who is the author?'], 'model': <OpenAIEmbeddingModeModel.TEXT_EMBED_ADA_002: 'text-embedding-ada-002'>, 'encoding_format': 'base64'}}
DEBUG:httpx._client:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
DEBUG:openai._base_client:HTTP Request: POST https://api.openai.com/v1/embeddings "200 OK"
HTTP Request: POST https://api.openai.com/v1/embeddings "200 OK"
HTTP Request: POST https://api.openai.com/v1/embeddings "200 OK"
DEBUG:llama_index.indices.utils:> Top 0 nodes:

Top 0 nodes:

Top 0 nodes:

Empty Response
May I know what make the problem?

should be index_id=vector_id
is that vector id empty or containing enough relevant information?

@hatianzhang Hi thanks for suggestion.
Unfortunately I have tried with index_id=vector_id and even index_id=vector_id. vector_id=vetor_idbut they didnt work either. I shall get back to you with the debug log
I print vector_id and it give me this : 28ddc079-bc01-4947-a543-455dbdb8700a
and I use it with
vector_id='28ddc079-bc01-4947-a543-455dbdb8700a' vector_index = load_index_from_storage(     vector_id=vector_id,storage_context=storage_context )
How do I know if it contains enough relevant information?

@hatianzhang Sorry for late reply.
I have fixed the index_id=vector_id  as below
`` vector_id='0999d828-4f66-46d4-a77c-68c4ce8aba6c'
vector_index = load_index_from_storage(
storage_context=storage_context, index_id=vector_id
)
chatgpt = OpenAI(temperature=0, model="gpt-3.5-turbo")
service_context_chatgpt = ServiceContext.from_defaults(
llm=chatgpt, chunk_size=1024
)
query_engine = vector_index.as_query_engine()
vector_response = vector_index.as_query_engine().query("Who is the author?")
display_response(vector_response) ``
And it still gives "Empty response". The debug log as below
INFO:llama_index.indices.loading:Loading indices with ids: ['0999d828-4f66-46d4-a77c-68c4ce8aba6c']
Loading indices with ids: ['0999d828-4f66-46d4-a77c-68c4ce8aba6c']
DEBUG:google.auth.transport.requests:Making request: POST https://oauth2.googleapis.com/token
Making request: POST https://oauth2.googleapis.com/token
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): oauth2.googleapis.com:443
Starting new HTTPS connection (1): oauth2.googleapis.com:443
DEBUG:urllib3.connectionpool:https://oauth2.googleapis.com:443 "POST /token HTTP/1.1" 200 None
https://oauth2.googleapis.com:443 "POST /token HTTP/1.1" 200 None
DEBUG:openai._base_client:Request options: {'method': 'post', 'url': '/embeddings', 'files': None, 'post_parser': <function Embeddings.create..parser at 0x7fbf4b87b430>, 'json_data': {'input': ['Who is the author?'], 'model': <OpenAIEmbeddingModeModel.TEXT_EMBED_ADA_002: 'text-embedding-ada-002'>, 'encoding_format': 'base64'}}
Request options: {'method': 'post', 'url': '/embeddings', 'files': None, 'post_parser': <function Embeddings.create..parser at 0x7fbf4b87b430>, 'json_data': {'input': ['Who is the author?'], 'model': <OpenAIEmbeddingModeModel.TEXT_EMBED_ADA_002: 'text-embedding-ada-002'>, 'encoding_format': 'base64'}}
DEBUG:httpx._client:HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
HTTP Request: POST https://api.openai.com/v1/embeddings "HTTP/1.1 200 OK"
DEBUG:openai._base_client:HTTP Request: POST https://api.openai.com/v1/embeddings "200 OK"
HTTP Request: POST https://api.openai.com/v1/embeddings "200 OK"
DEBUG:llama_index.indices.utils:> Top 0 nodes:

Top 0 nodes:

Final Response: Empty Response
I have checked with the index in firestore and it looks like this:

May I know if it containing enough relevant information?


False positives about missing props for functional components

With the current version the code below gives false positives about missing proptypes for all props.
import { PropTypes } from 'react'
const { element, node, string } = PropTypes

Card.propTypes = {
  title: string.isRequired,
  children: element.isRequired,
  footer: node
}

export default function Card ({ title, children, footer }) {
  return (

  )
}

eslint 2.2.0
eslint-plugin-react 4.1.0
eslint-config-standard-react 2.3.0
.eslintrc
{
  "parser": "babel-eslint",
  "extends": ["standard", "standard-react"],
  "rules" : {
    "react/react-in-jsx-scope": 0,
    "padded-blocks" : 0,
    "space-before-function-paren": 0,
    "no-multi-spaces": 0,
    "key-spacing": 0,
    "arrow-parens": 0
  }
}

I am still seeing false positives with the following syntax.  Not sure what I'm missing:
import React, { PropTypes } from 'react';
import {
    ProgressBar
} from 'react-bootstrap';

const TableProgressBar = React.createClass({

    render () {
        return (

                    <ProgressBar active now={100} />

{this.props.message}

        )
    }
});

TableProgressBar.propTypes = {
    message: PropTypes.string
};

export default TableProgressBar

So it looks like this might not be supported syntax according to this:
https://github.com/yannickcr/eslint-plugin-react/blob/master/docs/rules/prop-types.md
Looks like the Component.propTypes = ... is only supported with stateless functional components, but not with React.createClass.  I sorta feel like it should be though.  Thoughts?  If so, I can create a PR

It definitely shouldn't be because I'm pretty sure that doesn't work. With React.createClass, you need to include propTypes in the actual object passed to React.createClass.

No, you don't have to do it that way.  My entire app is structured like my example above and it works fine.
I think this prop-types rule is a great rule and I want to be able to apply it without restructuring every component in my application.

That's entirely undocumented and discouraged (since createClass is deprecated anyways) - I'm not sure it's a good thing to support.

React.createClass is not deprecated nor can I find anywhere that says this syntax is discouraged. But no worries. I'll refactor to adhere to the supported syntax.


tpl: Make the safeHTML and cousins accept more than strings

NOTE: This depends on https://github.com/spf13/cast/pull/22

3a2a4c3


Improve verbose output - print what file is being copied before the copy starts

What is your rclone version (eg output from rclone -V)
1.34
Which OS you are using and how many bits (eg Windows 7, 64 bit)
macOS 10.12.1, 64-bit
Which cloud storage system are you using? (eg Google Drive)
Amazon Drive
The command you were trying to run (eg rclone copy /tmp remote:tmp)
/usr/local/bin/rclone copy --verbose --stats 0 "/Volumes/Anime" "amazon:/Backup/Anime"
A log from the command with the -v flag (eg output from rclone -v copy /tmp remote:tmp)
2016/11/13 19:15:20 amazon drive root 'Backup/Anime': Waiting for transfers to finish
2016/11/13 19:35:03 Series/Working!/Season 4/WWW.Working!! - 07.mkv: Copied (new)

When using verbose output (useful from scripts), nothing is printed before a file is copied, only when it is complete. This can result in extremely long delays on large files during which the user doesn't know what is happening. I checked the complete output for the logged session above, and the file "WWW.Working!! - 07.mkv" was never mentioned prior to that entry. Note the long time (~20min) between the "Waiting for transfers to finish" and the "Copied" note.

+1, I've been bugged by this behavior as well.

Probably more than any other issue I've filed, this one is most important to me. Right now it's impossible to tell what rclone is doing, as the only thing on screen is the last thing it did. Hoping this is literally a one-line print statement.

Note that if you haven't disabled them the rclone stats show which files are transferring. You can change the frequency of them with the --stats flag.
Does that help?

Not really. I use the log for one key reason - to see a summary of what changed. When a job is done I want to be able to review everything that happened - copies, deletes, errors and so forth. The stats view doesn't really work for this as it's periodic and truncates the output. The verbose output is better (although I have to do massive filtering down of its output to get to what I want), but it suffers from the problem mentioned here - nothing is printed when a copy begins, only when it ends. That means when copying large files the output just stops, and you may not know what's going on for half and hour or more.


compose: Support channel in transforming stream-topic links.

This PR extends the work in #29302 to support the new "channel" operator,
which is synonymous to the "stream" operator, in transforming stream-topic urls
to the pretty syntax.

Screenshots and screen captures:

Self-review checklist

[x] Self-reviewed the changes for clarity and maintainability
(variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

[ ] Explains differences from previous plans (e.g., issue description).
[ ] Highlights technical choices and bugs encountered.
[ ] Calls out remaining decisions and concerns.
[ ] Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

[x] Each commit is a coherent idea.
[x] Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

[x] Visual appearance of the changes.
[ ] Responsiveness and internationalization.
[ ] Strings and tooltips.
[x] End-to-end functionality of buttons, interactions and flows.
[ ] Corner cases, error conditions, and easily imagined bugs.

Merged, thanks @kuv2707!


Column name and codebase check

This PR includes the following:

check the column names of the parsed EVL and EVR files to make sure we confirm with what Echoview uses
make TODO notes and raise issues on things that need overhaul or consolidation in the codebase
add lightweight test files into echoregions/test_data
clean up unused class attributes

I'll update this message as I draft this PR.

Column name check results:

Column name on EchoView: current name in echoregions

ID: region_id
Name: region_name
Selected: region_selected
Type: region_type
Class: region_classification
Notes: region_notes
Start: bounding_rectangle_left
End: bounding_rectangle_right
Dimension: ??

shows up as "2D" in Echoview but there is no corresponding fields in the file
probably region_structure_version="13" or file_type="EVRG"? (these are hard-coded for EVR files, according to Echoview documentation)

Detection Settings: region_detection_setting

Not showing up in Echoview but obviously used in their internal processing:

point_count
region_creation_type
bounding_rectangle_calculated
bounding_rectangle_top
bounding_rectangle_bottom


DartPad it's gonna huge thing, for that I suggest this ultimate suggestions !

Hello Dart & Flutter !
I really love waht are you doing and your huge success, so I'm happy to here with you.
I'm an absolute beginner but I do my to be in the line of this big progress of what can Dart do in the world of digital realtion with Flutter to Ui's interfaces technology innovation.
We have a big porject that I decide to be Dart & Flutter with Firebase and Google Cloud our main tools to make this project run here in Morocco.
For that after your last update about Firebase intergartion on DartPad and hiw much that I prefer this tool to make real production nut just in VSCode or Android Studio.
Based on this that's why I want to make this important suggests:

Make the DartPad a new IDE for Dart & Flutter With Firebase and also Google Cloud Console (Shell) for people and companies that focusing just in Dart & Flutter Technology. We saw how MS VSCode is running on the Web especially the Google Cloud Shell to make all the stuff just in the web without any local environement, so why not it will interetsing for us.
Dart & Flutter and Firebase is a full stack development process, and as I explaine in the first suggestion to make DartPad a new IDE based on Web and Hybrid Machines and also to be available for Windows, macOS and Linux (Or maybe as an application Mobile to make fun ). For that we want to bring all tools on DartPad like Packages, Extensions from other technologies integrations and more like that
Develop more features on development process just like in VSCode for example and bring the ability to test real apps with Hot Reload, Restart, DevTools, etc ...
If you will do this it's gonna be great !

With all my respects for your efforts !
best Regards

Thanks for the feedback!
DartPad is really intended to just be a playground for Dart and Flutter. dartpad.dev is a great place to share snippets or try out an idea or play with syntax, and the code also supports embedded pads in documentation and articles, and workshops for education.
Beyond that, I think DartPad is not the right starting place for a Web-based IDE for Dart/Flutter/Firebase projects. As you say, the new https://vscode.dev/ product is inspired, and Dart has really great support in most popular IDEs. There is probably a different starting point for writing such a web IDE.
That being said, to your last point:

Develop more features on development process just like in VSCode for example and bring the ability to test real apps with Hot Reload, Restart, DevTools, etc ...

These ideas are well within the realm of what DartPad aims to do, and I can see us supporting these features with high priority, in order to showcase the power of Dart and Flutter. Search the issue tracker for relevant individual issues and Thumbs Up your favorite asks.
CC @RedBrogdon

Thanks for your suggestions, @ImadBouirmane. Unfortunately I think a number of them are out of scope for the size of the team we have (while I'd love to more online IDEs featuring Flutter and Dart, we definitely aren't staffed to turn DartPad into one).
Some of your suggestions are likely to come to DartPad eventually, though, such as hot reload.


API Get Profile is required authentication ?

I tried calling GET https://conduit.productionready.io/api/profiles/:username without Token and it's returned 401 Unauthorized.
But from your API Spec. It's said that "Authentication optional". So, it should return profile if found or 404 if not.
Is there anything wrong or am I misunderstanding ??

Hi @koyoyo 👋
I was able to run the following curl with no authentication and didn't need to be authorized.  I used Postman and attached local Token to your testing site.


patch_size

I found that the patch_size of the network setting uses the initial 1, then the pixel will become a token. What is the reason for not using the image block (e.g, 4*4) as the token?

Please refer to #14

Your idea deserves a try. To be honest, I didn't try it because I believe image restoration is a very local problem. We should process the image pixel by pixel (attention between pixels). From my intuition, there should have a performance drop if we process them patch by patch (attention between patches such as what IPT does). Imagine that the patch embedding process (e.g., using a strided convolution layer) is not invertible. The spatial information within a patch is lost, which may have extreme burden for the model to reconstruct pixels (pixel is our final goal) given a patch feature.
However, it can reduce the computation burden if we do so. You can try to use it and see the results.

Your idea deserves a try. To be honest, I didn't try it because I believe image restoration is a very local problem. We should process the image pixel by pixel (attention between pixels). From my intuition, there should have a performance drop if we process them patch by patch (attention between patches such as what IPT does). Imagine that the patch embedding process (e.g., using a strided convolution layer) is not invertible. The spatial information within a patch is lost, which may have extreme burden for the model to reconstruct pixels (pixel is our final goal) given a patch feature.
However, it can reduce the computation burden if we do so. You can try to use it and see the results.

Feel free to open it if you have more questions.

Dose embed_dim in module 'PatchEmbed' work, if patch_size is 1? For example, image_size is 224, patch_size is 1, in_chans is 3, embed_dim is 96, what is the size of the output?


setting STOP_IF_NO_JSS_UPLOAD=False

Within the JssImporter folder, in which file do i set STOP_IF_NO_JSS_UPLOAD=False?

Please read the wiki FAQs - link


Tensornet representation

Description
Implement modelforge TensorNetRepresentation module, which matches the output of TensorEmbedding forward. In original implementation, TensorEmbedding is used by TensorNet class, where forward is called whenever updating TensorNet. The output of TensorEmbedding corresponds to "X" tensor in the TensorNet paper.
The test system is created according to the forward test case of TensorNet.
Todos

[x] Set up TensorNet embedding module
[x] Implement modelforge TensorNetRepresentation
[ ] Rename parameters from TensorNet with more intuitive names
[x] Match output X tensors

Questions

[ ] static_shapes
[ ] What does the steps mean when static_shapes==True?

Status

[ ] Ready to go

@wiederm
How edge_index, edge_vec, edge_weight are calculated are uncertain. The problem being:

edge_weight may not have units as distances, which causes confusion in the unit transformations in RSF.
The calculation is defined in torchmdnet.extensions.__init__, where torch.ops.torchmdnet_extensions.get_neighbor_pairs is called. I am unable to further trace back how this function is implemented.

@wiederm
It seems to me that input wrappers (NNPInput and *NeuralNetworkData) are trying to generalize the information needed to form a model in separated layers. However, I can't find where box and batch in TensorNet should be put. Which data structure should be modified? Or how do we manage input that couldn't be generalized in designed ways?
Just as a reminder, edge_index, edge_weight, edge_vec = self.distance(pos, batch, box) is in the forward function of class TensorNet in torchmd-net.

@wiederm
TensorNet uses pairlist with duplicated permute indices (0-1 and 1-0 are both included) and has a different order of these indices. But other than that, it seems that edge_weight==d_ij; edge_vec==r_ij. But is there an easier way to verify that? torch.allclose doesn't work directly since they are not the same in terms of dimensions and order.

@wiederm
I added a representation unit system to calculate the model with both angstrom and nanometer. Angstrom is used to compare the results with TensorNet, while I am also curious whether using nanometer inside makes a difference during training.
The only change I made that may affects 'main' branch is that 'CosineCutoff' now has a 'representation_unit' parameter that is set default with 'unit.nanometer'. (So I would assume that this won't even change anything outside tensornet.)

[ ] Does ANI use nanometer by their original implementation? If not, how did you manage to test ANI without using some inside unit transformation?

It seems to me that input wrappers (NNPInput and *NeuralNetworkData) are trying to generalize the information needed to form a model in separated layers. However, I can't find where box and batch in TensorNet should be put. Which data structure should be modified? Or how do we manage input that couldn't be generalized in designed ways?

We discussed this offline: edge_weight is the pairwise distance, and edge_vec is the pairwise distance vector. Box vector and batch are passed to the input and don't need to be generated. The data structure is the NNPInput dataclass.

TensorNet uses pairlist with duplicated permute indices
So does modelforge! All pairwise interactions are listed, that means every atom $i$ encounters every atom $j$ withing.a neighborhood.

ANI use nanometer by their original implementation

No, ANI uses angstrom. Testing is pretty simple, if you have a class that implements some transformation you can simply do:
input_data_in_angstrom = torch.abs(torch.randn(5,1)) * 5 # generate random data on interval [0,5]
reference_implementation = RefClass()
modelforge_implementation = MfClass()

assert torch.allclose(reference_implementation(input_data_in_angstrom/10)*10, modelforge_implementation(input_data_in_angstrom)) # different length units are simple scaling factors, if you scale the input and perform the invers scaling operation on the output you have undone the scaling operation

@wiederm
I may found a bug in TensorNet. They did apply cutoff function twice in their code, which is inconsistent with their paper. I currently sticked to they way they coded it, but we may need a further discussion on this issue.
In class 'ExpNormalSmearing':
def forward(self, dist):
    dist = dist.unsqueeze(-1)
    return self.cutoff_fn(dist) * torch.exp(
        -self.betas
        * (torch.exp(self.alpha * (-dist + self.cutoff_lower)) - self.means) ** 2
    )

The output of this forward function is called 'edge_attr' in 'TensorEmbedding'. This should be $e_k^{RBF}$ in their paper, without cutoff function applied.
However, in 'TensorEmbedding':
def _get_tensor_messages(
    self, Zij: Tensor, edge_weight: Tensor, edge_vec_norm: Tensor, edge_attr: Tensor
) -> Tuple[Tensor, Tensor, Tensor]:
    C = self.cutoff(edge_weight).reshape(-1, 1, 1, 1) * Zij
    eye = torch.eye(3, 3, device=edge_vec_norm.device, dtype=edge_vec_norm.dtype)[
        None, None, ...
    ]
    Iij = self.distance_proj1(edge_attr)[..., None, None] * C * eye
    Aij = (
        self.distance_proj2(edge_attr)[..., None, None]
        * C
        * vector_to_skewtensor(edge_vec_norm)[..., None, :, :]
    )
    Sij = (
        self.distance_proj3(edge_attr)[..., None, None]
        * C
        * vector_to_symtensor(edge_vec_norm)[..., None, :, :]
    )
    return Iij, Aij, Sij

Cutoff function C is timed again to I, A, and S.

@wiederm
The other thing that bothers me is that mathematically, Iij shouldn't be initialized with identity matrix, rather, it should be $\frac{1}{3}\rVert v\lVert^2\cdot I$ based on my calculation.

@wiederm I may found a bug in TensorNet. They did apply cutoff function twice in their code, which is inconsistent with their paper. I currently sticked to they way they coded it, but we may need a further discussion on this issue.
In class 'ExpNormalSmearing':
def forward(self, dist):
    dist = dist.unsqueeze(-1)
    return self.cutoff_fn(dist) * torch.exp(
        -self.betas
        * (torch.exp(self.alpha * (-dist + self.cutoff_lower)) - self.means) ** 2
    )

The output of this forward function is called 'edge_attr' in 'TensorEmbedding'. This should be ekRBF in their paper, without cutoff function applied.
However, in 'TensorEmbedding':
def _get_tensor_messages(
    self, Zij: Tensor, edge_weight: Tensor, edge_vec_norm: Tensor, edge_attr: Tensor
) -> Tuple[Tensor, Tensor, Tensor]:
    C = self.cutoff(edge_weight).reshape(-1, 1, 1, 1) * Zij
    eye = torch.eye(3, 3, device=edge_vec_norm.device, dtype=edge_vec_norm.dtype)[
        None, None, ...
    ]
    Iij = self.distance_proj1(edge_attr)[..., None, None] * C * eye
    Aij = (
        self.distance_proj2(edge_attr)[..., None, None]
        * C
        * vector_to_skewtensor(edge_vec_norm)[..., None, :, :]
    )
    Sij = (
        self.distance_proj3(edge_attr)[..., None, None]
        * C
        * vector_to_symtensor(edge_vec_norm)[..., None, :, :]
    )
    return Iij, Aij, Sij

Cutoff function C is timed again to I, A, and S.

@wiederm
The pairwise indices used in torchmd-net is different from that in tensornet, resulting in different dimensions inputed into nn.Linear, causing different output.
In modelforge, if we have 3 atoms [0, 1, 2, 3], pair_indices is: [[0, 0, 0, 1, 1, 2], [1, 2, 3, 2, 3, 3]]; meanwhile, in torchmd-net, edge_index is: [[1, 2, 2, 3, 3, 3, 0, 0, 1, 0, 1, 2], [0, 0, 1, 0, 1, 2, 1, 2, 2, 3, 3, 3]]. It is not a big problem since I just need to attach the other way around to the end. However, the radial symmetry vector corresponding to the indices needs to be input into a nn.Linear layer, and in the previous example, the output contains 6 v.s. 12 atom pairs.
The real problem here is that in torchmd-net, since the first 6 pairs are the same as the last 6 pairs, I assume the output of that linear layer should also be the case (the first 6 tensors are the same as the last 6 tensors), while it's not the case. Thus in later summation, results from two ways are different.
One way that can definitely fix this problem is to convert the modelforge index system to the torchmd-net one inside representation module. But I think that may not be what we want in modelforge. We should discuss how we can solve the problem.

Note the boolean set in the Pairlist: https://github.com/choderalab/modelforge/blob/5a312b0fe1c9383d7d4fef4651a1e36088a47e53/modelforge/potential/models.py#L56
If this is set to True (the default property) it will return the same number of pairs as the pairlist in tensornet. ANI is an outlier and requires this boolean to be set to False.

I am not compley sure that I follow this:

The real problem here is that in torchmd-net, since the first 6 pairs are the same as the last 6 pairs, I assume the output of that linear layer should also be the case (the first 6 tensors are the same as the last 6 tensors), while it's not the case. Thus in later summation, results from two ways are different.

Can you provide a minimum example that examplifies your concern?

If possible, it might be very useful to discuss this in a working test

Note the boolean set in the Pairlist:
https://github.com/choderalab/modelforge/blob/5a312b0fe1c9383d7d4fef4651a1e36088a47e53/modelforge/potential/models.py#L56
If this is set to False (the default property) it will return the same number of pairs as the pairlist in tensornet. ANI is an outlier and requires this boolean to be set to True.

Thanks! It has been fixed by now.

@wiederm I may found a bug in TensorNet. They did apply cutoff function twice in their code, which is inconsistent with their paper. I currently sticked to they way they coded it, but we may need a further discussion on this issue.
In class 'ExpNormalSmearing':
def forward(self, dist):
    dist = dist.unsqueeze(-1)
    return self.cutoff_fn(dist) * torch.exp(
        -self.betas
        * (torch.exp(self.alpha * (-dist + self.cutoff_lower)) - self.means) ** 2
    )

The output of this forward function is called 'edge_attr' in 'TensorEmbedding'. This should be ekRBF in their paper, without cutoff function applied.
However, in 'TensorEmbedding':
def _get_tensor_messages(
    self, Zij: Tensor, edge_weight: Tensor, edge_vec_norm: Tensor, edge_attr: Tensor
) -> Tuple[Tensor, Tensor, Tensor]:
    C = self.cutoff(edge_weight).reshape(-1, 1, 1, 1) * Zij
    eye = torch.eye(3, 3, device=edge_vec_norm.device, dtype=edge_vec_norm.dtype)[
        None, None, ...
    ]
    Iij = self.distance_proj1(edge_attr)[..., None, None] * C * eye
    Aij = (
        self.distance_proj2(edge_attr)[..., None, None]
        * C
        * vector_to_skewtensor(edge_vec_norm)[..., None, :, :]
    )
    Sij = (
        self.distance_proj3(edge_attr)[..., None, None]
        * C
        * vector_to_symtensor(edge_vec_norm)[..., None, :, :]
    )
    return Iij, Aij, Sij

Cutoff function C is timed again to I, A, and S.

Was this resolved? It seems that they wanted to make sure that the outputs of intermediate linear layers also decay to 0 at the cutoff distance, even before they multiply by the cutoff-filtered distances.

@wiederm I may found a bug in TensorNet. They did apply cutoff function twice in their code, which is inconsistent with their paper. I currently sticked to they way they coded it, but we may need a further discussion on this issue.
In class 'ExpNormalSmearing':
def forward(self, dist):
    dist = dist.unsqueeze(-1)
    return self.cutoff_fn(dist) * torch.exp(
        -self.betas
        * (torch.exp(self.alpha * (-dist + self.cutoff_lower)) - self.means) ** 2
    )

The output of this forward function is called 'edge_attr' in 'TensorEmbedding'. This should be ekRBF in their paper, without cutoff function applied.
However, in 'TensorEmbedding':
def _get_tensor_messages(
    self, Zij: Tensor, edge_weight: Tensor, edge_vec_norm: Tensor, edge_attr: Tensor
) -> Tuple[Tensor, Tensor, Tensor]:
    C = self.cutoff(edge_weight).reshape(-1, 1, 1, 1) * Zij
    eye = torch.eye(3, 3, device=edge_vec_norm.device, dtype=edge_vec_norm.dtype)[
        None, None, ...
    ]
    Iij = self.distance_proj1(edge_attr)[..., None, None] * C * eye
    Aij = (
        self.distance_proj2(edge_attr)[..., None, None]
        * C
        * vector_to_skewtensor(edge_vec_norm)[..., None, :, :]
    )
    Sij = (
        self.distance_proj3(edge_attr)[..., None, None]
        * C
        * vector_to_symtensor(edge_vec_norm)[..., None, :, :]
    )
    return Iij, Aij, Sij

Cutoff function C is timed again to I, A, and S.

Was this resolved? It seems that they wanted to make sure that the outputs of intermediate linear layers also decay to 0 at the cutoff distance, even before they multiply by the cutoff-filtered distances.

Yes, this has been resolved! We came to the same conclusion and left it in the code.

Closing PR since it is superseeded by PR #181


Improved naive bayes model

Linked Issue
Closes: #7
Describe the changes you've made
I have improved the existing Naive Bayes model by 1% after analyzing the trend in the dataset.
Type of change
What sort of change have you made:

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Code style update (formatting, local variables)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?

Initially plotted the SNS Heat and Cluster Map.
Found that the Uniformity of cell size column is highly correlated which is causing the model to perform poorly.
After removing it can see a 1% betterment in the Naive Bayes Model and also a significant improvement in other models excluding the logistic regression model.

Checklist:

[x] My code follows the guidelines of this project.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly wherever it was hard to understand.
[x] I have made corresponding changes to the documentation.
[x] My changes generate no new warnings.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] Any dependent changes have been merged and published in downstream modules.

Screenshots

Original
Updated

It has actually improved the f1 score and accuracy. Since it's rounded off to 2 decimals there ain't a difference in CM. It has performed better in the test dataset rather than in the training dataset.
Also, my bad that I had made a PR with the wrong file since I had many files while trying to improve.


Missing device configuration: Lockly GUARD DEADBOLT 728F-ZU

Checklist

[X] It is not in the configuration DB

[X] It was not merged recently or has a pending PR

[X] The device was interviewed completely and has no missing or undefined IDs

Which device is missing?
LOCKLY GUARD Z-Wave version smart lock
Manufacturer ID
0x043D
Product Type
0x03
Product ID
0x0077
Firmware Version
04.04.00
Is the device listed on the Z-Wave Alliance website?
https://products.z-wavealliance.org/products/4716
Do you have a manual?
No response

Technical manual

The link you shared seems to be for the PGD728F ZPU, not the PGD728F ZU. I can't find the latter on the ZWA page, so we have to go purely by the manual here.

The link you shared seems to be for the PGD728F ZPU, not the PGD728F ZU. I can't find the latter on the ZWA page, so we have to go purely by the manual here.

I noticed that and made a best effort “closest” link.  I reached out to the vendor for support and they emailed me a PDF so I will attach that

Vendor-provided manual seems to match the pictures I took; I'll update the description.
Zwave Technical Manual.pdf


Support serde derivated structures out of the box

It would be great to support serde structures to create the UI, this will make it much simpler and take advantages of all crates that already use serde.

Yep, it would be great to support all serde types.
However, I think it is impossible to do this directly (like impl EguiStruct for T, where T: Serde; it will lead to conflicting trait implementations), but via some wrapper(struct EguiStructSerdeWrapper(impl Serde)) it may be possible.
I will investigate this some day, but rather not in the nearest future.
As of now, for immutable view egui_json_tree could be used (it's far from perfection in this use case, but may be sometimes enough).

As of now, for immutable view egui_json_tree could be used (it's far from perfection in this use case, but may be sometimes enough).

Crate egui_json_tree recently supports editing the Json tree as well, which makes it a reasonable generic replacement. Still egui_struct would be more reasonable for a generic-purpose property editor but it would need at least vec (and maybe hashset) element add/remove support.


⚠ Bofrost.de has degraded performance

In f7206c1, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1406 ms

Resolved: Bofrost.de performance has improved in 61fed20 after 10 minutes.