Lucene 9.9: Benchmark HNSW improvements

Follow up to this: https://github.com/castorini/anserini/issues/2288#issuecomment-1840803314
It'd be great to benchmark HNSW, reproducing the experiments here: Vector Search with OpenAI Embeddings: Lucene Is All You Need. arXiv:2308.14963.
It's actually quite simple, we've gotten it down to a single command:
python src/main/python/run_regression.py --download --index --verify --search --regression msmarco-passage-openai-ada2

See: https://github.com/castorini/anserini/blob/master/docs/regressions/regressions-msmarco-passage-openai-ada2.md
@jpountz @benwtrent @ChrisHegarty Do either of you have cycles to try it out?

I’ll try to find time to run this.

Reporting on experiments with code on #2302 at commit 0b9f79e "Added -noMerge option".
MS MARCO v1 passage, dev queries, cosDPR-distil model. This is the model reported here:

Anserini Gets Dense Retrieval: Integration of Lucene's HNSW Indexes
End-to-End Retrieval with Learned Dense and Sparse Representations Using Lucene

Experimental Setup
Comparing HNSW fp32 vs. int8. In both cases, I'm forcing no merge to reduce variance on indexing times (which varies from trial to trial depending on which segments get selected for merging). For both, we end up with 16 segments.
Experiments on my Mac Studio, stats:

Mac Studio, M1 Ultra (20 cores; 16 performance and 4 efficiency), 128 GB RAM
macOS Sonoma 14.1.2; openjdk 11.0.13 2021-10-19.
Using 16 threads.

Four trials for each condition. Commands:
python src/main/python/run_regression.py --index --verify --search --regression msmarco-passage-cos-dpr-distil-hnsw >& logs/log.msmarco-passage-cos-dpr-distil-hnsw.lucene990.fp32.1
python src/main/python/run_regression.py --index --verify --search --regression msmarco-passage-cos-dpr-distil-hnsw-int8 >& logs/log.msmarco-passage-cos-dpr-distil-hnsw.lucene990.int8.1

python src/main/python/run_regression.py --index --verify --search --regression msmarco-passage-cos-dpr-distil-hnsw >& logs/log.msmarco-passage-cos-dpr-distil-hnsw.lucene990.fp32.2
python src/main/python/run_regression.py --index --verify --search --regression msmarco-passage-cos-dpr-distil-hnsw >& logs/log.msmarco-passage-cos-dpr-distil-hnsw.lucene990.fp32.3
python src/main/python/run_regression.py --index --verify --search --regression msmarco-passage-cos-dpr-distil-hnsw >& logs/log.msmarco-passage-cos-dpr-distil-hnsw.lucene990.fp32.4

python src/main/python/run_regression.py --index --verify --search --regression msmarco-passage-cos-dpr-distil-hnsw-int8 >& logs/log.msmarco-passage-cos-dpr-distil-hnsw.lucene990.int8.2
python src/main/python/run_regression.py --index --verify --search --regression msmarco-passage-cos-dpr-distil-hnsw-int8 >& logs/log.msmarco-passage-cos-dpr-distil-hnsw.lucene990.int8.3
python src/main/python/run_regression.py --index --verify --search --regression msmarco-passage-cos-dpr-distil-hnsw-int8 >& logs/log.msmarco-passage-cos-dpr-distil-hnsw.lucene990.int8.4

Results
Indexing:
# fp32
Total 8,841,823 documents indexed in 00:31:04
Total 8,841,823 documents indexed in 00:31:08
Total 8,841,823 documents indexed in 00:30:34
Total 8,841,823 documents indexed in 00:30:24
                                 avg 00:30:48

# int8
Total 8,841,823 documents indexed in 00:35:55
Total 8,841,823 documents indexed in 00:35:32
Total 8,841,823 documents indexed in 00:35:42
Total 8,841,823 documents indexed in 00:36:15
                                 avg 00:35:51
                                 +16% slowdown

Retrieval:
# fp32
6980 queries processed in 00:02:33 = ~45.34 q/s
6980 queries processed in 00:02:33 = ~45.57 q/s
6980 queries processed in 00:02:33 = ~45.33 q/s
6980 queries processed in 00:02:33 = ~45.55 q/s
                                  avg 45.45 q/s

# int8
6980 queries processed in 00:01:48 = ~64.54 q/s
6980 queries processed in 00:01:48 = ~64.50 q/s
6980 queries processed in 00:01:48 = ~64.56 q/s
6980 queries processed in 00:01:48 = ~64.45 q/s
                                  avg 64.51 q/s
                                   +42% speedup

Finally, effectiveness: remains the same, no difference.

Finally, effectiveness: remains the same, no difference.

🎉

 +16% slowdown

On index and initial flush, this isn't surprising. We build the graph with float32 and then have the small additional overhead of calculating quantiles and storing the quantized representation of everything.
But, I have noticed that merging is faster (about 30-40%).

@benwtrent merging is difficult to benchmark because I get high variance in running times... my diagnosis is that running time is idiosyncratically dependent on which segments get selected for merging... is this a convincing explanation?

@lintool I understand :)
I am happy to see that effectiveness is still, well, effective.

Hrm... update: trying the same experiments, but with OpenAI embeddings. Getting errors:
...
2023-12-14 20:46:43,421 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:260) - 53.93% of files completed, 4,830,000 documents indexed
[2336.220s][warning][gc,alloc] pool-2-thread-16: Retried waiting for GCLocker too often allocating 244355 words
2023-12-14 20:47:44,104 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:260) - 53.93% of files completed, 4,830,000 documents indexed
[2373.702s][warning][gc,alloc] pool-2-thread-8: Retried waiting for GCLocker too often allocating 249714 words
2023-12-14 20:48:44,511 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:260) - 53.93% of files completed, 4,830,000 documents indexed
2023-12-14 20:49:46,485 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:260) - 53.93% of files completed, 4,830,000 documents indexed
2023-12-14 20:50:49,971 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:260) - 53.93% of files completed, 4,830,000 documents indexed
2023-12-14 20:51:52,476 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:260) - 53.93% of files completed, 4,830,000 documents indexed
2023-12-14 20:52:53,176 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:260) - 53.93% of files completed, 4,830,000 documents indexed
2023-12-14 20:53:56,204 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:260) - 53.93% of files completed, 4,830,000 documents indexed
2023-12-14 20:54:58,288 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:260) - 53.93% of files completed, 4,830,000 documents indexed
[2792.142s][warning][gc,alloc] pool-2-thread-5: Retried waiting for GCLocker too often allocating 249321 words
[2793.036s][warning][gc,alloc] pool-2-thread-8: Retried waiting for GCLocker too often allocating 249713 words
2023-12-14 20:56:13,898 INFO  [main] index.AbstractIndexer (AbstractIndexer.java:260) - 53.93% of files completed, 4,830,000 documents indexed
[2869.427s][warning][gc,alloc] pool-2-thread-9: Retried waiting for GCLocker too often allocating 247479 words
[2872.043s][warning][gc,alloc] pool-2-thread-3: Retried waiting for GCLocker too often allocating 237701 words

@benwtrent any ideas? cc/ @ChrisHegarty @jpountz
How should I start debugging?

do these warnings result in indexing errors? or is it "just" polluting the output logs?
it might be the GC not managing to free up enough memory (also considering the larger embeddings' size of 1536), either we can give the JVM a larger heap or allow more retries from the GC (e.g., passing -XX:GCLockerRetryAllocationCount=100 to the indexing java command)

We're still having issues with openai-ada2-int8, but #2302 (Lucene 9.9.1) has been merged into master. Will create new issue focused on debugging this.

Ref: #2314

Ref: #2318

No further follow-up, closing.


Not storing empty MVT tiles

Is it somehow possible to check MVT (I am aware that geojson can be easily checked) tile after it is fetched from tile server that it does not contain any data and not store it into MBTiles or zip file? Because in some areas there are plenty of empty tiles in higher zoom levels and I think that it is not necessary to store them. By empty tiles I mean that they does not contain any features.

MBTiles has a feature that may take care of this for you automatically
(it’s useful for “empty” “water" tiles), we’d would need to confirm that’s
working for this config – you could verify that with existing setup. For
ZIP that’d be a new feature, we’re happy to take pull requests here.
Caution that your technique must be paired with a map style that default
“no tile data” to either a “land" color or a “water” color else the map
will look like it has holes in it. For most (all?) Mapzen basemap styles
that’s specified for the land (e.g.: scene: { background: { color:
global.earth1 }). You could swap that for water easily enough with a scene
file YAML import or JS mix-in (only the scene file YAML import is supported
at this time on ES).
On Sat, Sep 16, 2017 at 9:59 AM, rwrx<EMAIL_ADDRESS>wrote:

Is it somehow possible to check MVT (I am aware that geojson can be easily
checked) tile after it is fetched from tile server that it does not contain
any data and not store it into MBTiles or zip file? Because in some areas
there are plenty of empty tiles in higer zoom levels and I think that it is
not necessary to store them. By empty tiles I mean that they does not
contain any features.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/tilezen/tilepacks/issues/9, or mute the thread
https://github.com/notifications/unsubscribe-auth/AA0EO5i7_fBEqYG42nKPbN5GeOM9sDGzks5si_51gaJpZM4PZ4Pz
.

I am not sure how to verify it, but to me it seems that it is not taking care of this automatically when I look at generated MBTiles files. They have all the tiles.

This thread has good background reading for you:
https://github.com/osm2vectortiles/osm2vectortiles/issues/42
On Sat, Sep 16, 2017 at 11:20 AM, rwrx<EMAIL_ADDRESS>wrote:

I am not sure how to verify it, but to me it seems that it is not taking
care of this automatically when I look at generated MBTiles files. They
have all the tiles.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/tilezen/tilepacks/issues/9#issuecomment-329986109,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA0EO4bxBBrB-HHrbQA7w9sbhd5SUKqLks5sjBF0gaJpZM4PZ4Pz
.

Thank you a lot for this link. I have added into tilepack code to not store tiles which contains only one earth feature and zero other features. This will basically omit all tiles which are empty "earth" tiles. Thank you. I will close this issue.


Kanban review

Group: @redmarverick @Javed-Shahbaz @SergioPeralta22 @Balikuddembe
Project link: https://github.com/redmarverick/Final-Capstone-Backend/projects/1
Front End Repository: https://github.com/redmarverick/Final-Capstone-Frontend

Hi @redmarverick,
Wow, you did it 🎉

Your project is complete!  There is nothing else to say other than... it's time to merge it :shipit:
Congratulations! 🎉
To highlight

The tasks have descriptions, assignees and have been pointed according to spec✔

Optional suggestions
Every comment with the [OPTIONAL] prefix won't stop the approval of this PR. However, I strongly recommend you to take them into account as they can make your code better. Some of them were simply missed by the previous reviewer and addressing them will really improve your application.
Cheers and Happy coding!👏👏👏
Feel free to leave any questions or comments in the PR thread if something is not 100% clear.
Please, remember to tag me in your question so I can receive the notification.

As described in the Code reviews limits policy  you have a limited number of reviews per project (check the exact number in your Dashboard). If you think that the code review was not fair, you can request a second opinion using this form.


Add LIS3DH accelerometer

Very rough start to add LIS3DH accelerometer. Currently present on the :srhub target

Closing this since it turned out that we didn't use it. I hope this can eventually be turned into a proper Elixir library or perhaps a gist for anyone with the accelerometer.


Pinout of module when using it as a remote

Here is the pinout of the 5 button remote.

Here is the pinout of the ON/OFF remote, if you flash its firmware to the module.

Buttons are pushed by pulling the pin the GND with a 487 ohm resistor. Except paring, which is pushed by short to GND.
LED is powered by a 2N7002ET MOSFET. If the Draco module is used, the MOSFET can be omitted.

The pinout of the module when used in the mains outlet:

Thanks for the information!
Can you confirm its part number? The E1524 remote is already in the overview, but it seems that ikea now sells it as the E1810 (not sure though).

The new remotes contains the Draco module.

Is some one knowing wot the TP1 and TP2 is connected to the module ?
I was hoping its the debug pins for SWD so not need using brute force for dumping the firmware but i have not testing it then i have not opening my remotes and dont like destroying them.


Minor: Extract parent/child limit calculation into a function, improve docs

Which issue does this PR close?

Part of https://github.com/apache/datafusion/issues/10292
Rationale for this change
While reviewing the code for PushDownLimit this logic is slightly tricky and a natural candidate to extract into a function to allow docstrings as well as to make the refactoring to avoid copies slightly easier
What changes are included in this PR?

Extract parent/child limit calculation into a function
improve doc formatting

Are these changes tested?
Existing CI

Are there any user-facing changes?

Thanks for the review @comphead


Fix breaking CBMC proof for DNSgetHostByName_a/_cancel & ReadNameField

Fix the breaking CBMC proof for DNSgetHostByName_a & DNSgetHostByName_cancel & ReadNameField.
Description
Added a #define in all three Makefile.json to declare an essential macro ipconfigDNS_CACHE_NAME_LENGTH used in CBMC proofs which is not defined if (ipconfigUSE_DNS_CACHE == 0).
Previous PR https://github.com/aws/amazon-freertos/pull/1928 disabled the use of DNS cache which broke the proofs.
Checklist:

[x] I have tested my changes. No regression in existing tests.
[ ] My code is Linted.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

It looks like the ReadNameField proof is still failing.

/bot run checks


Cursor manages to leave the screen, hyprland crashes when opening an app in the glitched workspace.

Hyprland Version
 System/Version info
Hyprland, built from branch  at commit 33a7b7bb6b307d6e4a093f75ffdda0419cd7ffaf  ().
Date: 2024-05-11
Tag: , commits:

flags: (if any)

System Information:
System name: Linux
Node name: redyf
Release: 6.6.30
Version: #1-NixOS SMP PREEMPT_DYNAMIC Thu May  2 14:32:50 UTC 2024

GPU information:
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU116 [GeForce GTX 1650 SUPER] [10de:2187] (rev a1) (prog-if 00 [VGA controller])

os-release: ANSI_COLOR="1;34"
BUG_REPORT_URL="https://github.com/NixOS/nixpkgs/issues"
BUILD_ID="24.05.20240509.f1010e0"
DOCUMENTATION_URL="https://nixos.org/learn.html"
HOME_URL="https://nixos.org/"
ID=nixos
IMAGE_ID=""
IMAGE_VERSION=""
LOGO="nix-snowflake"
NAME=NixOS
PRETTY_NAME="NixOS 24.05 (Uakari)"
SUPPORT_URL="https://nixos.org/community.html"
VERSION="24.05 (Uakari)"
VERSION_CODENAME=uakari
VERSION_ID="24.05"

plugins:

Bug or Regression?
Bug
Description
The cursor manages to leave the screen if you drag it to the right side. Switching workspaces doesn't work all the time, and if it doesn't work, the workspace number won't be displayed on waybar. Hyprland crashes when I open an application in the glitched workspace.
Example: I'm on workspace 1 and want to go to workspace 3, I press the keybind to go there and both my workspace number on waybar and the mouse cursor disappear. If I open any application in this "glitched" workspace, hyprland crashes.
How to reproduce
1- Install Hyprland with this flake: inputs.hyprland.url = "git+https://github.com/hyprwm/Hyprland?submodules=1;
2- Drag your cursor to the right side of the screen, it's going to leave the scale.
3- Open any application, Hyprland will crash.
hyprlandCrashReport189536.txt
Crash reports, logs, images, videos
https://github.com/hyprwm/Hyprland/assets/98139059/9d98cad7-5d76-4dbe-8be5-d5bc99ed04d7
https://cdn.discordapp.com/attachments/1094394444574371873/1238997819830833212/issue.mp4?ex=66415223&is=664000a3&hm=90706c9bf8aa60ffab79ca2d93075d2f00fd682688cccc3877ec7c1465f5b148& (discord link because github says the video has been corrupted even though it's working on my machine).
hyprlandCrashReport201115.txt
hyprlandCrashReport189536.txt

After hours looking for an explanation I realized that there is a second monitor being detected on hyprctl monitors (I don't have 2 monitors so I'm confused). That ghost monitor was on position 1920x0 so that's why the cursor could leave the screen, it's like going to the other monitor. Please ignore this issue :+1:

its a recent kernel/driver bug

its a recent kernel/driver bug

I see, that started happening after I updated my system so it makes sense. Spent hours yesterday trying to figure out hahaha. Thanks for telling me


Microsoft.VisualStudioCodeInsiders_v1.47.6.da73d2d115

Microsoft.VisualStudioCodeInsiders_v1.47.6.da73d2d115
Microsoft Reviewers: Open in CodeFlow

Closing as a new pull request has been submitted.


I can't seem to convert even a simple setup.cfg file ?

I have a pretty simple setup.confg file -- only the core project functionality:
It is a setuptools project (which I think is always the case for setup.cfg, but I get this result when I try to run it:
WARNING: setup.cfg: setuptools sections ['metadata', 'options', 'options.entry_points', 'options.package_data']; porting depends on which toolset to use. For example, hatch (https://pypi.org/project/hatch/) can port metadata to pyproject.toml.

I'm confused -- this seems like the very basic, normal stuff -- can't I even get the toml for metadata?
What am I missing?
-Thanks

pyproject-migrator doesn't want to prescribe which toolset to use for packaging; it migrates other things.
If you want to migrate to e.g. Hatch, the "migrating existing documentation" documentation for Hatch says to use hatch new --init. Other packaging toolsets may have migration tools of their own.
You can also even keep using setup.cfg and use setuptools's build backend.

Fair enough -- you may want to make that a bit more clear in the docs.
However, IIUC, some sections are not build-system specific.
metadata for sure, and I think scripts as well.
My goal was to convert a setuptools setup.cfg based setup to a setuptools pyproject.yaml setup. Granted, setuptools should provide this (and oddly doesn't seem to), so what I did was use hatch to convert to a hatch setup, and then swap out the hatch-sepcific parts.
and that worked great for the metadata and scripts sections :-)
So you could provide something quite useful here, if you want to.


[Feature] Rope drawing / rendering

[ ] Fix width implementation in 3D case.
[ ] Update references (Ensure it builds in CI)

There is a major limitation when handling width in the 3D case: when the curve tangent is colinear with the view direction, we should draw the width alongside all directions in the plane (there is not a single well defined normal anymore).


Unclear description in  section "OAuth2 key-value parameters"

Unclear description in section https://learn.microsoft.com/en-us/azure/active-directory-b2c/claim-resolver-overview#oauth2-key-value-parameters
The description specifies that any parameter included as part of an oauth2 request can be mapped to a claim. However, it doesn't seem possible to map {OAUTH-KV:any custom query string} when making requests to ../token

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 4fc94d79-52c2-d03a-9821-32c048ffa020
Version Independent ID: e4303183-d476-6104-fcaf-00e3beb23787
Content: Claim resolvers in custom policies - Azure AD B2C
Content Source: articles/active-directory-b2c/claim-resolver-overview.md
Service: active-directory
Sub-service: b2c
GitHub Login: @kengaderdus
Microsoft Alias: kengaderdus

@sslilllund
I'm going to assign this to the document author so they can take a look at it accordingly.
@kengaderdus
Can you please check and add your comments on this doc update request as applicable.

@sslilllund
We're sorry for the delay. Our product team is investigating this issue. We'll notify you when we've something. Meanwhile, can you share any errors you observed, if any?

@sslilllund
Do you've any errors, if any, or a way to reproduce your scenario?

#please-close. The customer didn't respond.


bugfix/modal-c-transition

Linked Issue
Closes #2239
Description
Fixed the issue by checking the client height and comparing it to the window height and then adding the overflow based on the result.

Note: cOverflow is added directly to the classes of the backdrop because otherwise the reactivity won't be triggered on every change.

Changsets
Instructions: Changesets automate our changelog. If you modify files in /packages, run pnpm changeset in the root of the monorepo, follow the prompts, then commit the markdown file. Changes that add features should be minor while chores and bugfixes should be patch. Please prefix the changeset message with feat:, bugfix: or chore:.
Checklist
Please read and apply all contribution requirements.

[x] This PR targets the dev branch (NEVER master)
[x] Documentation reflects all relevant changes
[x] Branch is prefixed with: docs/, feat/, chore/, bugfix/
[x] Ensure Svelte and Typescript linting is current - run pnpm ci:check
[x] Ensure Prettier linting is current - run pnpm format
[x] All test cases are passing - run pnpm test
[x] Includes a changeset (if relevant; see above)

@endigo9740 this won't solve #1870, but I though separating them will simplify the review.
If this solution is accepted we can close #2241 . Thank you @CoreyT355 for helping with this bug.

@endigo9740 this won't solve #1870, but I though separating them will simplify the review. If this solution is accepted we can close #2241 . Thank you @CoreyT355 for helping with this bug.

@Mahmoud-zino That's a nice solution. I haven't worked with the svelte:window piece yet, so I didn't think about it when the idea of catching the window height came up.
Thanks for finding a solution here. Hopefully this gets merged in.

@CoreyT355 @Mahmoud-zino I've been handling PR reviews on Sunday's (tomorrow for me) so expect a proper audit then.

@Mahmoud-zino just a couple small nitpicks here, but overall I like this. It seems to be a great solution to the issue at hand.

@endigo9740 I just checked it again, this is the effect of having max-h-screen, deleting it solved the issue.

@Mahmoud-zino That's awesome. In order to properly test this, can you do me a favor and implement a new component modal example that is full screen? Just follow the lead of the current examples. Without that we don't have a great way to test all three conditions I mentioned above. The contents of the modal can be anything - even just placeholder text that says: "Full Screen Example". I'll polish it a bit more later.

@Mahmoud-zino I've moved, renamed, and updated the Modal docs to showcase the new "full" modal option. I've also update the contents of the modal to provide some simpler styles for showcasing how this can work.
Note there is currently no way to disable the backdrop > modal-transition built in padding. But I think that's an issue we should address in Skeleton v3.
I also see how you're handling modals that are oversized vertically. The scrollbar won't appear as you resize your window, but will appear if that size is too large when the modal appears. I think that's a happy medium for now. Again, I'd like to tackle the root issues for this in v3.
Assuming you're happy with my changes, feel free to approve this PR and I'll aim to merge right away. Thanks for the hard work on this one!

@endigo9740 I can't approve my own PR, but it looks good to me :+1:


Fix #52, add install and hide combinators

@jdegoes Could you please review?
With this approach, we avoid the pi-types issue by assuming the placeholder type is something small and finite.

I'm good merging this in, at least on an experimental basis until we figure all this stuff out.

^^^ @paf31


macOS has no SNMP support

Is your feature request related to a problem? Please describe.
The PHP installed on macOS has no SNMP support. The SNMP support is baked-in into core PHP and not available as a shared module.
Some time ago SNMP was kicked out of the default homebrew install because of reported crashed while used with Apache caused by linking order. Since this is not a shared extension it cannot be re-installed using pecl.
Describe the solution you'd like
macOS users most often simply add the kabel/homebrew-php-ext tap which contains extensions which were unstable for one reason or another (or conflicted with something more mainstream).
I think this tap should be included in macOS builds.

This can be added as a step after PHP setup in individual workflows as per the requirement of the project.
For example
brew tap kabel/php-ext
brew install<EMAIL_ADDRESS>

Thank you, I didn't think about that :)


[Searchable Snapshots] [Low Level Design] Block Based Storage

This document outlines the low level design proposal for implementing block-based storage. High level design document and proposal: #3869
Overview
Block based file system will enable fetching parts of the Lucene IndexInput files from the snapshot within the repository instead of downloading the entire file on disk — only download the bytes accessed by the query.
BlockedIndexInput
The solution implements a wrapper around the IndexInput class to manage the block calculation, fetching and seeking mechanisms. This wrapper will work as a virtual file which will the utilized by Lucene to read index files and will internally keep track of the necessary blocks and the calculation required to fetch other blocks as per the query.
VirtualFileIndexInput
Additionally, another wrapper will be used to fetch the virtual file data (https://github.com/opensearch-project/OpenSearch/blob/main/server/src/main/java/org/opensearch/repositories/blobstore/BlobStoreRepository.java#L205)which consists of the metadata around the segment files for Lucene. This will work on a “fetch-when-read” basis to download the entire file onto disk when needed by the service.
The design has been broken down into two phases -

Phase 1: This phase will consist of adding in the block based fetching and virtual file mechanism without any cache. To overcome the refetch logic for blocks and files, there will be an on-disk check for the file to verify it has been already fetched previously. This naive implementation does not account the disk cleanup for unused blocks and is solely focused on implementing the block-based fetch logic.
Phase 2: This phase will add in the caching mechanism required to keep the necessary blocks on disk. The implementation will add a wrapper mechanism with a RefCount mechanism to keep track of the files which are in use and will also utilize an eviction mechanism to further optimize disk storage.

Low Level Design
Phase 1: Without Cache
As described above, phase 1 will only implement the mechanisms necessary to enable a block-based fetching mechanism for segment files. In addition to the interface definitions, we will implement the block calculation and fetch logic within the BlockedIndexInput class.
The properties and methods like getBlock*, getCurrentBlock*, blockSize, blockMask, blockSizeShift will be utilized to enable the block calculation logic for the actual segment file described by fileInfo.
The fetchBlock, downloadBlockAsync, downloadTo will handle the fetching logic for the blocks as well as virtual files.

Phase 2: With Cache
The implementation from Phase 1 will be followed up to enable caching for the blocks leading to deletion based on cache eviction for reduced storage needs.
FileCachedIndexInput will implement a RefCount mechanism to keep track of open handles as well as ensure the file is cached within a BlockCache/FileCache.

Directory
ReadOnlyDirectory will be implemented to block writes for the Store and will complement the ReadOnlyEngine for Searchable Snapshots. It will utilize the IndexInput classes described above to open the index for reads for a remote snapshot.

Quality design! Thanks @kotwanikunal
What about the snapshots themselves? today we can create a snapshot under any name, containing any subset of the indices from the cluster. And to make matters more complex - several snapshots could contain the same index. Will this feature require the snapshot to be named a certain way? or that it contains only one index?

Thanks @AmiStrn!
The mechanism that we are currently designing will restore the metadata for the indices within the snapshot when that property is passed in and will skip restoring the segment files.
It will use the same properties as the current snapshot restore process - naming conventions/alias conventions and other restrictions.
Once the metadata restore is complete, the new file system will be utilized to perform block-based fetching of the segment files.
@andrross can chime in if I have missed anything on the API front.

I'm a bit confused by the terminology - block storage vs. object storage. Today snapshots are mainly stored on object storage. Where are the details about implementing block storage over an object storage?

The term "block" is being used a bit loosely here. The high level idea is that when Lucene needs to read a part of a segment file, instead of downloading the entire file onto the local disk cache this will only download the part of the file necessary and store them as logical "blocks" on the local disk. The terminology is perhaps a bit confusing but this doesn't mean that we'll be using a remote block store (e.g. GCP Persistent Disk, AWS EBS, etc). The remote storage remains the object stores (all supported repository implementations).

Good one @kotwanikunal , super minor comment / suggestion to go from BlockedIndexInput to BlockIndexInput: blocked prefix is a bit confusing in this context (at least to me, but that is sibjective for sure). Also fits well into block device, block storage ...

Thanks @reta! Sure. Matching the conventions sounds like a good plan.
I will look into the SlicedIndexInput implementation to check if there is any pre-existing logic that can be wired in. We do have a POC implementation out here (still a WIP and not production ready) - https://github.com/andrross/OpenSearch/pull/101 for the design described above.
It would be great to have any feedback or inputs.

Implemented as a part of #4892


glossaryTerm: tags lost  if added from the "bulk" screen

Affected module
Glossary. Issue noticed in 1.1, still valid in 1.2
Describe the bug
glossaryTerm: tags lost  if added from the "bulk" screen
To Reproduce
I edit glossaryTerm tags from the Glossary:

Even if Client has a tag "tag1"

When editing, from the glossary screen - the tag is not visible and and when I add a new one - the original is lost, as visible from the version history:

Expected behavior
tags are not deleted
Additional context
Add any other context about the problem here.

@karanh37 what operation are we doing here, is it PUT or PATCH


Support function array_to_bitmap in StarRocks 【Easy】

Requirement Description
input: ARRAY_INT ARRAY_SMALLINT ARRAY_TINYINT
output:TYPE_BITMAP
construct a bitmap from array,if input was NULL  then output was null. empty bitmap will be return if array is empty
eg:
array_to_bitmap([1,2,3]) -> bitmap(1,2,3)
array_to_bitmap(NULL) -> NULL
array_to_bitmap([]) -> bitmap()
Mentor
@stdpain
Registration procedure
Step 1. Add a comment when you sign up. The comment can be a brief self-introduction and reason for signing up, including what you do, what you're good at, what you're interested in and your technology stack, and how familiar you are with StarRocks. Example:

I pick! My name is Johnny. I am from StarRocks, and I have been working on database kernel development for one year. I am interested in OLAP, good at C++ programming, and have been using StarRocks for six months. I chose to solve this issue because I am familiar with the execution engine.

Step 2. On your WeChat, add the StarRocks community assistant "StarRocks-1" as a WeChat friend and leave your GitHub ID. The assistant will invite you to a WeChat group based on your picked issue.
Step 3. The mentor will communicate with all the participants who picked this issue and assign the task to the best match.
Before you start

Read the StarRocks GitHub workflow carefully, which provides the configurations of BE and FE and describes how to submit a pull request.
To learn more about StarRocks, read StarRocks documentation.

Pull request requirements

After you complete code development, upload the code to the StarRocks code repo as a pull request (PR).

PR title format: [SR GeekCamp] + PR title
Labels ：[Geek Camp 2022/feature]

After you submit your PR, comment on the status of your PR submission as requested under the Pinned Issue [StarRocks geek Camp 2022] page.

Comment format: "I have finished Issue #" + PR link id

@wuqiao Hello, I would like to complete this current pr

I pick! My name is Monster. I'm from ziroom, and I've been working as a data developer for a year now. I am interested in OLAP, good at Java programming and have not used StarRocks. I chose to solve this problem because I am familiar with bitmap as well as RBM.

My name is WangMan. I'd like to pick this issue. I started using it very early StarRocks. From version 1.14 to 2.1.  I chose to solve this problem because I am familiar with using starrocks and  I am big data development engineer from beijing  and good at Java programming and and a little knowledge of C + + programming.I hope I can contribute a little to the community. Complete the task

My name is xu cheng，I am instrested in olap


fix(upstream): Syncing with upstream ipfs-service-provider

:tada: This PR is included in version 1.4.2 :tada:
Your semantic-release bot :package::rocket:


 This PR adds built-in support for using the ECR-based images.
Issues Resolved
n/a
Check List

[X] New functionality includes testing.

[X] All tests pass

[X] New functionality has been documented.

[X] New functionality has javadoc added

[X] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@ginkel thanks for pull request @ginkel , please check the official recommendations for image name substitution from Testcontainers: https://www.testcontainers.org/features/image_name_substitution/ (also documented  https://github.com/testcontainers/testcontainers-java/issues/4219 and https://github.com/testcontainers/testcontainers-java/pull/3102)

@reta Thanks for your speedy reply! If you consider a built-in allow-listing of the ECR images inappropriate, feel free to disregard my PR. The reason for creating it was that I was actually surprised having to manually declare the ECR-hosted official image as compatible. That's easy to do, but was somewhat unexpected...

Thanks @ginkel , I think the issue is, broadly, to support any Docker repository in a way that is aligned with Testcontainers. I believe we should not take the path of ad-hoc ECR / GCR / ... support.


windows exe error

In Ton Stratum Miner windows x64, when i choose any windows item Cuda or OpenCl, this error shown:
Error: spawn C:\Users\USERNAME\AppData\Local\Temp\22e1fAeKI1ij3EDjQbG20CO23cd\resources\bin\cuda.exe ENOENT
what is the problem? and what to do?

I ran it as administrator and problem solved.


Prevent password manager from detecting fields

User Story
As an applicant
I want my password manager to not detect fields while I'm completing the application
so that the pop-up doesn't appear while I'm entering my responses.
Description
1Password recognizes many of the input fields as passwords, even though they aren't. This interrupts users' flow through the form and can get in the way of filling it out, especially when the password manager box overlaps with the date picker.
Doesn't occur with Safari & Apple's password manager. Doesn't occur with SAASPASS & Firefox. Doesn't occur with Norton & Chrome, which TELUS may use.
Currently occurs for Project title, Geographic location, Project start date, Completion date, Current employment, Total person months of employment to be created (x2), Applicant business number, all fields on the Organization location page, Website.
Acceptance Criteria

[ ] Scenario: auto-complete off for project pages
Given that an applicant is on the project pages (form pages 1-14) of an application
when they select an input field
then a browser or password manager will not suggest anything for the text fields.

[ ] Scenario: auto-complete on for organization pages
Given that an applicant is on the project pages (form pages 15-20) of an application
when they select an input field
then a browser or password manager may suggest something for the text fields.

Likely a 1Password or Mac issue.

I assume the title here needs an update (and probably the description?), from what I remember with the meeting we're more concerned with the autofill rather than the 1password issue right?

Let's change it! We have the power!


Unable to create resource using Inline source

What happened?

Trying to import terraform module to crossplane.
Package installed : crossplane/provider-terraform:master
ProviderConfig looks like below:
apiVersion: tf.crossplane.io/v1alpha1
kind: ProviderConfig
metadata:
  name: aws
spec:
  credentials:
    - secretRef:
        namespace: crossplane-system
        name: aws-creds
        key: creds
      source: Secret
      filename: aws-creds.json

AWS providerconfig is as below:
apiVersion: aws.crossplane.io/v1beta1
kind: ProviderConfig
metadata:
  name: default
spec:
  credentials:
    source: Secret
    secretRef:
      namespace: crossplane-system
      name: aws-creds
      key: creds

Resource.yaml looks like this.
apiVersion: tf.crossplane.io/v1alpha1
kind: Workspace
metadata:
  name: s3bucket
  annotations:
    crossplane.io/external-name: tfbucket
spec:
  forProvider:
    source: Inline
    module: |
      resource "aws_s3_bucket" "b" {
        bucket = "my-tf-test-bucket"
        tags = {
          Name        = "crossplane"
          Environment = "Demo"
        }
      }

      resource "aws_s3_bucket_acl" "example" {
        bucket = aws_s3_bucket.b.id
        acl    = "private"
      }

      output "url" {
              value = aws_s3_bucket.b.id
      }
  writeConnectionSecretToRef:
    namespace: crossplane-system
    name: tfinline
  providerConfigRef:
    name: aws

When I do a kubectl apply : I am getting the below error
Message:               observe failed: **cannot list Terraform resources: no state file was found!**
Reason:                ReconcileError
Status:                False
Type:                  Synced

Events:
Type     Reason                         Age                   From                                Message

Warning  CannotObserveExternalResource  35s (x10 over 5m11s)  managed/workspace.tf.crossplane.io  cannot list Terraform resources: no state file was found!
Can you please tell , if there are any errors ?
I am able to create resources through crossplane native providers. But not through this, it would really great if you can help on this.
How can we reproduce it?

What environment did it happen in?
Crossplane version: crossplane/crossplane:v1.6.4

@sharigitpersonal I believe your Terraform module needs a terraform block to configure the backend:
terraform {
        backend "kubernetes" {
          secret_suffix     = "xp"
          namespace         = "default"
          in_cluster_config = true
        }
      }

and you may need to configure the Terraform aws provider region:
provider "aws" {
              region = "%s"
            }

Hope this helps

@bobh66  , Thanks for your help. I tried it .
The new yaml looks like this
apiVersion: tf.crossplane.io/v1alpha1
kind: Workspace
metadata:
  name: s3bucketnew
spec:
  forProvider:
    source: Inline
    module: |
      resource "aws_s3_bucket" "b" {
        bucket = "testcrossplane"
        tags = {
          Name        = "crossplane"
          Environment = "Demo"
        }
      }

      resource "aws_s3_bucket_acl" "example" {
        bucket = aws_s3_bucket.b.id
        acl    = "private"
      }

      output "url" {
              value = aws_s3_bucket.b.id
      }

      provider "default" {
              region = "ca-central-1"
                    version = "4.8.0"
              }

      terraform {
            backend "kubernetes" {
                secret_suffix = "xp"
                namespace      = "crossplane-system"
                in_cluster_config = "true"
           }
      }
  providerConfigRef:
    name: default

the providerconfig yaml looks like this
apiVersion: tf.crossplane.io/v1alpha1
kind: ProviderConfig
metadata:
  name: default
spec:
  credentials:
    - secretRef:
        namespace: crossplane-system
        name: aws-creds
        key: creds
      source: Secret
      filename: aws-creds.conf

Error I am getting is
Status:
  At Provider:
  Conditions:
    Last Transition Time:  2022-03-28T12:24:07Z
    Message:               connect failed: cannot initialize Terraform configuration: failed to query available provider packages
    Reason:                ReconcileError
    Status:                False
    Type:                  Synced
Events:
  Type     Reason                   Age              From                                Message
  ----     ------                   ----             ----                                -------
  Warning  CannotConnectToProvider  1s (x2 over 6s)  managed/workspace.tf.crossplane.io  cannot initialize Terraform configuration: failed to query available provider packages

kubectl get pkg shows 👍
NAME                                                          INSTALLED   HEALTHY   PACKAGE                                                  AGE
configuration.pkg.crossplane.io/xp-getting-started-with-aws   True        True      registry.upbound.io/xp/getting-started-with-aws:v1.6.4   6d4h

NAME                                                       INSTALLED   HEALTHY   PACKAGE                                AGE
provider.pkg.crossplane.io/crossplane-provider-aws         True        True      crossplane/provider-aws:v0.25.0        6d4h
provider.pkg.crossplane.io/crossplane-provider-jet-aws     True        True      crossplane/provider-jet-aws:v0.4.2     3d19h
provider.pkg.crossplane.io/crossplane-provider-terraform   True        True      crossplane/provider-terraform:master   3d6h

Can you please shed some light on this ?

Hi @sharigitpersonal , thanks for trying out provider-terraform! Please try https://github.com/crossplane-contrib/provider-terraform/blob/master/examples/providerconfig-aws.yaml example

@ytsarev  , thanks a lot.  Your example gave some idea on it.
bucket.yaml
apiVersion: tf.crossplane.io/v1alpha1
kind: Workspace
metadata:
  name: finals3
spec:
  forProvider:
    source: Inline
    module: |
      resource "aws_s3_bucket" "b" {
        bucket = "finalcros"
        tags = {
          Name        = "crossplane"
          Environment = "Demo"
        }
      }

      resource "aws_s3_bucket_acl" "example" {
        bucket = aws_s3_bucket.b.id
        acl    = "private"
      }

      output "url" {
              value = aws_s3_bucket.b.id
      }

  providerConfigRef:
    name: aws-ref

providerconfig.yaml
apiVersion: tf.crossplane.io/v1alpha1
kind: ProviderConfig
metadata:
  name: aws-ref
spec:
  credentials:
    - secretRef:
        namespace: crossplane-system
        name: aws-creds
        key: creds
      source: Secret
      filename: awscred
  configuration: |
      terraform {
        backend "kubernetes" {
          secret_suffix     = "test"
          namespace         = "crossplane-system"
          in_cluster_config = true
        }
      }
      provider "aws" {
        shared_credentials_file = "awscred"
        region = "ca-central-1"
      }

Bucket was created in AWS. I will do few more remote execution to see how this works profoundly . Appreciate your help.
@bobh66 @ytsarev


Strobealign BAM --min-cov

For some arcane reason, when using BAM files output by Strobealign v0.7.1 the --min-cov filter removes all alignments (noted in the MetaGP remap loop process). Need to investigate the SAM/BAM query coverage calculation and compare with PAF in Strobealign.

This was due to SAMBAM CIGAR bugs in strobealign 0.7.0 - should now be fixed in 0.8.0


Bug: 'sdk current' does not list current candidates

Bug report

Issuing the command 'sdk current' does not list any candidates though they are installed and usable.
This is not a huge deal but would be more convenient if it worked like the web documentation states.
To reproduce

Basic usage... run the command and observe the output.  screenshot follows:
p.s.  This is "immediately" after installation of SDKMAN and 'sdk install' of java and maven.

System info

Windows 10, Cygwin
GNU bash, version 5.2.12(1)-release (x86_64-pc-msys)
SDKMAN 5.17.0

What happens if you open a new terminal and run the same command?

@marc0der Same:

p.s.  and probably un-related but... the 'sdk current' command takes quite a while to execute.  Is this known/expected?  As I only have a few SDKs installed, the execution time surprises my assumptions.  This is also more curiosity than anything else.  I enjoy the work in this product so please don't think I'm only complaining about it :)

Update:  I need to amend my original post.  The issue is occuring on Git Bash/MinGW.  The following screenshot is taken from another computer running Cygwin and it works as documented (so must be something local but I don't know where to start):

Good to know, thanks for the update. Also worth noting that this doesn't happen on Linux or macOS.

I just did the research and it seems that the git bash does not allowed symbolics links by default (on windows)
So in order to get that working refer to https://github.com/orgs/community/discussions/23591#discussioncomment-3241019


web scenes don't seem to import correctly during ops server install

web scene seems to be corrupt after installing ops server.  it needs to be manually recreated; i can't find anyway to edit the layers that make up the scene.

Assigned to @elinz. If you want me to look into this, let me know.

@conklinbd If you can take a look at this that would be great. I'm looking into another publishing issue right now.

@kerryrobinson Is the web scene in question the one listed in the issue #1322 ?

Yes I believe so.
Sent from my Samsung Galaxy smartphone.
-------- Original message --------
From: Eric Linz<EMAIL_ADDRESS>Date: 15/10/2015 17:46 (GMT+01:00)
To: Esri/ops-server-config<EMAIL_ADDRESS>Cc: Kerry Robinson<EMAIL_ADDRESS>Subject: Re: [ops-server-config] web scenes don't seem to import correctly during ops server install (#1309)
@kerryrobinsonhttps://github.com/kerryrobinson Is the web scene in question the one listed in the issue #1322https://github.com/Esri/ops-server-config/issues/1322 ?
—
Reply to this email directly or view it on GitHubhttps://github.com/Esri/ops-server-config/issues/1309#issuecomment-148428621.

This was resolved in #1322 so I am closing this issue.


Export XLog does not contain the specified extension

Still open
It was closed in #15 but importing/exporting the Receipt log some extensions are still missing

Hi, Ale,
due to the extension is not standard. When you import the event log with OpenNaive strategy, it could read the extensions. Please choose the one called Lenient to read the event log with extension.
regards
Kefang


bpf: add tail_call_{static,dynamic} helpers

See commit msg.

test-me-please

test-me-please

test-me-please

test-me-please

Coverage remained the same at 37.834% when pulling 543172e69f2caad91c3edcb16a4fec0f40273cf5 on pr/tail-call into e6371991b7eb9cdf9c06f36d8e1492fbc23536b1 on master.

retest-4.19

retest-4.9

retest-runtime

Travis CI twice in a row blocked by Github:
$ ./.travis/prepare.sh
https://github.com/llvm/llvm-project/releases/download/llvmorg-10.0.0/clang+llvm-10.0.0-x86_64-linux-gnu-ubuntu-18.04.tar.xz:
2020-05-12 09:08:55 ERROR 429: too many requests. [...]

retest-4.9

retest-runtime

K8s-1.11-Kernel-netnext + K8s-1.17-Kernel-4.19 green.

test-me-please

runtime-4.9 currently broken in master (https://jenkins.cilium.io/job/Ginkgo-CI-Tests-runtime-Pipeline/), same (unrelated) issue.

retest-runtime

retest-4.19


Challenge-custom services

Challenges should have a way to declare a new service (e.g., pwn.college/workspace/vulnwebsite).

I'm assuming this connects to #568, as in declared services will show up in the corresponding dropdowns

Yep! Maybe I should have made it one issue...


Inconsistent synchronization of some variables in SpdySessionHandler

Field io.netty.handler.codec.spdy.SpdySessionHandler.lastGoodStreamId
Synchronized 50% of the time:
Unsynchronized access at SpdySessionHandler.java:[line 148]
Unsynchronized access at SpdySessionHandler.java:[line 243]
Synchronized access at SpdySessionHandler.java:[line 736]
Synchronized access at SpdySessionHandler.java:[line 851]

Field io.netty.handler.codec.spdy.SpdySessionHandler.receivedGoAwayFrame
Synchronized 50% of the time
Unsynchronized access at SpdySessionHandler.java:[line 361]
Synchronized access at SpdySessionHandler.java:[line 723]

Field io.netty.handler.codec.spdy.SpdySessionHandler.localConcurrentStreams
Synchronized 50% of the time
Unsynchronized access at SpdySessionHandler.java:[line 598]
Synchronized access at SpdySessionHandler.java:[line 728]

Field io.netty.handler.codec.spdy.SpdySessionHandler.sentGoAwayFrame
Synchronized 75% of the time
Unsynchronized access at SpdySessionHandler.java:[line 150]
Synchronized access at SpdySessionHandler.java:[line 723]
Synchronized access at SpdySessionHandler.java:[line 849]
Synchronized access at SpdySessionHandler.java:[line 850]

Field io.netty.handler.codec.spdy.SpdySessionHandler.remoteConcurrentStreams
Synchronized 50% of the time
Unsynchronized access at SpdySessionHandler.java:[line 318]
Synchronized access at SpdySessionHandler.java:[line 728]

Message from FindBugs:
The fields of this class appear to be accessed inconsistently with respect to synchronization.  This bug report indicates that the bug pattern detector judged that
The class contains a mix of locked and unlocked accesses,
The class is not annotated as javax.annotation.concurrent.NotThreadSafe,
At least one locked access was performed by one of the class's own methods, and
The number of unsynchronized field accesses (reads and writes) was no more than one third of all accesses, with writes being weighed twice as high as reads
A typical bug matching this bug pattern is forgetting to synchronize one of the methods in a class that is intended to be thread-safe.
You can select the nodes labeled "Unsynchronized access" to show the code locations where the detector believed that a field was accessed without synchronization.
Note that there are various sources of inaccuracy in this detector; for example, the detector cannot statically detect all situations in which a lock is held.  Also, even when the detector is accurate in distinguishing locked vs. unlocked accesses, the code in question may still be correct.
@trustin @normanmaurer @fredericBregier @jpinner
Please, check it out.

Let me fix this


SVG loading breaks vue production build

Hello :) It's a little hard to debug and understand, so there's still a strong chance that this isn't something to fix in ripple, but possibly something to add to docs for ripple users.

version
issue

1.19.8
investigated

something in November 2020
raised but not investigated

We recently inherited a vue app, with an outstanding CI issue, that vue-cli-service build --mode production yields a build error (included at the bottom, to reduce noise).
Investigation has been difficult, due to webpack chains being a bit magical. But there are a few things we've found that might be relevant:

There is a PR (#774) that removes svg loading rules from the vue example app.
Removing svgRule.uses.clear(); from our vue.config.json before the svg-sprite-loader rules removes the error (but vic-logo.svg is broken).
The build error surfaces in ripple-pagination, but separator.svg remains unaffected if the svg rule is cleared.
All the answers I can find on the internet talk about server-side rendering issues, which makes sense in context of the error, but feels like a red herring in context of SVGs.

So, hopefully y'all have seen something like this before, and the fix is easy and obvious and we just didn't realise :pray:
But, worst case, something is actually broken, and people haven't noticed (like no one's building vue apps atm, or they haven't noticed the footer is missing an image, or they're deploying staging builds to prod, or who knows).

Build error:
 error  in ./node_modules/@dpc-sdp/ripple-pagination/PrevNext.vue?vue&type=style&index=0&lang=scss&

Syntax Error: ReferenceError: document is not defined

 @ ./node_modules/@dpc-sdp/ripple-pagination/PrevNext.vue?vue&type=style&index=0&lang=scss& 1:0-475 1:0-475
<EMAIL_ADDRESS><EMAIL_ADDRESS> @ ./node_modules/cache-loader/dist/cjs.js??ref--14-0!./node_modules/thread-loader/dist/cjs.js!./node_modules/babel-loader/lib!./node_modules/ts-loader??ref--14-3!./node_modules/cache-loader/dist/cjs.js??ref--0-0!./node_modules/vue-loader/lib??vue-loader-options!./src/components/search/SearchResult.vue?vue&type=script&lang=ts&
 @ ./src/components/search/SearchResult.vue?vue&type=script&lang=ts&
 @ ./src/components/search/SearchResult.vue
 @ ./node_modules/cache-loader/dist/cjs.js??ref--14-0!./node_modules/thread-loader/dist/cjs.js!./node_modules/babel-loader/lib!./node_modules/ts-loader??ref--14-3!./node_modules/cache-loader/dist/cjs.js??ref--0-0!./node_modules/vue-loader/lib??vue-loader-options!./src/views/scheme/SearchScheme.vue?vue&type=script&lang=ts&
 @ ./src/views/scheme/SearchScheme.vue?vue&type=script&lang=ts&
 @ ./src/views/scheme/SearchScheme.vue
 @ ./src/router/index.ts
 @ ./src/main.ts
 @ multi ./src/main.ts

Thanks @abubics for raise the issue here!
For SVG icons, yes, we did a change on how to add custom icons. However, it's not a breaking change, your app can work without migrating to the new implementation. Do you know if your project has custom Ripple icons? If not, you can just remove old implementation like here https://github.com/dpc-sdp/ripple/pull/774/files#diff-e98f43f1a2c175915ef9aea0f011c5085b58c9fa241d6d20d19a9ef28b00c7eeL1
If you do, you can still remove as above and migrate to the new way to register custom icon https://github.com/dpc-sdp/ripple/blob/master/packages/components/Atoms/Icon/README.md#registering-custom-icons
Examples: https://github.com/dpc-sdp/ripple/tree/master/examples/basic-examples/assets/ripple-icon
For the error:
 error  in ./node_modules/@dpc-sdp/ripple-pagination/PrevNext.vue?vue&type=style&index=0&lang=scss&

Syntax Error: ReferenceError: document is not defined

I can't reproduce it in my example vue cli app when I add PrevNext component then run vue-cli-service build --mode production. Also I couldn't see  document is used in PrevNext.vue so I am not able to debug it. The error self not make much sense to me.
I will suggest try migrate the svg icon then see if the issue fixed.
It also will be helpful if you can create a repo which can reproduce the issue. Cheers!

Thanks for the quick response, @tim-yao!
I can see a couple of custom <rpl-icon ...> tags in the codebase, I'll check them out, and try the new custom registration. If that doesn't get us anywhere, I'll see if we have time to make a repro repo . . . we have some delivery pressure atm.
I did look through the pagination package, and anything else that seemed to be implicated, and (just like you) also couldn't find any document references, so I assumed it's something weird in the webpack setup.
And also, since it was raised months ago (before we inherited it), I guessed it's also unlikely to be related to that PR's changes. Thanks for confirming :)

Thanks @abubics It sounds the Webpack build process called some client only code. document is a browser only thing. I think your app is a client side app not using server side rendering right? I will leave you to investigate and find more information. Thanks!

Yeah, exactly. We'll jump back in this thread once we figure more out :)

We figured this out. We were customising the svgRule in vue.config.js. We need to use url-loader so that https://github.com/dpc-sdp/ripple/blob/dd1afe52b9ecdab8d0d5310eba8984aee5d0b45d/packages/components/Organisms/SiteFooter/SiteFooter.vue#L28 works. The right way to do that seems to be:
    config.module.rule('svg').exclude
      .add(resolve(__dirname, path.dirname(require.resolve('@dpc-sdp/ripple-global/package.json'))));

    config.module
      .rule('svg-url')
      .test(/\.svg$/i)
      .use('url-loader').loader('url-loader')
      .end();

Thanks @gazal-k for updating what you got! That make sense if you have your own loader. I will close this issue now as it's resolved in your side.


fix an error log about bgp_next_hop filter

Description
The bgp_next_hop isn't defined in some route rules, and the following
is log of the bird:
bird: filters, line 18: ~ applied on unknown type pair

So, before we use bgp_next_hop, we should check if the bgp_next_hop is defined or not:
filter calico_ipip {

  if ( net ~ <IP_ADDRESS>/18 ) then {
    if defined(bgp_next_hop) && (bgp_next_hop ~ <IP_ADDRESS>/24 ) then {
      krt_tunnel = "";
    } else {
      krt_tunnel = "tunl0";
    }
    accept;
  }

  accept;
}

Todos

[x] Tests
[x] Documentation
[ ] Release note

Release Note

None required

@nighmeng:  Do you know which rules that the bgp_next_hop will not be set on?  In these cases should we fall back to using the gw or next hop addr?

@robbrockbank we don't need to fall back, because the bgp_next_hop isn't defined only when the gw is <IP_ADDRESS>.
For example, the following is the whole route table of my server:
# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
<IP_ADDRESS>         <IP_ADDRESS>   <IP_ADDRESS>         UG    0      0        0 bond0
<IP_ADDRESS>   <IP_ADDRESS>  <IP_ADDRESS> UG    0      0        0 tunl0
<IP_ADDRESS>    <IP_ADDRESS> <IP_ADDRESS> UG    0      0        0 tunl0
<IP_ADDRESS>   <IP_ADDRESS>  <IP_ADDRESS> UG    0      0        0 tunl0
<IP_ADDRESS>    <IP_ADDRESS>  <IP_ADDRESS> UG    0      0        0 tunl0
<IP_ADDRESS>   <IP_ADDRESS>  <IP_ADDRESS> UG    0      0        0 tunl0
<IP_ADDRESS>  <IP_ADDRESS>  <IP_ADDRESS> UG    0      0        0 tunl0
<IP_ADDRESS>  <IP_ADDRESS>         <IP_ADDRESS> U     0      0        0 *
<IP_ADDRESS>  <IP_ADDRESS>         <IP_ADDRESS> UH    0      0        0 cali7c3eedf162e
<IP_ADDRESS>  <IP_ADDRESS>  <IP_ADDRESS> UG    0      0        0 tunl0
<IP_ADDRESS>  <IP_ADDRESS>  <IP_ADDRESS> UG    0      0        0 tunl0
<IP_ADDRESS>     <IP_ADDRESS>         <IP_ADDRESS>     U     1006   0        0 bond0
<IP_ADDRESS>      <IP_ADDRESS>         <IP_ADDRESS>     U     0      0        0 docker0
<IP_ADDRESS>   <IP_ADDRESS>         <IP_ADDRESS>   U     0      0        0 bond0

Only when the calico_ipip receives the following routes, it print the bgp_next_hop isn't defined.
<IP_ADDRESS>  <IP_ADDRESS>         <IP_ADDRESS> U     0      0        0 *
<IP_ADDRESS>  <IP_ADDRESS>         <IP_ADDRESS> UH    0      0        0 cali7c3eedf162e

So we should check if the bgp_next_hop is defined or not before we use it.

I've tested this with a rebase and it passed, so merging despite the semaphore failure.

Thanks @nightmeng - this is now merged into the latest master.


Add Loggit

Full disclosure: I'm the author of Loggit.
Loggit is end-to-end encrypted and open source (AGPL-3.0). Additionally, 3.0 is self-hosted. You can read more about that announcement here.
The website also has no analytics or tracking (only page views are counted when loaded).

Maybe it would have a better fit under the "Utilities" category?

Thank you so much for making this app! It seems very cool and great, capability of self-hosting is awesome!
Wish the website was more clear about where/how to install/use it. The first thing you see is that you can pay for it, but not that is open-source and that you can self-host it. Someone like me would just click out of it instead of sticking-in and thinking of donating to the project.

@pluja I'm happy to move it around if you want.
@Splarkszter I'm planning to redo the homepage since it's been like that for a while, and it's had a few new features added since. I'll take that into consideration!

@BrunoBernardino please move it under utilitites and I'll merge!

Done @pluja ! Also @Splarkszter hopefully the new landing page makes it easier to see there's a self-hosting option!

Also @Splarkszter hopefully the new landing page makes it easier to see there's a self-hosting option!

Very well! Thank you so much! The 'Watch Demo' box and the text below are missaligned with the left side but is not an big issue. haha
Thank you for all your efforts everyone!

The 'Watch Demo' box and the text below are missaligned with the left side but is not an big issue.

Can you show me a screenshot and tell me what's the browser/OS you're using? I see it well aligned on Firefox and Chrome, Linux.

Can you show me a screenshot and tell me what's the browser/OS you're using? I see it well aligned on Firefox and Chrome, Linux.

Linux Mint - Librewolf [Tho it is on all browsers]

Probably i'm just too picky don't worry.

Interesting! We're using the same (I'm on Manjaro now, but was on Mint for a long time). The difference seems to be in the installed fonts. I might need to rethink my strategy around them, but I tried to pick fonts that look and feel very similarly, that are generally available across devices, but it seems Ubuntu (I think that's the one triggering for you) uses a bit more. I just tweaked it so it should look fine now, hopefully.


Cannot setup triggers for Monitor

I’ve created a monitor based on Extraction Query. I am trying to create trigger for that, when I press the create button, it doesn’t do anything. I mean it neither creates that trigger nor shows any error message. How to resolve this issue?
Please help!!

Hi @rajansapra007 can you provide more details about your monitor and trigger setup. Please use the community forums at https://discuss.opendistrocommunity.dev/ for questions so that the larger community can share their know-how too.


add cronjobs to list of required permissions

Hey, I just made a Pull Request!
Adds reference to cronjobs access neccessary for recent backend change.
:heavy_check_mark: Checklist

[ ] A changeset describing the change and affected packages. (more info)
[x] Added or updated documentation
[ ] Tests for new functionality and regression tests for bug fixes
[ ] Screenshots attached (for UI changes)
[x] All your commits have a Signed-off-by line in the message. (more info)

@mclarke47 for visibility.

@benjdlambert Can you hold off merging this. I think I got it wrong. I am testing now, shouldnt be too long.

Need a DCO sign off here too @punkle 👍

@punkle is this ready to go after the DCO Signoff?

@benjdlambert I think its correct, but I havent quite verified it yet. However I think this pull request will bring it closer. We can merge this and Ill follow up with a doc improvement if I am wrong.

Hey @benjdlambert and @mclarke47 I can confirm that adding the batch jobs and cronjobs as follows removes the following error from the kube plugin:
Errors:
Cluster: dev0
Error fetching Kubernetes resource: '/apis/batch/v1/jobs', error: UNKNOWN_ERROR, status code: 403
Error fetching Kubernetes resource: '/apis/batch/v1/cronjobs', error: UNKNOWN_ERROR, status code: 403

I do not have permission to merge this change.


[iOS Verifier App]Nullable Values in Verifieable Credentials Lead To Unscannable Code

Describe the bug
The following valid code with null-values in optional fields does not get scanned by the verifier app. Instead a pop-up window "Bar-Code Error" appears (see attached photo).
Attention: it is a valid code but with null values.
Expected behaviour
The app should be able to scan the code.
Technical Details:
Verifier App <IP_ADDRESS>(33) Hash & iOS Verifier 1.3.1 (31) Bloom

The same code scanned bei Android-Verifier bring an another message:
"X Invalid
Time before issueance date"
Please: what ever, but the code is readable for the app and Android and IOS should brimg the same message but not different messages as in this case :-).

Retest failed: Issue persists iOS Verifier 1.3.1 (36)

tvt, 11.05.2022:
The same code scanned bei Android-Verifier and  bring an another message, but
at the end the JWT and not the verifiable Credential is noticed by the app and an error-message is displayed.


api: set proper defaults for our share booleans

Use the "annotations" provided by kubebuilder to specify how to treat the boolean values when they're not explicitly set.

This isn't as standalone as I thought. This needs to make use of apiextensions.k8s.io/v1 and because of that probably needs to be rebased on the patches in PR #31.

This is now based on PR #31 . This was done because setting the controller-gen to use apiextensions.k8s.io/v1 would side effect all the crd files, including the ones were in the process of deleting and create a bunch of churn and merge conflicts.
Please review PR #31 first. Sorry for the noise. :-)


setDimensions sets height based on first slide

With slides of varying height, the setDimensions should go through all the slide heights. Currently it's just getting firstSlide.offsetHeight. This hides the slides that are taller then the first one as overflow. setDimensions function on Line 639

I'd like to fix this since it is affecting us. Is there a preferred solution?
Some alternatives that come to mind are:

Setting the height to the max height of all children.
Let the browser compute the height automatically. (Seems to require removing position: absolute on slides.)
Allow the user to override the slideHeight. (Supporting 'auto' requires same as above.)

I took a quick stab at 3, but removing position:absolute results in some weirdness... See https://github.com/aij/nuka-carousel/commit/3ea4c7830b9e5023140585fe06604de22783a713
I'd prefer to avoid spending too much time going down that road if an entirely different approach would be preferable.

Ok if you want to do it, provide a prop called heightMode. Make the valid options adaptive & max and default to max.
max takes the tallest slide, and checks and sets on every change. adaptive sets the height to the height of the tallest slide.

So, to be clear, both max and adaptive use the height of the tallest slide, but max checks on every change whereas adaptive checks less frequently? How much less frequently?

Hmm, actually, I'm guessing what you mean is that both will check on every call to setDimensions, but max will only grow the slideHeight whereas adaptive will also shrink it.

Indeed

Will this change be published on npm anytime soon?


Update Markdown to include Scripture refs and TOC

@lwalen, updated the Markdown to include Scripture references and a superficial but not adequately working table of contents, as it contains no in-page links.  However, we have a problem: Did you do what I think is called a rebase merge when you added my prior content? Because whenever I submit a PR now, it wants me to re-merge the same previous commits all over again.  I don't know what to do about this...

Hey @lwalen, I have updated the readme but had to remove the source files and output as a strategy for working around your potential re-base merge, or whatever it was you initially used when committing my content. Can you accept this PR and then I'll add the files back in and re-submit with another PR? Thanks.
For reference, I have added the ability to reference every paragraph of every chapter with the \label command using the book doc class and the preface and appendix have been added with signatories/subscribers.

This is how it should look according to my own version (without the preliminary personal note):

Thanks! Could you make the markdown file a new file called something like lbcf_with_references.md? I think it would be nice to have both versions available.
Looks like there are conflicts in the readme, too.
I appreciate the additions!

Thanks! Could you make the markdown file a new file called something like lbcf_with_references.md? I think it would be nice to have both versions available.

Not quite sure what you mean above here, as I regrettably have not saved your original Markdown source, though I could potentially go back through the history if you like. I thought I remembered your original having scripture refs but maybe I added those. Did you only have confessional statements?

Looks like there are conflicts in the readme, too.

Well, I've checked both readme files and I'm pretty sure mine has the same original content as yours but I've made a couple of additions. As previously stated, there's something about yours and my git workflows which are conflicting but we both don't seem to know why. If you manually accept all of my changes then you will still have everything you put in the readme yourself.
My concern here is mainly regarding line blaming. Out of curiosity, what are your default line-endings? Mine are set to commit Unix-style, checkout as-is.

I appreciate the additions!

No worries, I have enjoyed it.

Okay, so it seems I can fix the conflicts myself. With that done, I'll create the two MD files.

@lwalen, all done. It should be okay for you to merge now.


Give TestEvictAndLoadChunkDescs more time to actually evict

Obviously, it's really bad to depend on timing here. The proper fix
would be to have something like WaitForIndexing for other things to
wait for, too.
For now, let's see if the wait time increase fixes the issue.
@RichiH This should fix the last flaky test you reported in https://github.com/prometheus/prometheus/issues/1337 . Since It's not flaky in my environment, could you try if it helps for you?

@RichiH Ping?

:+1:


Line Feature not Working.

I am trying to build an OpenStreetMap Component, but i can't seem to use the Line feature.
Every other feature works.
i copy pasted the line code from your example and i won't get a line to show up. It seems like this isn't the intended behaviour.
Here is part of my razor file:
<OpenStreetMap @ref="_map" Style="height:480px" Class="card" @bind-Center="_center" ScaleLineUnit="ScaleLineUnit.Metric" Zoom="@_zoom" ZoomChanged="zoom" OnClick="OnMapClick" OnPointerMove="OnPointerMove">
    <Popup>

            @if (context is Marker marker)
            {
<EMAIL_ADDRESS><EMAIL_ADDRESS>/ @marker.Coordinate.Y

            }

            else if (context is Line line)
            {
<EMAIL_ADDRESS>
            }

    </Popup>
    <Features>
        <Marker Type="MarkerType.MarkerPin" Coordinate="new Coordinate(x, y)" Title="You are Here" Popup></Marker>
         <Circle Center="new Coordinate(x, y)" Radius="50" BackgroundColor="#2222AA66"></Circle>
         <Point Coordinate="new Coordinate(x, y)" BorderSize="4" BorderColor="red" BackgroundColor="cyan" Radius="3"></Point>
         <Point Coordinate="new Coordinate(x2, y2)" BorderSize="4" BorderColor="red" BackgroundColor="cyan" Radius="3"></Point>
         <Line @ref="testref" Points="@route" BorderColor="cyan" BorderSize="20" BackgroundColor="cyan" Title="Test"></Line>
     </Features>

</OpenStreetMap>

 @code {
    private Coordinate _center = new(x, y);
    private Coordinate[] route = { new Coordinate(x, y), new Coordinate(x2, y2) };
    # private Coordinate[] route = new []{ new Coordinate(x, y), new Coordinate(x2, y2) };
    private OpenStreetMap _map;

x,y,x2 and y2 are double values. On my Map everything appears except the line. I tried to have the coordinates as a reference or in the Line element but both yielded the same results,

I can confirm. OpenLayers API cannot transform arrays of coordinates with internal methods correctly. (Thought that this used to work). I need to implement this in interop js. Fix is on the way...


legendItems.enter(...).call is not a function

Hello,
I am running into some errors in getting line charts to work.  I am using angular1.5, I have checked the codepen with 1.5 and works fine.
But for some reason I get some errors :
app.js:137 TypeError: legendItems.enter(...).call is not a function
    at Legend.update (app.js:11735)
    at EventManager.t (app.js:10240)
    at EventManager.trigger (app.js:10888)
    at updateAll (app.js:12989)
    at m.$digest (app.js:162)
    at m.$apply (app.js:165)

I am copying the data and options from http://n3-charts.github.io/line-chart/#/examples#column_series
Any ideas?
Best regards
Joao Garin

Will close this issue. Working with version 3.5.15 of d3.
Was using version 3.1


Setup Projects

Description
At this time, there is not an easy way for the team to manage the current roadmap and engineer progress within the Momentum Design project. Using GitHub Projects, outline the roadmap as well as engineer flows needed to complete tasks within this project.


[ bug ]用的好好的，今天突然登录不了

重现步骤

之前用的都好好的，今天登录突然报错
错误信息：
服务器错误Traceback (most recent call last): File "/opt/venv4archery/lib/python3.8/site-packages/django/db/backends/base/base.py", line 219, in ensure_connection self.connect() File "/opt/venv4archery/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner return func(*args, **kwargs) File "/opt/venv4archery/lib/python3.8/site-packages/django/db/backends/base/base.py", line 200, in connect self.connection = self.get_new_connection(conn_params) File "/opt/venv4archery/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner return func(*args, **kwargs) File "/opt/venv4archery/lib/python3.8/site-packages/django/db/backends/mysql/base.py", line 234, in get_new_connection return Database.connect(**conn_params) File "/opt/venv4archery/lib/python3.8/site-packages/MySQLdb/init.py", line 130, in Connect return Connection(*args, **kwargs) File "/opt/venv4archery/lib/python3.8/site-packages/MySQLdb/connections.py", line 185, in init super().init(*args, **kwargs2) MySQLdb._exceptions.OperationalError: (1049, "Unknown database 'archery'") The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/opt/archery/common/auth.py", line 61, in authenticate user = Users.objects.get(username=username) File "/opt/venv4archery/lib/python3.8/site-packages/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/opt/venv4archery/lib/python3.8/site-packages/django/db/models/query.py", line 425, in get num = len(clone) File "/opt/venv4archery/lib/python3.8/site-packages/django/db/models/query.py", line 269, in len self._fetch_all() File "/opt/venv4archery/lib/python3.8/site-packages/django/db/models/query.py", line 1308, in _fetch_all self._result_cache = list(self._iterable_class(self)) File "/opt/venv4archery/lib/python3.8/site-packages/django/db/models/query.py", line 53, in iter results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size) File "/opt/venv4archery/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1154, in execute_sql cursor = self.connection.cursor() File "/opt/venv4archery/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner return func(*args, **kwargs) File "/opt/venv4archery/lib/python3.8/site-packages/django/db/backends/base/base.py", line 259, in cursor return self._cursor() File "/opt/venv4archery/lib/python3.8/site-packages/django/db/backends/base/base.py", line 235, in _cursor self.ensure_connection() File "/opt/venv4archery/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner return func(*args, **kwargs) File "/opt/venv4archery/lib/python3.8/site-packages/django/db/backends/base/base.py", line 219, in ensure_connection self.connect() File "/opt/venv4archery/lib/python3.8/site-packages/django/db/utils.py", line 90, in exit raise dj_exc_value.with_traceback(traceback) from exc_value File "/opt/venv4archery/lib/python3.8/site-packages/django/db/backends/base/base.py", line 219, in ensure_connection self.connect() File "/opt/venv4archery/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner return func(*args, **kwargs) File "/opt/venv4archery/lib/python3.8/site-packages/django/db/backends/base/base.py", line 200, in connect self.connection = self.get_new_connection(conn_params) File "/opt/venv4archery/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner return func(*args, **kwargs) File "/opt/venv4archery/lib/python3.8/site-packages/django/db/backends/mysql/base.py", line 234, in get_new_connection return Database.connect(**conn_params) File "/opt/venv4archery/lib/python3.8/site-packages/MySQLdb/init.py", line 130, in Connect return Connection(*args, **kwargs) File "/opt/venv4archery/lib/python3.8/site-packages/MySQLdb/connections.py", line 185, in init super().init(*args, **kwargs2) django.db.utils.OperationalError: (1049, "Unknown database 'archery'")

截图

docker logs archery -f --tail=10
File "/opt/venv4archery/lib/python3.8/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/opt/venv4archery/lib/python3.8/site-packages/django/db/backends/mysql/base.py", line 234, in get_new_connection
return Database.connect(**conn_params)
File "/opt/venv4archery/lib/python3.8/site-packages/MySQLdb/init.py", line 130, in Connect
return Connection(*args, **kwargs)
File "/opt/venv4archery/lib/python3.8/site-packages/MySQLdb/connections.py", line 185, in init
super().init(*args, **kwargs2)
django.db.utils.OperationalError: (1049, "Unknown database 'archery'")
版本信息
应用版本/分支：Release V1.8.1
部署方式：Docker

django.db.utils.OperationalError: (1049, "Unknown database 'archery'")
访问不到数据库了，看看本地的数据库吧。

django.db.utils.OperationalError: (1049, "Unknown database 'archery'") 访问不到数据库了，看看本地的数据库吧。

发现，docker部署的mysql的默认密码被别人利用，直接将库给删了。我现在要修改默认配置，发现一直没有用


Backend does not handle patterns in remove-trip modifications

The dev version of the UI allows the selection of patterns to remove (https://github.com/conveyal/analysis-ui/blob/da466b67075451883b3f6eabb3b6d3572ccda186/lib/utils/convert-to-r5-modification/remove-trips.js)
But the backend isn't ready to handle them (see https://github.com/conveyal/analysis-backend/blob/master/src/main/java/com/conveyal/taui/models/RemoveTrips.java)
This leads to errors on the back-end such as:
com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "patterns" (class com.conveyal.taui.models.RemoveTrips), not marked as ignorable (10 known properties: "showOnMap", "expanded", "routes", "id", "description", "variants", "scenario", "name", "feed", "trips"])
 at [Source: {"type":"remove-trips","id":"d8f9816f-a67a-42e0-a011-25eb606bb113","showOnMap":true,"expanded":false,"variants":[true,true,true,true],"scenario":"d399f7a1-7b65-4d31-a577-06b9811886da","description":null,"name":"Remove extra Newburyport/Rockport trips","feed":"MBTA","routes":["CR-Newburyport"],"trips":["CR-Weekday-Aug14th-17-NR-100"],"patterns":["03068262-5829-4873-b846-b0c0c78b32ca"]}; line: 1, column: 348] (through reference chain: com.conveyal.taui.models.RemoveTrips["patterns"])
        at com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:51) ~[analyst-master-bf00b18.jar:1.1]
        at com.fasterxml.jackson.databind.DeserializationContext.reportUnknownProperty(DeserializationContext.java:836) ~[analyst-master-bf00b18.jar:1.1]
        at com.fasterxml.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:1045) ~[analyst-master-bf00b18.jar:1.1]
        at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1352) ~[analyst-master-bf00b18.jar:1.1]
        at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownVanilla(BeanDeserializerBase.java:1330) ~[analyst-master-bf00b18.jar:1.1]
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:262) ~[analyst-master-bf00b18.jar:1.1]
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:159) ~[analyst-master-bf00b18.jar:1.1]
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:134) ~[analyst-master-bf00b18.jar:1.1]
        at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:122) ~[analyst-master-bf00b18.jar:1.1]
        at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:93) ~[analyst-master-bf00b18.jar:1.1]
        at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:131) ~[analyst-master-bf00b18.jar:1.1]
        at com.fasterxml.jackson.databind.deser.impl.TypeWrappedDeserializer.deserialize(TypeWrappedDeserializer.java:42) ~[analyst-master-bf00b18.jar:1.1]
        at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3702) ~[analyst-master-bf00b18.jar:1.1]
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2714) ~[analyst-master-bf00b18.jar:1.1]
        at com.conveyal.taui.controllers.ModificationController.createOrUpdate(ModificationController.java:36) ~[analyst-master-bf00b18.jar:1.1]

which are handled in the UI with the message "Error parsing modification JSON from client"

To reiterate my comment on a corresponding pull request, it looks like we have model objects in the backend that must exactly match the structure of the modification classes in R5. This is very strange to me, is this an intentional decision?

Not sure if it was intentional, but there are a few things that are "shared" that could be handled better. The Analysis Backend dependency on r5 is odd.

Conceptually, there are two things: the R5 computation implementation and the R5 API model classes. They happen to be bundled together in one project for convenience. Ideally analysis-backend should only depend on R5 to use those model classes - that's why I find this bizarre. Why would analysis-backend define its own versions of the main thing it needs to import from R5 (the data models it serializes to talk to R5)?


Memory Recovery Hackery

Just an experiment to run on staging. I hope we don't have to merge (a slightly better version of) this.

This was just a Plan B approach. Properly fixed in https://github.com/iasql/iasql/pull/1911


Some Emails arrive twice

I recently made a config change and see a weird problem since then. Cannot really tell what is going on and probably need some help on this.
I have 4 domains configured. Every domain has a main-account<EMAIL_ADDRESS>For each of those four accounts, I have a forward to a gmail address. Furthermore, there are some additional accounts for special cases which (I don't think) relate to this problem.
<EMAIL_ADDRESS>me<EMAIL_ADDRESS>@domain1.tld<EMAIL_ADDRESS>
<EMAIL_ADDRESS>me<EMAIL_ADDRESS>@domain2.tld<EMAIL_ADDRESS>
<EMAIL_ADDRESS>me<EMAIL_ADDRESS>@domain3.tld<EMAIL_ADDRESS>
<EMAIL_ADDRESS>additional1<EMAIL_ADDRESS><EMAIL_ADDRESS>additional2<EMAIL_ADDRESS>
<EMAIL_ADDRESS>me<EMAIL_ADDRESS>@domain4.tld<EMAIL_ADDRESS>
Because gmail is blocking some forwarded emails (ARC), I wanted to additionally store all emails in the mailserver's mailboxes as well and changed my config to:
<EMAIL_ADDRESS>me<EMAIL_ADDRESS>@domain1.tld me<EMAIL_ADDRESS>
<EMAIL_ADDRESS>me<EMAIL_ADDRESS>@domain2.tld me<EMAIL_ADDRESS>
<EMAIL_ADDRESS>me<EMAIL_ADDRESS>@domain3.tld me<EMAIL_ADDRESS>
<EMAIL_ADDRESS>additional1<EMAIL_ADDRESS><EMAIL_ADDRESS>additional2<EMAIL_ADDRESS>
<EMAIL_ADDRESS>me<EMAIL_ADDRESS>@domain4.tld me<EMAIL_ADDRESS>
Everything seems to work as expected. Except that some emails seem to be duplicated and arrive twice in gmail. It is only mails being sent from different server systems to my domain1.tld. This domain is purely used for machine-communication (cron mails, backup notifications, etc). All this emails are being duplicated. If I use the same account and manually send me an email, I only receive it once. Also my normal emails to the other three domains arrive as they should.
 Any idea where to start looking? Or is my setup screwed? I am a little bit confused about the name overloading between accounts and aliases in the configuration. Maybe something like this would be better?
<EMAIL_ADDRESS>me<EMAIL_ADDRESS>@domain1.tld<EMAIL_ADDRESS>
<EMAIL_ADDRESS>me<EMAIL_ADDRESS>@domain2.tld<EMAIL_ADDRESS>
<EMAIL_ADDRESS>me<EMAIL_ADDRESS>@domain3.tld<EMAIL_ADDRESS>
<EMAIL_ADDRESS>additional1<EMAIL_ADDRESS><EMAIL_ADDRESS>additional2<EMAIL_ADDRESS>
<EMAIL_ADDRESS>me<EMAIL_ADDRESS>@domain4.tld<EMAIL_ADDRESS>
Or will that only forward the<EMAIL_ADDRESS>emails to Gmail?

Found this: https://github.com/tomav/docker-mailserver/issues/683
Seems related, but still cannot find a solution. It seems that google sees the duplication (sometimes) and delivers only one of the two mails to the mailbox.
The maillog shows clearly that mails are sent out to google twice.
Is there a way to configure catch-all with a local mailbox + a forward for ALL adresses on the domain?

Duplicate of #683


MAT.jl can't find 'MATLAB_class'

I get the following error when I use matread from the MAT.jl package.
HDF5-DIAG: Error detected in HDF5 (1.10.4) thread 0:
  #000: H5A.c line 425 in H5Aopen(): unable to load attribute info from object header for attribute: 'MATLAB_class'
    major: Attribute
    minor: Can't open object
  #001: H5Aint.c line 478 in H5A__open(): unable to load attribute info from object header for attribute: 'MATLAB_class'
    major: Attribute
    minor: Can't open object
  #002: H5Oattribute.c line 515 in H5O__attr_open_by_name(): can't locate attribute: 'MATLAB_class'
    major: Attribute
    minor: Object not found
ERROR: Error opening attribute /#subsystem#/MATLAB_class
Stacktrace:
 [1] error(::String, ::String, ::String, ::String) at ./error.jl:42
 [2] h5a_open at /Users/Zach/.julia/packages/HDF5/Zh9on/src/HDF5.jl:2279 [inlined]
 [3] a_open(::HDF5.HDF5Group, ::String) at /Users/Zach/.julia/packages/HDF5/Zh9on/src/HDF5.jl:852
 [4] a_read(::HDF5.HDF5Group, ::String) at /Users/Zach/.julia/packages/HDF5/Zh9on/src/HDF5.jl:1269
 [5] m_read(::HDF5.HDF5Group) at /Users/Zach/.julia/packages/MAT/mPSjZ/src/MAT_HDF5.jl:198
 [6] read(::MAT.MAT_HDF5.MatlabHDF5File, ::String) at /Users/Zach/.julia/packages/MAT/mPSjZ/src/MAT_HDF5.jl:261
 [7] read(::MAT.MAT_HDF5.MatlabHDF5File) at /Users/Zach/.julia/packages/HDF5/Zh9on/src/datafile.jl:45
 [8] matread(::String) at /Users/Zach/.julia/packages/MAT/mPSjZ/src/MAT.jl:133
 [9] top-level scope at REPL[6]:1

The errors from HDF5 are pretty clear but I don't know why this is an issue with the file I have because it's doesn't seem to be improperly formatted. I was able to get it to load in Matlab fine and followed the instructions here to ensure that I have a supported file type. Which produce 'MATLAB 7.3 MAT-file, Platform: GLNXA64, Created on: Thu Aug 15 10:14:40 2019 HDF5 schema 1.00 . .
Update:
I've gone back through and tried to figure out what's wrong. For some reason a mat file with these variables loads fine:

all_id : 1065x1 double
loaded_bd_network : 68x68x1065 logical
loaded_tensor : 4-D double
mode3 : 1x20 cell

But these variables error:

all_id : 1997x1 string
loaded_bd_network : 68x68x1997 logical
loaded_tensor : 4-D double
mode3 : 1x8 cell

I also have this problem. I did some trials and the error appears only for -v7.3 format, when I have a datetime object inside.

I'm also in the same situation. Though there are no datetime objects in my mat files.

It may be worth mentioning that any file I've encountered this error with here also doesn't open in anything but Matlab. In other words, I can't open it in Python or R.

Hmm, strange, I've managed to open them in python3 using h5py. I'll dig into it on the weekend. I'll make some test files and try and spot the difference.

Could you try with the branch from this PR to see if you can load the file? It won't make any unsupported types work, but at least the rest of the file will be readable.
In julia:
]add https://github.com/jebej/MAT.jl.git#patch-1


Merging 2.1 -> develop-2.2 for enhanced kv679 logging.

Bring in enhanced kv679 logging from 2.1 branch, thanks to @russelldb. Merges fixes for RIAK-2375 to develop-2.2

@borshop merge

RIAK-2375


Add TextEditingController parameter

I need the parameter "TextEditingController" for initialValue like initialValue of "TextFormfield".

Am also facing this same issue

I think we should be able to trigger a search on init, instead of manually modifying text property. Does this approach show progress indicator while performing search on initialization?

I think we should be able to trigger a search on init, instead of manually modifying text property. Does this approach show progress indicator while performing search on initialization?


Is there a way to force it to run with admin privileges?

As the title suggests I need a way to make my application always run with admin privileges, I compiled the electron app using electron-packager and set the exe to always admin but how can I do it with this package as well?

Right click on your exe -> Check run this program as administrator


[SPARK-24928][SQL] Optimize cross join according to stats

What changes were proposed in this pull request?
The cartesian product of 2 RDDs perform a nested loop. This means that the iterator for the inner RDD is built as many times as the number of rows of the outer one. If the two RDDs have a very different size, the performance difference can be huge.
As there is no way to know which is the best RDD to choose as outer one (since we don't know the sizes), this cannot be addressed at RDD level. Only a comment has been added to warn/help the user to be careful about how they write their code.
The PR proposed to add an optimizer rule which uses statistics collected on tables in order to change the sides of the cartesian product so that the outer table is the smaller one.
How was this patch tested?
added test suite

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1853/
Test PASSed.

Test build #94293 has started for PR 22008 at commit 1caf256.

Test build #94293 has finished for PR 22008 at commit 1caf256.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
trait JoinHelper extends PredicateHelper

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94293/
Test FAILed.

retest this please

Test build #94365 has started for PR 22008 at commit 1caf256.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1906/
Test PASSed.

Test build #94365 has finished for PR 22008 at commit 1caf256.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
trait JoinHelper extends PredicateHelper

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94365/
Test FAILed.

Merged build finished. Test FAILed.

retest this please

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/1916/
Test PASSed.

Test build #94378 has started for PR 22008 at commit 1caf256.

Test build #94378 has finished for PR 22008 at commit 1caf256.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
trait JoinHelper extends PredicateHelper

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94378/
Test PASSed.

cc @wzhfy