Consider to support packets path probing?

I have checked the features of ovn and not found the feature that support packets path probing. I also search some relevant papers, such as the below papers:

SDN traceroute: Tracing SDN Forwarding without Changing Network Behavior
SDNProbe: Lightweight Fault Localization in the Error-Prone Environment
SDN Spotlight: A real-time OpenFlow troubleshooting framework

So I want to know whether we have a plan to develop the feature or tool that support packets path probing.

@gujun4990 one thing that might be interesting to check out is https://github.com/retis-org/retis

wow, thank you. I will prepare to investigate the project.

I have review the work about retis project. Nice to satisfy my requirements. I will continue to follow the project. Thanks again.


Create Task from tasks page should not assume org

Currently importing creates the task using the first org that is available:
https://github.com/influxdata/influxdb/blob/c20422beee7985d8bb7d7be4abf987556865c2a0/ui/src/tasks/actions/v2/index.ts#L436

Import overlay might need an org dropdown??

this should now use the org context that the user is in when they create the task.


Expose multiple ports from a service

Tell us about your request
Based on the requirements for an integration PoC for Meilisearch I want to be able to expose two ports from the container. One port will be the terminal accessible from the browser and the other will be the port from where the service itself can be accessed from.
Which service(s) is this request for?
Both
Describe the solution you'd like
A configuration in the docker-compose would be hopefully future proof would look like.
x-uffizzi:
  ingress:
    service: meilisearch
    port: 7681 # default port
  expose:
  - service: meilisearch
    ports:
    - "3000:7700"
  - service: frontend
    ports:
    - "3001:7700"

In the above ingress spec would cover the default endpoint. UFFIZZI_URL:80 will point to this.
The expose spec is a list which would contain other services/ports to be exposed. In the above example, the first service is the same default service exposed at ingress but the container is different. The ports directive will give all the ports exposed from the container. It should be possible to forward requests from one port exposed on the ingress to a different port on the container. In this case requests coming to UFFIZZI_URL:3000 would be forwarded to the 7700 port of the meilisearch container. Similarly for the next example, requests coming to UFFIZZI_URL:3001 would be forwarded to the 7700 port of the frontend container.

Are both of these ports serving HTTP? Or other TCP services? UDP?
Uffizzi only supports a single HTTP service. This assumption is baked into our entire stack.
  For example, https://example.app.uffizzi.com/ could go to the frontend container and requests for https://example.app.uffizzi.com/meilisearch/ could go to the other container.

Docker Compose already supports both expose and ports syntaxes (See a description of there differences here). We don't need to nest it under x-uffizzi.
They are both sub-elements of the service definition which can include a list of ports. For example:
myservice:
  image: foo:latest
  ports:
    - "3000"
    - "3000-3005"
    - "8000:8000"
    - "9090-9091:8080-8081"
    - "49100:22"

myservice:
  image: foo:latest
  expose:
    - "3000"
    - "8000"

I don't think the ports and expose keywords do anything on Uffizzi. All of a deployment's containers share the same IP network stack within the same k8s Pod.

 For example, https://example.app.uffizzi.com/ could go to the frontend container and requests for https://example.app.uffizzi.com/meilisearch/ could go to the other container.

@axisofentropy yes both are HTTP services. It is a good idea I can try that. It just might work for them.

I don't think the ports and expose keywords do anything on Uffizzi. All of a deployment's containers share the same IP network stack within the same k8s Pod.

@gadkins I agree with @axisofentropy here.
On another note can this multiplexing be a part of the uffizzi configuration. It would be a pain to write the nginx config and build a new nginx image everytime. Would be nice to have the multiplexing be a part of the uffizzi config


Read me

Added a gif and instructions for running vno run build --ssr.  pay special attention to the folder formatting in my gif vs the others,
QUESTION:
does it make sense to have someone with a mac make their folder theme dark and recreate the build --ssr gif on the same OS as the original gifs, or have me overwrite the old mac gifs with new gifs from ubuntu, or accept that they just wont match exactly and live with it?

wrong pull request


How to choose Intel Arc GPU in 227-whisper-subtitles-generation

Describe the bug
import whisper
model = whisper.load_model(model_id.value, "cpu")
pass
Expected behavior
There is no any option to choose GPU, only CPU or CUDA.

No issue now


2nd Dev test: Circuit Construction Kit: DC PhET-iO

This will be the 2nd dev test. First one was completed in https://github.com/phetsims/qa/issues/772.  The new URL for testing is:
https://phet-dev.colorado.edu/html/circuit-construction-kit-dc/1.3.0-dev.15/phet-io/
Addressed issues were reviewed in master.  Some of the discovered issues from https://github.com/phetsims/qa/issues/772 were postponed to a future milestone.
Should I copy/paste the template from https://github.com/phetsims/qa/issues/772?

@samreid Yes, please use either that template or a new one from the repo.

QA is done

Thanks!  I marked the open issues for the DC PhET-iO milestone.  Closing.


DNS: Actually use the configuration value 'primary'

We started reading that value in a previous PR, but didn't make use of it.

Giving this a speedy merge as it's blocking v0.28.0 and I just tested it successfully live.


Job PublishBatch long processing time SlotWaitTime

Contact Details
<EMAIL_ADDRESS>Version
8.x
On which operating system(s) are you experiencing the issue?
Linux
Using which broker(s) did you encounter the issue?
In-Memory
What are the steps required to reproduce the issue?
This has been tested on both 8.0 and 8.1

1. Define a job which takes a little processing time `Task.delay(50)`;
2. Create a Job with a concurrency limit of 1 `.SetConcurrentJobLimit(1)`
3. Then publish a batch of 10 jobs using `bus.PublishBatch(jobs)`
4. Watch the log output for when the jobs are processed.

What is the expected behavior?
That the jobs queued would be processed within a few seconds, since the sum of job processing time within the Run methods are ~500 ms.
What actually happened?
That takes ~300 seconds to execute.
I suspect the issue is with the SlotWaitTime, where in the current way it is designed all jobs tries to take the available slot as they come in, only a single gets the slot and the rest waits 30 seconds. After 30 seconds the remaining 9 jobs tries again to take the slot, 1 gets the slot, the remaining 8 sleeps again for 30 seconds, and so on.
I have tried setting the SlotWaitTime to 10 seconds, and as suspected, then the tasks complete in about ~100 seconds.
Related log output, including any exceptions
Gist log
Link to repository that demonstrates/reproduces the issue
No response

As you experienced, the slot wait time is what determines the delay before a job will again try to allocate a job slot. This is working as designed.


Can specify length of password generated

Added small textbox that can edit the length of the password generated - without having to go through the config file.

Thanks for the PR! This is a feature I have wanted to add for some time, a review is coming up.


NIFI-13457 Remove nifi-python-extensions

Summary
NIFI-13457 Removes the nifi-python-extensions module from the repository following the initial migration of these Python Processors to the nifi-python-extensions repository for separate distribution as described in NIFI-13445.
Tracking
Please complete the following tracking steps prior to pull request creation.
Issue Tracking

[X] Apache NiFi Jira issue created

Pull Request Tracking

[X] Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
[X] Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

[X] Pull Request based on current revision of the main branch
[X] Pull Request refers to a feature branch with one commit containing changes

Verification
Please indicate the verification steps performed prior to pull request creation.
Build

[X] Build completed using mvn clean install -P contrib-check

[X] JDK 21

Licensing

[ ] New dependencies are compatible with the Apache License 2.0 according to the License Policy
[ ] New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

[ ] Documentation formatting appears as expected in rendered files

changeset looks great.  Doing a full local build and will run a quick test on docker based deploy
but this combined with the import of these components into the nifi-python-extensions and we're in good shape!


jest support microsoft VSTS  #help wanted

Is it possible to do test reports on microsoft VSTS (https://docs.microsoft.com/en-us/vsts/test/run-automated-tests-from-test-hub?view=vsts) like on jenkins test job to get the report.

Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.


EntryStream.sorted()

Calling sorted() on an EntryStream<String, String> throws the following exception:

Exception in thread "main" java.lang.ClassCastException: java.util.HashMap$Node cannot be cast to java.lang.Comparable
at java.util.Comparators$NaturalOrderComparator.compare(Comparators.java:47)
at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355)
at java.util.TimSort.sort(TimSort.java:220)
at java.util.Arrays.sort(Arrays.java:1512)
at java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:348)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at one.util.streamex.AbstractStreamEx.forEach(AbstractStreamEx.java:356)
at one.util.streamex.EntryStream.forKeyValue(EntryStream.java:1345)

Please change sorted() to sort by the HashMap key first and then by value.

Sorry, but I cannot fix this as it would break the contract of the Stream interface the EntryStream class implements. See the specification:

Returns a stream consisting of the elements of this stream, sorted according to natural order. If the elements of this stream are not Comparable, a java.lang.ClassCastException may be thrown when the terminal operation is executed.

As EntryStream implements Stream, it must obey its contract. Here as stream elements (entries of HashMap) are not Comparable, no sorting could be performed. If you have your custom map implementation which produces comparable entries, then EntryStream.sorted() will work.
You may enforce custom sorting order supplying your own comparator (e.g. Map.Entry.comparingByKey().thenComparing(Map.Entry::getValue)).


choose a template engine

TODO: copy wiki page here, contact professional services team

Not sure if understand what you mean here. I thought there is only 1 engine called scala templates, right?

It is the default template engine, but you can use others, too.

Maybe you can list examples and what would be a benefit of using them?

:) We have a meeting scheduled for this today.

Still would be great to know in advance what technologies do we consider here.

Flexible template engine used, by default Sunrise will use Handlebars.


Migrate build scripts to cmake

Task
Remove task.js and migrate all build related functionality to cmake.

I made a CmakeLists file that can be used to generate a Makefile that can be used to compile and build a static library. Here it is:
https://github.com/augre/cryptid-native/blob/master/CMakeLists.txt
The tests can be added to but I didn't worry about that now.

@augre
Cool, thanks for sharing your solution! We are still quite unsure regarding the transition to CMake (as task handles quite a lot of complex build logic), but we will definitely use the file you provided as a starting point! Thank you!


Registry tool: add registry rules for Score

Add Score checks to registry check command.

rule group
rule number
resource
field
type
severity
check

Score
1100
artifacts/apihub-styleguide

ERROR
must load into proto type

Score
1100
artifacts/apihub-lint-errors

ERROR
must load into proto type

Score
1100
artifacts/apihub-lint-summary

ERROR
must load into proto type

Score
1100
artifacts/apihub-lint-warnings

ERROR
must load into proto type

Score
1100
artifacts/score-security-audit

ERROR
must load into proto type

Score
1100
artifacts/score-lint-error

ERROR
must load into proto type

Score
1100
artifacts/score-accuracy

ERROR
must load into proto type

Score
1100
artifacts/score-lang-reuse

ERROR
must load into proto type

Score
1100
artifacts/score-receipt

ERROR
must load into proto type

Score
1100
artifacts/scorecard-lint-summary-proto

ERROR
must load into proto type

Score
1100
artifacts/scorecard-lint-summary-openapi

ERROR
must load into proto type

Score
1100
artifacts/score-definition

ERROR
must load into proto type

Score
1100
artifacts/scorecard-quality

ERROR
must load into proto type

These use the same internal mime types and so are already handled by apihub rule1002.


ARROW-6052: [C++] Split up arrow/array.h/cc into multiple files under arrow/array/, move ArrayData to separate header, make ArrayData::dictionary ArrayData

One meaningful change in this patch was to change ArrayData::dictionary to be shared_ptr<ArrayData>, which was not difficult.
Otherwise, this allows some files that only need ArrayData or just Array to not include a bunch of unnecessary header code. The increase in number of lines of code is pretty much all due to license headers.
I did a modest amount of IWYU cleaning, there are plenty of still usages of "arrow/array.h" that might be better replaced by more specific headers.
As part of this patch I disabled build/include_what_you_use in cpplint per ARROW-8994. If we're going to spend time doing IWYU cleaning (which IMHO is overall worthwhile) we should rely on its output rather than having IWYU checks by two different tools.

Probably be good to merge this soon to limit rebase headaches

I'll rebase and address the comments, and then merge later today.

Appveyor looks fine: https://ci.appveyor.com/project/wesm/arrow/builds/33267645. I'm going to go ahead and merge this


feat: use blake2 for variable hashcode

Description
Fixes #1146.
Instead of a fast hash non-cryptographic hash use cryptographically secure hash. Previously it was possible for a malicious circuit designer to define a circuit in a way which would omit either boolean marking or non-native element range checking by creating collisions between variables. This PR avoids this by using a strong hash function instead.
NB! It uses as hash outputs only 16 bytes instead of full 32 bytes for saving some memory.
Type of change

[x] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[x] This change requires a documentation update

How has this been tested?
Existing tests pass
How has this been benchmarked?
For example, I compiled the BW6-671 PLONK verifier in BN254 as one of the biggest circuits to get estimates in possible performance bottlenecks. Results:
# PRE patch

SCS:
* time 37.39810504s
* mem 32498MB

R1CS:
* time 46.508562287s
* mem 65606MB

# POST patch

SCS:
* time 37.215583182s
* mem 32430MB

R1CS:
* time 47.082183625s
* mem 66455MB

So it seems that for SCS it actually improves performance (but only very little). For R1CS we have approx 1% more memory allocations and 1% slower compiler. I think we could also try to optimize the hash code computation a little by using fixed buffers and hasher as the compiler is sequential anyway. And we can provide the destination to store the hash output instead of allocating new.
This makes the code a little less readable though, but may be worth it?
Checklist:

[x] I have performed a self-review of my code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] I have added tests that prove my fix is effective or that my feature works
[x] I did not modify files generated from templates
[x] golangci-lint does not output errors locally
[x] New and existing unit tests pass locally with my changes
[x] Any dependent changes have been merged and published in downstream modules

the impact of doing a collision resistant hashcode was if I recall, quite high in term of perf for the compile phase of a large circuit.
Can you do a quick benchmark (frontend.Compile) of a large circuit before / after the PR?
edit: sorry missed the benchmark in the description

Yup, I also recall that it was supposed to be slower, and tested runtime and profiled memory usage. I guess with Blake and 16 byte output its still reasonable.
I could test binary heavy stuff though separately (with a lot of MarkBoolean) to confirm


Refactoring Superuser contract to allow Owners to transfer ownership …

Fixes #50
Refactor to the Superuser contract to allow owners that are not superusers to transfer the ownership of the contract.

[X] 📘 I've reviewed the OpenZeppelin Contributor Guidelines
[X] ✅ I've added tests where applicable to test my new functionality.
[X] 📖 I've made sure that my contracts are well-documented.
[X] 🎨 I've run the JS/Solidity linters and fixed any issues (npm run lint:all:fix).

In Ownable.sol I extracted the functionality of transferOwnership to an internal function and moved the renounceOwnership function above the transferOwnership function.

Thanks @pmosse and @shrugs!

Great! @frangio


Detected Blocking Call

Before you submit a new bug report, please check that

[X] I have read frequently asked questions and limitations
[X] I'm running the newest version of Holidays, or the next pre-release.
[X] I have enabled debug logging for my installation.

Describe the bug
Errors found in log file
Configuration
Home Assistant

Core
2024.6.4
Supervisor
2024.06.2
Operating System
12.4
Frontend
20240610.1

Debug logs
2024-06-27 13:08:04.013 DEBUG (MainThread) [custom_components.holidays] Setting Complete from ConfigFlow
2024-06-27 13:08:17.195 DEBUG (MainThread) [custom_components.holidays.calendar] (Complete) Calling update
2024-06-27 13:08:17.195 DEBUG (MainThread) [custom_components.holidays.calendar] (Complete) Country Holidays with parameters: country: CA, subdivision: BC, observed: True
2024-06-27 13:08:17.204 WARNING (MainThread) [homeassistant.util.loop] Detected blocking call to open inside the event loop by custom integration 'holidays' at custom_components/holidays/__init__.py, line 133: return holidays.country_holidays(country, **kwargs) (offender: /usr/local/lib/python3.12/gettext.py, line 537: with open(mofile, 'rb') as fp:), please create a bug report at https://github.com/bruxy70/Holidays/issues
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 223, in <module>
    sys.exit(main())
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 209, in main
    exit_code = runner.run(runtime_conf)
  File "/usr/src/homeassistant/homeassistant/runner.py", line 190, in run
    return loop.run_until_complete(setup_and_run_hass(runtime_config))
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 672, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 639, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1988, in _run_once
    handle._run()
  File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 680, in _async_handle_interval_callback
    self.config_entry.async_create_background_task(
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 1164, in async_create_background_task
    task = hass.async_create_background_task(target, name, eager_start)
  File "/usr/src/homeassistant/homeassistant/core.py", line 860, in async_create_background_task
    task = create_eager_task(target, name=name, loop=self.loop)
  File "/usr/src/homeassistant/homeassistant/util/async_.py", line 37, in create_eager_task
    return Task(coro, loop=loop, name=name, eager_start=True)
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 1050, in _async_update_entity_states
    await entity.async_update_ha_state(True)
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 941, in async_update_ha_state
    await self.async_device_update()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1289, in async_device_update
    await self.async_update()
  File "/config/custom_components/holidays/calendar.py", line 276, in async_update
    await self._async_load_holidays()
  File "/config/custom_components/holidays/calendar.py", line 103, in _async_load_holidays
    hol = create_holidays(
  File "/config/custom_components/holidays/__init__.py", line 133, in create_holidays
    return holidays.country_holidays(country, **kwargs)

2024-06-27 13:08:17.208 DEBUG (MainThread) [custom_components.holidays.calendar] (Complete) Found these holidays:
  2023-01-01: New Year's Day
  2023-01-02: New Year's Day (observed)
  2023-02-20: Family Day
  2023-04-07: Good Friday
  2023-05-22: Victoria Day
  2023-07-01: Canada Day
  2023-08-07: British Columbia Day
  2023-09-04: Labour Day
  2023-09-30: National Day for Truth and Reconciliation
  2023-10-09: Thanksgiving Day
  2023-11-11: Remembrance Day
  2023-12-25: Christmas Day
  2024-01-01: New Year's Day
  2024-02-19: Family Day
  2024-03-29: Good Friday
  2024-05-20: Victoria Day
  2024-07-01: Canada Day
  2024-08-05: British Columbia Day
  2024-09-02: Labour Day
  2024-09-30: National Day for Truth and Reconciliation
  2024-10-14: Thanksgiving Day
  2024-11-11: Remembrance Day
  2024-12-25: Christmas Day
  2025-01-01: New Year's Day
  2025-02-17: Family Day
  2025-04-18: Good Friday
  2025-05-19: Victoria Day
  2025-07-01: Canada Day
  2025-08-04: British Columbia Day
  2025-09-01: Labour Day
  2025-09-30: National Day for Truth and Reconciliation
  2025-10-13: Thanksgiving Day
  2025-11-11: Remembrance Day
  2025-12-25: Christmas Day
2024-06-27 13:08:17.208 DEBUG (MainThread) [custom_components.holidays.calendar] (Complete) Looking for the next holiday
2024-06-27 13:08:17.208 DEBUG (MainThread) [custom_components.holidays.calendar] (Complete) next_date (2024-07-01), today (2024-06-27)

I also get this error every time I restart HA


Small changes

Hi Luke,
I never would have dreamt that a lib would be created on top on Permission 😍😍
Thank you so much!!!
I tried to give back a bit with this PR ;)

Hey Damien! 🎉
First of all thanks for your kind words! I really love Permission so you did great job on that! 💯 As for pull request, it seems also really nice, thank you! I've made some comments but that shouldn't be a problem 🐼

😱 Sorry for all the errors, I coded without the compiler...
I rebased the PR, it should be fine now ;)
BTW do you plan to support Carthage ?

No worries 🐼 It looks perfect now, thanks again! About Carthage, of course, just I'm not that experienced with Cathage so wanted to do it later, but if you have any experience, I would be more than happy to see a PR from you :)

I've also added you as a collaborator to this project, if you don't mind 🎉

Ahah great thanks!


CentOS tf examples

hi,
i'm new to terraform. i read example files but there was not example due to install centos vm (guest).
i want to know are there any differences if you want to install different distro?
i'd be happy if you add some basic centos tf configuration on 0.13 .
thanks alot.

Hello mahdiarfrm  ,
To install CentOS , you need to point terraform to a libvirt-compatible image, such as the qcow2 images found here
Next, create a "libvirt_volume" resource that points to a local or remote copy of your image. I prefer to use local copies:
resource "libvirt_volume" "minecraft_vol" {
  name   = "minecraft_vol"
  pool   = "default"
  format = "qcow2"
  source = "/home/inflatador/images/CentOS-Stream-GenericCloud-8-20210603.0.x86_64.qcow2"
}

From there, configuration is identical to any other distro. Check the examples folder of this repo for specifics. Good luck!

Hi @mahdiarfrm,
thanks to @inflatador for providing tips, this can be closed?


imageLazilyCachedAtIndex problem

Hi @raphaelschaad !
I'm trying  to retrieve all the images of a gif, howerver this only works when the size of the gif is less than 1.5 mb. Is that normal ?
If yes, do you another way to retrieve all the images of a gif ?
Thank you :)

You could poll FLAnimatesImage the same way FLAnimatedImageView does to
retrieve all the frames of the GIF; and it should work for larger ones as
well.
On 03-Mar-2015, at 7:51 pm, Roger Ingouacka<EMAIL_ADDRESS>wrote:
Hi @raphaelschaad https://github.com/raphaelschaad !
I'm trying to retrieve all the images of a gif, howerver this only works
when the size of the gif is less than 1.5 mb. Is that normal ?
If yes, do you another way to retrieve all the images of a gif ?
Thank you :)
—
Reply to this email directly or view it on GitHub
https://github.com/Flipboard/FLAnimatedImage/issues/67.

Thanks @caughtinflux
Do you have a little example, that shows me how to do this ?

@caughtinflux  ?

Have a look at FLAnimatedImage's display link callback. However, FLAnimatedImage isn't exactly built for this; you could simply use one of the GIF to Animated UIImage libraries and get an array of all the frames in a GIF.

FLAnimatedImage isn't designed to pull each of the frames of an animated GIF, it's designed to play animated GIFs while minimally impacting performance, which means it often dynamically loads frame in and out of memory. If you need to get the frames of an animated GIF I'd recommend using a CGImageSource and CGImageSourceCreateImageAtIndex as we do here.


How can we copy and view the data just from text editor?

URL for relevant page?
What products and version are you using?
Where did you look before opening the issue?

In what form do you want to data? The actual on disk files? Raw query results? Line protocol?

Closing this since no response from @mraj16. If we get more information, this can be reopened.


Incorrect drawing order of multiple styles

The third style is drawn second. Full code in the sandbox:
https://codesandbox.io/p/sandbox/heuristic-tdd-ymsvrf

Thanks for the bug report. Can you please provide an example that uses OpenLayers styles directly, without ol-mapbox-style?

VectorTileLayer and StyleFunction too:
https://codesandbox.io/p/sandbox/ol-styles-forked-n2sycq?workspaceId=4c85bd12-2428-448e-8638-7ecd9fd42a9f
But a regular StyleFunction has a slightly different drawing order in principle.

There is a suggestion. The problem is in ZIndex. Without it, the order of styles is predictable:
https://codesandbox.io/p/sandbox/ol-styles-function-forked-57rksy

The above example (with zIndex) looks correct too. Or am I missing something?
I think the problem might be decluttering - when the layer is configured with declutter: true and the circle styles with declutterMode: 'none' (this is what ol-mapbox-style does). Is that the case?

I changed the example for clarity. The second style overlaps the third.

Thanks, @levelost. This should help to investigate the issue further.

Good find, @levelost! Would you be able to create a pull request?


Bugfix/fix errored pds download

See internal detail at https://jira.corp.code42.com/browse/INTEG-1124.
If the GetFile request to the storage node fails, today this is resulting a stream that contains an error message from the server being returned instead of properly failing. This appears to have been caused by some differences in http headers.
To test:
# this hash produces the error in our "partner" account, should result in a raised`Py42Error`.
sdk.securitydata.stream_file_by_sha256("23df162cb046316bed3091cc41bc721a5677963ceefa912e64fb3d7293db1434")

# also ensure that the normal use case still works.
# Inspect the contents of written file to verify it does not simply contain an error message.
# from the "demo" account
stream = sdk.securitydata.stream_file_by_sha256("098db7aff65b03a60eb784d4005f08325243d8e747f52a98244ff6116171f56d")
with open("testfile.txt", "wb") as test:
    for chunk in stream.iter_content():
        test.write(chunk)

What about the other stream_from_backup method? Does that have this issue?

@unparalleled-js good question, I could try to web restore the same file that the hash is trying to restore and see.

hmm, the error that was happening on the server went away now and these files are both successfully being downloaded. For what it's worth, I tested this while the error was still reproducible, and it did the right thing.
I think we'll have to hold off on any changes to stream_file_from_backup until we have evidence / reason to believe that it is not working properly.

I found a different "bad"' hash, updated the description.

Codecov Report

Merging #134 into master will decrease coverage by 0.07%.
The diff coverage is 92.85%.

@@            Coverage Diff             @@
##           master     #134      +/-   ##
==========================================
- Coverage   91.36%   91.29%   -0.08%
==========================================
  Files          61       61
  Lines        2073     2079       +6
==========================================
+ Hits         1894     1898       +4
- Misses        179      181       +2

Impacted Files
Coverage Δ

src/py42/response.py
94.91% <88.88%> (+0.36%)
:arrow_up:

src/py42/_internal/client_factories.py
88.42% <100.00%> (+0.09%)
:arrow_up:

src/py42/_internal/clients/storage/storagenode.py
100.00% <100.00%> (ø)

src/py42/modules/securitydata.py
94.83% <100.00%> (+0.60%)
:arrow_up:

src/py42/_internal/compat.py
70.00% <0.00%> (-30.00%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b40bb7a...bedaadf. Read the comment docs.


progress bar

Adds a simple progress bar for db initialization. Will be connected to dat download progress, bur for now just provides the UI.

optional info we can show from dat:

amount of sources downloading from
total download speed
estimated time to completion

ah sweet, we'll probably want some of those permanently in the bottom bar (where transfer speed is now), because we'll want them during other downloads too.
maybe this? (where "results" only appears once you start searching)
@Blahah is that the extra info you wanted in the footer? other stuff too?

Yes! this is what I was imagining :)

All this stuff is now included - your designs were used @freeman-lab but it was implemented as choo views. Will land in v1.


azure-mgmt-compute 0.31.0

Generated from RestAPI PR: https://github.com/Azure/azure-rest-api-specs/pull/642

Coverage increased (+0.2%) to 56.344% when pulling 119b98ed10881764dcf48a0dfe5ea518b3423c02 on AutorestCI:RestAPI-PR642 into 742566545d7c6a7ee42c575a75b21cb8697e12c4 on Azure:dev.


docs: correction for TerminalError

Codecov Report
Base: 83.74% // Head: 83.64% // Decreases project coverage by -0.10% :warning:

Coverage data is based on head (e4c8212) compared to base (7aef562).
Patch has no changes to coverable lines.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #53      +/-   ##
==========================================
- Coverage   83.74%   83.64%   -0.11%
==========================================
  Files          21       21
  Lines        2989     2989
==========================================
- Hits         2503     2500       -3
- Misses        317      319       +2
- Partials      169      170       +1

Impacted Files
Coverage Δ

generate.go
63.56% <0.00%> (-1.17%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Prognostic run doesn't output all requested outputs

The prognostic run will not fail as expected when an unknown variable is requested AND kind: interval is used. For example, this diagnostic configuration will not result in a crash:
            {
                "name": "diags_3d.zarr",
                "times": {"kind": "interval", "frequency": 900},
                "chunks": {"time": 12},
                "variables": ["not an available input variable"],
            }

This happens because of this line of code:
https://github.com/VulcanClimateModeling/fv3net/blob/1861596078da06665862aa05080b58bc1e5be359/workflows/prognostic_c48_run/runtime/diagnostics/manager.py#L153

Fixing this bug breaks some of the tests. The existing behavior of All() is not compatible with error checking.


Snippet insertion repeating character(<)

Hi, first I would like to say that your extension is very handy.
The extension works flawlessly when directly typing the tagname: it considers what already has been written and then completes it with all missing symbols and characters.
But when you start typing with the < symbol (e.g. like this  <tagName) the extension appears to have 2 problems, or are they intended features?
Considering VSCode 1.10.2

First problem:
When typing <tagName it doesn't present any possible snippets, at all. Unless you press Enter after selecting the tagName in the auto-complete list, then CTRL+Space, then, and only then, the snippet shows.
e.g.:
First attempt with CTRL+Space after typing <a, only the IDE a tag appears:

With no snippet option:

After selecting the Auto-complete a tag, and hitting CTRL+Space again, the snippet option finally appears:

Second problem:
Continuing the above process, after hitting Enter over the a snippet option:

The extension places the snippet, but it doesn't take the symbol < into consideration, leaving it, which I think it shouldn't.

hello @tradeli ,
thanks for the details explanation.
firstly, I made this extension was vscode had very little support for html autocomplete. the extension in its basic format is just snippet list that the vscode snippet engine feeds the workflow when it thinks a match is possible, the extension contains no logic to be smart about when to appear or how to handle the pre or post characters. if you are seeing the extension's suggestions below the default vscode html suggestions I believe there is a switch to set in the settings to allow the extensions to precede the default language type suggestions, again this is not just for this extension, this is for all snippet type extensions.
I have had many people enquire about similar features you have referred to, as to be able to understand what is on the page and the extension should provide that, I have been thinking of making a full fledged html extension that handles all user needs with that regard I just haven't had time to do so.
I hope this answers some of your thoughts.

Yes, you did answer, thank you!
I'll improve the configuration of my installed VSCode.
I guess I should also try and get better at programming, so I can help you with it! Maybe one day :)


Disable pointer-events on unread indicators

The unread indicator was blocking me from clicking on a server to switch to it

Thanks for the PR :)


Collector Chart for Google Cloud Exporter

Hi guys,
I'm trying to deploy OpenTelemetry Collector using Helm.
I've tried the deployment by following the example chart.
mode: deployment

ports:
  jaeger-compact:
    enabled: false
  jaeger-thrift:
    enabled: false
  jaeger-grpc:
    enabled: false
  zipkin:
    enabled: false

config:
  receivers:
    jaeger: null
    prometheus: null
    zipkin: null
  exporters:
    logging:
      verbosity: "detailed"
    googlecloud:
      retry_on_failure:
        enabled: false
      log:
        default_log_name: opentelemetry.io/collector-exported-log
  processors:
    memory_limiter:
      check_interval: 1s
      limit_percentage: 65
      spike_limit_percentage: 20
    batch: {}
    resourcedetection:
      detectors: [gcp]
      timeout: 10s
  service:
    pipelines:
      traces:
        receivers: [otlp]
        processors: [memory_limiter, batch]
        exporters: [logging,googlecloud]
      metrics:
        receivers: [otlp]
        processors: [memory_limiter, batch]
        exporters: [logging,googlecloud]
      logs:
        receivers: [otlp]
        processors: [memory_limiter, batch]
        exporters: [logging,googlecloud]

But I get difficulties putting the service account JSON for google cloud exporter.
I did this for docker:
docker run --volume "$(pwd)/cloud-trace-sa.json:/etc/otel/key.json" --volume "$(pwd)/config.yaml:/etc/otelcol-contrib/config.yaml" --env GOOGLE_APPLICATION_CREDENTIALS=/etc/otel/key.json -p 4317:4317 -p 4318:4318 -p 55681:55681 --rm otel/opentelemetry-collector-contrib
Did you guys know how to put the service account?

@tatashidayat I am not totally following your question.  Does the Google Cloud Exporter require a secret stored in an env var called GOOGLE_APPLICATION_CREDENTIALS and you're wondering how to set that secret and make it accessible with a k8s deployment?

@TylerHelmuth Google Cloud Exporter use env called GOOGLE_APPLICATION_CREDENTIALS to get the service account file path. My question is how set that ENV and put my service account json into the container path.

@tatashidayat you can use the extraEnvs field in the values.yaml to set environment variables.

That's nice @TylerHelmuth, how about put my service account (as a json file) into the container?
I wonder if we can mount a file from a secret

@tatashidayat you can use extraVolumes and extraVolumeMounts to add secrets

Thanks @TylerHelmuth, It works by adding:
extraVolumes:
  - name: secret-mount
    secret:
      secretName: otel-gcloud-sa
      optional: false

extraVolumeMounts:
  - name: secret-mount
    mountPath: /etc/otel

extraEnvs:
- name: GOOGLE_APPLICATION_CREDENTIALS
  value: /etc/otel/creds


Update to v1.6.1

Version bump, merging immediately.

docker-library PR: https://github.com/docker-library/official-images/pull/577

Sorry, was afk. Thanks for fixing this so quick @pesho! :tada:


JS heap blows up when running beacon node

Describe the bug

When running a synced lodestar beacon node (cmd: /lodestar beacon --network pyrmont --rootDir .pyrmont/ --eth1.providerUrl https://goerli.infura.io/v3/3ee38f4a9c4447dfbd050e5afbb4b9c6) and the validator (cmd: ./lodestar validator --network pyrmont --rootDir .pyrmont/ --graffiti "lodestar wuz here") with an activated pyrmont validator, on pyrmont, everything runs smoothly for about 16-18 hours (beacon node is stable, validator is attesting, etc.), and then I got an OOM issue.  I'm not sure if it's connected to the fact that both the validator and the beacon node were ran together, but the fact that other people have ran the beacon node without an OOM issue for quite some time (and that no one has really been running an active testnet validator alongside the beacon nod in lodestar, to my knowledge) gives me a hunch that it's connected to the validator.
EDIT: this issue happens when running just the beacon node by itself and does not require the validator to be ran to replicate the issue.
This is what my heap space looked like in grafana (the big empty space is between when the crash happened and when I noticed the crash)

here is my crash report:
https://pastebin.com/WJCEe6R6

External memory was slightly increasing over time as well:

External memory was slightly increasing over time as well:

here is my crash report:
report.20210128.061107.19505.0.001.json.txt

here is my crash report:
report.20210128.061107.19505.0.001.json.txt

I have the same issue without running a validator (beacon node only)
2021-01-28 23:31:00 [SYNC]             info: Imported blocks for slots blocks=357503,357504,357506,357507,357508,357509,357510,357511,357512,357513,357514,357516,357517,357518,357519,357520,357521,357522,357523,357524,357525,357526,357527,357528,357529,357530,357531,357532,357533,357535,357536,357537,357538,357539,357540,357542,357543,357544,357545,357546,357547,357548,357549,357550,357551,357552,357553,357555,357556,357557,357558,357559,357560,357561,357562,357563,357565,357566
2021-01-28 23:31:00 [CHAIN]            info: Block processed slot=357503, root=0x2b084f567f44a0c158fda41f6e32a3b694fc5f9eadfdb1ba6fefb1a34a7435b4

<--- Last few GCs --->

[81004:0x30f7230] 24156899 ms: Mark-sweep 2023.8 (2079.1) -> 2021.7 (2052.6) MB, 1199.8 / 0.9 ms  (+ 0.0 ms in 29 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 1210 ms) (average mu = 0.109, current mu = 0.009) finalize[81004:0x30f7230] 24158437 ms: Mark-sweep 2023.1 (2052.6) -> 2022.0 (2058.1) MB, 1529.9 / 0.8 ms  (average mu = 0.052, current mu = 0.005) allocation failure scavenge might not succeed

<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x140a8f9]
Security context: 0x2cc1aa9008d1 <JSObject>
    1: /* anonymous */(aka /* anonymous */) [0x31043e746e89] [/home/user/lodestar/packages/lodestar-beacon-state-transition/lib/fast/util/epochProcess.js:~60] [pc=0x2d03f8603781](this=0x1510846804b1 <undefined>,0x2d9c88a08c89 <Object map = 0x14d1e318e809>,7303)
    2: iterNode(aka iterNode) [0x31043e746f11] [/home/user/lodestar/packages/persistent-ts/lib/Vector.j...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0xa18150 node::Abort() [node]
 2: 0xa1855c node::OnFatalError(char const*, char const*) [node]
 3: 0xb9715e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
 4: 0xb974d9 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
 5: 0xd54755  [node]
 6: 0xd54de6 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [node]
 7: 0xd616a5 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [node]
 8: 0xd62555 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
 9: 0xd6500c v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
10: 0xd2ba2b v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node]
11: 0x106e15c v8::internal::Runtime_AllocateInOldGeneration(int, unsigned long*, v8::internal::Isolate*) [node]
12: 0x140a8f9  [node]

I have the same issue without running a validator (beacon node only)
2021-01-28 23:31:00 [SYNC]             info: Imported blocks for slots blocks=357503,357504,357506,357507,357508,357509,357510,357511,357512,357513,357514,357516,357517,357518,357519,357520,357521,357522,357523,357524,357525,357526,357527,357528,357529,357530,357531,357532,357533,357535,357536,357537,357538,357539,357540,357542,357543,357544,357545,357546,357547,357548,357549,357550,357551,357552,357553,357555,357556,357557,357558,357559,357560,357561,357562,357563,357565,357566
2021-01-28 23:31:00 [CHAIN]            info: Block processed slot=357503, root=0x2b084f567f44a0c158fda41f6e32a3b694fc5f9eadfdb1ba6fefb1a34a7435b4

<--- Last few GCs --->

[81004:0x30f7230] 24156899 ms: Mark-sweep 2023.8 (2079.1) -> 2021.7 (2052.6) MB, 1199.8 / 0.9 ms  (+ 0.0 ms in 29 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 1210 ms) (average mu = 0.109, current mu = 0.009) finalize[81004:0x30f7230] 24158437 ms: Mark-sweep 2023.1 (2052.6) -> 2022.0 (2058.1) MB, 1529.9 / 0.8 ms  (average mu = 0.052, current mu = 0.005) allocation failure scavenge might not succeed

<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x140a8f9]
Security context: 0x2cc1aa9008d1 <JSObject>
    1: /* anonymous */(aka /* anonymous */) [0x31043e746e89] [/home/user/lodestar/packages/lodestar-beacon-state-transition/lib/fast/util/epochProcess.js:~60] [pc=0x2d03f8603781](this=0x1510846804b1 <undefined>,0x2d9c88a08c89 <Object map = 0x14d1e318e809>,7303)
    2: iterNode(aka iterNode) [0x31043e746f11] [/home/user/lodestar/packages/persistent-ts/lib/Vector.j...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0xa18150 node::Abort() [node]
 2: 0xa1855c node::OnFatalError(char const*, char const*) [node]
 3: 0xb9715e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
 4: 0xb974d9 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
 5: 0xd54755  [node]
 6: 0xd54de6 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [node]
 7: 0xd616a5 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [node]
 8: 0xd62555 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
 9: 0xd6500c v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
10: 0xd2ba2b v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node]
11: 0x106e15c v8::internal::Runtime_AllocateInOldGeneration(int, unsigned long*, v8::internal::Isolate*) [node]
12: 0x140a8f9  [node]

I found some promises retained in the memory along with its data (CachedValidatorsBeaconState + EpochContext) and it keeps accumulated.

I tried to add myId to all suspected promises during epoch transition to find out which promises were retained and it's https://github.com/ChainSafe/lodestar/blob/master/packages/lodestar/src/chain/blocks/process.ts#L148
the strange thing is it only happens to some blocks randomly and they stayed in the memory even we processed them 10mins - 20mins ago, i.e. archived blocks passed those slots. @ChainSafe/eth2-0 do any of us have any ideas?

I've never seen anything like this, to narrow down the possibilities I would suggest

Making runStateTransition function sync and confirming if the leak persists
Using a vanilla version of sleep(0) and confirming if the leak persists
Reaching out to Chromium or NodeJS with this heap and see if they have some advice on what to do next

I wonder when the last time we ran the beacon node without getting an OOM was.  I hadn't ran the beacon node into regular sync in quite some time before I ran it and found this issue.  So, maybe one thing that could help is if we retrace our steps and look at the history of what was modified in the code section that Tuyen linked above (and/or the runStateTransition function itself) between the time were last able to run beacon node without OOM and now.

I wonder when the last time we ran the beacon node without getting an OOM was? I hadn't ran the beacon node into regular sync in quite some time before I ran it and found this issue. So, maybe one thing that could help is if we retrace our steps and look at the history of what was modified in the code section that Tuyen linked above (and/or the runStateTransition function itself) between the time were last able to run beacon node without OOM and now.

I've tried version 0.13.0 in 1h and still saw the memory increased significantly, we should sync our node more frequently from now on since we have contavo servers now. Initially we couldn't test Pyrmont sync in our old server due to 2GB RAM limitation, the other thing is blockchain in Pyrmont is not as long as the current.
In my local environment, I used to increase heap size option for Pyrmont in order to sync and restarted 1-2 times.

I've never seen anything like this, to narrow down the possibilities I would suggest

Making runStateTransition function sync and confirming if the leak persists
Using a vanilla version of sleep(0) and confirming if the leak persists
Reaching out to Chromium or NodeJS with this heap and see if they have some advice on what to do next

thanks for the suggestion. I tested the vanilla sleep(0) and still saw the issue. I'm testing a sync runStateTransition now and let's see how it works on server 01.
To make a sync runStateTransition(), I have to move sleep(0) to the outer for loop so it's kind of similar to my previous try. A debug in my local environment, after 2h test, is good - there is no CachedValidatorsBeaconState instances leaked and of course no Promise instances keeping them. I still see the memory slowly increased, not sure if it's because of my --inspect flag.

I still see the memory slowly increased, not sure if it's because of my --inspect flag.

Is the rate of memory increase much lower than on the current master?

I still see the memory slowly increased, not sure if it's because of my --inspect flag.

Is the rate of memory increase much lower than on the current master?

yeah just to give you an idea after I synced Pyrmont in 2h:

master should be >3GB or even 4GB at taht time, normally I stopped after 1h
this branch consume only 2GB at that time and it increased at a much slower speed, the increasement maybe due to my debug (--inspect) but at least I don't see CachedValidatorsBeaconState being leaked like in master

unfortunately I couldn't test on contavo-01 due to #1990 and lost all peers, it stopped at epoch 12468 so I have to switch back to that issue atm.

the other time I run a sync stateTransition in 2.5h, the heap ranges from 1.1GB to 1.7GB stably

One thing I noticed is the heap issue only seems to happen during regular sync.
The initial sync has a low and steady heap usage.

@tuyennhv , curretly running the beacon node on the tuyen/experiment-sync-runStateTransition branch.  i'm still waiting to see what happens with regard to the heap space usage, but i'm noticing that when i try to run the validator on the same branch alongside the beacon node (once the beacon node is in regular sync), then the validator consistently does not publish it's attestations (according to beaconcha.in: https://pyrmont.beaconcha.in/validator/adb60c447b8b59bfa0df75b88bd9ee4bd393258f917ab54eee6520787f3346b6b36fab203cd45baf7d308576a067cb3e#attestations), even though I see in the attestations being made in the validator logs.
For example here's some of the logs I see:
2021-02-02 21:53:43 []                 info: on new block epoch epoch=17249, validator=0xadb60c447b8b59bfa0df75b88bd9ee4bd393258f917ab54eee6520787f3346b6b36fab203cd45baf7d308576a067cb3e
2021-02-02 21:57:07 []                 info: Handling attestation duty slot=551985, committee=27, validator=0xadb60c447b8b59bfa0df75b88bd9ee4bd393258f917ab54eee6520787f3346b6b36fab203cd45baf7d308576a067cb3e
2021-02-02 21:57:11 []                 info: Signed new attestation block=0x18c153f031843f3e364de5a000927c93fe44ccc72b692ebe0ec5dfc4fd5a28d5, committeeIndex=27, slot=551985
2021-02-02 21:57:11 []                 info: Published new attestation slot=551985, committee=27, attestation=0x26d09431e78cf7e0fedd9fa0adec32d430987f333bb94e7c323a28e651a8e90b, block=0x18c153f031843f3e364de5a000927c93fe44ccc72b692ebe0ec5dfc4fd5a28d5, validator=0xadb60c447b8b59bfa0df75b88bd9ee4bd393258f917ab54eee6520787f3346b6b36fab203cd45baf7d308576a067cb3e
2021-02-02 21:57:15 []                 info: Aggregating attestations committeeIndex=27, slot=551985
2021-02-02 21:57:16 []                 info: Published signed aggregate and proof committeeIndex=27, slot=551985

and yet, if you look at the beaconcha.in link I provided above, the validator apparently missed that epoch.
I tried to replicate this behavior on master and was not able to.  So, something in that branch is making the validator (or perhaps the validator's interactions with the beacon node) act funny, it seems.

to follow up on the above comment, sometimes the failed attestation publish will show up in validator logs, but not always.  like this:
2021-02-02 22:07:47 []                error: Failed to publish attestation message=ATTESTATION_ERROR_ATTESTATION_ALREADY_KNOWN
2021-02-02 22:07:51 []                 info: Aggregating attestations committeeIndex=4, slot=552038
2021-02-02 22:07:51 []                error: Failed to produce aggregate and proof message=No matching attestations found for attestationData

so, the heap blew earlier for me this time (on tuyen/experiment-sync-runStateTransition) than on master.  looks like it may have plugged one memory issue but then introduced another

I got an out of memory crash today running on Mainnet on Digital ocean nodes

==== JS stack trace =========================================

Security context: 0x179d4cdc08a1 <JSObject>
    0: builtin exit frame: BigInt(aka BigInt)(this=0x1b5ec21004a9 <undefined>,40,0x1b5ec21004a9 <undefined>)

    1: readOnlyMap [0x3dd077e32789] [/usr/app/node_modules/@chainsafe/ssz/lib/backings/tree/array.js:~211] [pc=0x1c31e130bcdf](this=0x16c3a3cd2b01 <TreeHandler map = 0xc0f0df1f81>,0x174a8ddbf0c1 <Tree map = 0xd54264f1491>,0x174a8ddbf089 <JSFunction (sfi = 0x2ecf077fbfd1)>)
    2: process...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memo

UPDATE: After some hours from the Out Of Memory crash, memory is climbing again. The worrying part is that the added memory directly affects performance, reducing the sync speed by half. Why do you think the machine slows so much when memory is high? It can be due to frequent swaps or garbage collection runs?

One somewhat easy fix would be to reduce the state caches sizes while we're initial-syncing. Right now we're storing a BUNCH of states that we really don't need to keep around, which accounts for most of that memory.

One somewhat easy fix would be to reduce the state caches sizes while we're initial-syncing. Right now we're storing a BUNCH of states that we really don't need to keep around, which accounts for most of that memory.

Can you spec a proposal in a new issue to implement this? Sounds very important and urgent.

here's a proposal I have #2051

To debug the stairstepping memory, lets take a heap snapshot when the stairstep is low, then another when the stairstep is higher. And compare the snapshots.

I managed to capture a heapdump before the stair step and after the stair step (see the last two blue lines). They are 1 hour apart but relatively close to the step, given that it takes 3-5min to capture a heapdump.

Note: All heapsnapshots are available in our cloud node Contabo03 in /root/. Please feel free to scp the files to your local environment and investigate what's going on.
I've loaded add compared the heapsnapshots and the only big diff is (system). Any clue what's this?

Some nodes I've been running for +7 days show no memory leak
Node A
Note the 7 day scale in this chart

Node B Note the 30 day scale in this chart

Some or many of the improvements that we introduced in the last few months fixed all major leaks. I'll close the issue for now. Let's open a new one if new leaks appear when stressing the node in new situations

This happens again to my node after upgrading from 0.18.0 to 0.27.0
Aug-11 14:44:15.595 []                 info: Lodestar version=0.27.0 master 2c010b25, network=mainnet
Aug-11 14:44:15.680 [DB]               info: Connected to LevelDB database name=/srv/chain/lodestar/chain-db
Aug-11 14:44:20.333 []                 info: Initializing beacon state from anchor state slot=1217312, epoch=38041, stateRoot=0x677f9dcaf9e0cfa49586a020259c06480082988f5409472dd6c68127d8f43f80
Aug-11 14:44:32.248 [METRICS]          info: Starting metrics HTTP server port=8008
(node:1717966) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 uncaughtException listeners added to [process]. Use emitter.setMaxListeners() to increase limit
Aug-11 14:44:32.335 [API]              info: Started REST api server address=http://<IP_ADDRESS>:9596, namespaces=["beacon","config","events","node","validator"]
Aug-11 14:44:43.342 [NETWORK]          info: PeerId 16Uiu2HAkxpvEzCjLDCEKK4sxGH8TzjAr3j3H135brtQghsqo6Re9, Multiaddrs /ip4/<IP_ADDRESS>/tcp/9000,/ip4/<IP_ADDRESS>/tcp/9000
Aug-11 14:44:43.343 []                 warn: Low peer count peers=1
Aug-11 14:44:43.344 []                 info: Searching for peers - peers: 1 - finalized: 38039 0xbf67…e44c - head: 1217312 0xf5c7…19cd - clockSlot: 1821821
Aug-11 14:44:54.492 []                 info: Syncing - 1.2 days left - 5.74 slots/s - finalized: 38041 0xf5c7…19cd - head: 1217376 0x7be4…ee3b - clockSlot: 1821822 - peers: 2
Aug-11 14:45:15.248 []                 info: Syncing - 1.8 days left - 3.89 slots/s - finalized: 38043 0x7be4…ee3b - head: 1217440 0xfb97…2967 - clockSlot: 1821824 - peers: 4
Aug-11 14:45:17.465 []                 info: Syncing - 1.9 days left - 3.68 slots/s - finalized: 38043 0x7be4…ee3b - head: 1217441 0x8d03…ce63 - clockSlot: 1821824 - peers: 4
Aug-11 14:45:29.142 []                 info: Syncing - 1.8 days left - 3.88 slots/s - finalized: 38044 0x13cc…e6de - head: 1217503 0x010a…83d0 - clockSlot: 1821825 - peers: 5
Aug-11 14:45:45.344 []                 info: Syncing - 1.9 days left - 3.60 slots/s - finalized: 38046 0xd1ba…1503 - head: 1217537 0x8cf6…f5cf - clockSlot: 1821826 - peers: 5
Aug-11 14:45:53.848 []                 info: Syncing - 2 days left - 3.53 slots/s - finalized: 38047 0x584c…b199 - head: 1217569 0x3778…1bc7 - clockSlot: 1821827 - peers: 5
Aug-11 14:46:05.112 []                 info: Syncing - 2 days left - 3.56 slots/s - finalized: 38048 0x56f1…c00e - head: 1217618 0xf015…7905 - clockSlot: 1821828 - peers: 8
Aug-11 14:46:17.310 []                 info: Syncing - 2 days left - 3.43 slots/s - finalized: 38049 0x6fd4…fecb - head: 1217636 0x94a0…138f - clockSlot: 1821829 - peers: 10

<--- Last few GCs --->

[1717966:0x362f180]   122575 ms: Scavenge 2028.8 (2040.6) -> 2027.9 (2040.6) MB, 10.2 / 3.8 ms  (average mu = 0.107, current mu = 0.016) allocation failure
[1717966:0x362f180]   122656 ms: Scavenge 2028.8 (2040.6) -> 2028.0 (2040.6) MB, 10.1 / 3.7 ms  (average mu = 0.107, current mu = 0.016) allocation failure
[1717966:0x362f180]   122729 ms: Scavenge 2028.9 (2040.6) -> 2028.0 (2040.6) MB, 10.0 / 3.7 ms  (average mu = 0.107, current mu = 0.016) allocation failure

<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x140a7f9]
Security context: 0x1829873c08d1 <JSObject>
    1: new constructor(aka Tree) [0x1879803bbb89] [/home/user/lodestar/node_modules/@chainsafe/persistent-merkle-tree/lib/tree.js:~13] [pc=0x3ab22a5e1dba](this=0x0dfa9e31b5a9 <Tree map = 0x2372219894e9>,0x0ad8d4252111 <Node map = 0x237221989499>,0x0dfa9e31b571 <JSFunction (sfi = 0x1ad02c97d7b9)>)
    2: ConstructFrame [pc: 0x138b8c0]
    3: getProperty [0x70c51b8...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0xa18150 node::Abort() [node]
 2: 0xa1855c node::OnFatalError(char const*, char const*) [node]
 3: 0xb9705e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
 4: 0xb973d9 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
 5: 0xd54655  [node]
 6: 0xd54ce6 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [node]
 7: 0xd615a5 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [node]
 8: 0xd62455 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
 9: 0xd63aff v8::internal::Heap::HandleGCRequest() [node]
10: 0xd12565 v8::internal::StackGuard::HandleInterrupts() [node]
11: 0x106dba6 v8::internal::Runtime_StackGuard(int, unsigned long*, v8::internal::Isolate*) [node]
12: 0x140a7f9  [node]

Shall I open a new ticket or shall we stick to this one?

@q9f recently we run the nodes with 6GB heap memory https://github.com/ChainSafe/lodestar/blob/0add9d11d619e8f096d5dbf54ec40a6cf7b14f8f/docker-compose.yml#L18
there's an ongoing effort to reduce memory now at https://github.com/ChainSafe/lodestar/issues/2885

Thanks, that works well! (for now)


[Help Wanted] Use as a CSV Editor?

Hi, thanks for the great plugin! Works as expected.
I have a small feature request that I think this plugin will be great for - can you shed some light on how this plugin might be used to modify usual CSV files, i.e. not for resource purposes? I think the plugin has all the components but I just need some guidance on how to repurpose it to just for editing CSV.
Essentially I am looking for something like this for Visual Studio Code: https://marketplace.visualstudio.com/items?itemName=janisdd.vscode-edit-csv
E.g.

Employee ID
Employee Year
Employee Department

001
2
Accounting

003
2
Finance

Thanks!

Hello! Interesting use case.
It's definitely already possible: Import/Export CSV -> Specify columns and datatypes (there are enums too) -> Confirm and edit.


Reduce STAmount public interface (RIPD-867)

Reviews: @scottschurr, @HowardHinnant, @vinniefalco

Less is more. Except for the price of XRP.

Brilliant cleanup :+1:

My comments are all just suggestions except for the removal of the unused comparison operators from STAmount.cpp.  Very nice job!

Looks great.  I'm a fan of @HowardHinnant's suggestion regarding the comparison operators.  But, to my eyes, everything looks fully functional as is.  :+1:.

I have implemented Howard's suggestion (please verify it's right!) and dried up the interface more. Reviewers: assemble!

:+1:

Looks swell to my eyes.
:+1:

I've squashed the commit down and am marking it as "Passed". Thanks for the careful review everyone!

6f5d8bba2d3be19535e5e20e95003cfe2d07f67c
5a7fa8cfa9edf880073cb37b97670e9ba4222787


The Rotate style functionality does not work when adding a  between Accordion.Item and Accordion.Header

Description of the bug
In Team-studio, we usean Accordion in a 'lage' page. We display the Accordion.header and NavigationMenu in one block by using a <div> that wraps both of them in flex style. However, this <div> prevents the rotate functionality from working, causing the accordion arrow to always point down. We are wondering if the design team can address this issue so that we can use a <div> between Accordion elements.
This video with <div>:
https://github.com/digdir/designsystem/assets/90609090/66cd3ffc-52dd-4a30-9d38-8fc8ae2a84dd
This one without <div>:
https://github.com/digdir/designsystem/assets/90609090/1d62594a-534b-4eef-8b62-77f7474310df
Here is link to our issue :
https://github.com/Altinn/altinn-studio/issues/11484
Steps To Reproduce

Go to lage page .
open and close the accodrion , notice that the arrow still pointed down.

Additional Information
No response

Thanks for reporting this!
Having a quick look, it looks like this is due to the css selector for flipping the chevron expecting elements in a specified order: https://github.com/digdir/designsystem/blob/5d4c040f511c5e7edcb099cde9c62e98dd11ec2f/packages/react/src/components/Accordion/Accordion.module.css#L146

Thanks for reporting this!
Having a quick look, it looks like this is due to the css selector for flipping the chevron expecting elements in a specified order:
https://github.com/digdir/designsystem/blob/5d4c040f511c5e7edcb099cde9c62e98dd11ec2f/packages/react/src/components/Accordion/Accordion.module.css#L146
The other selectors should also be revised to ensure they don't break if something is used like this.

I'll get on this issue. We can make another issue to go over other selectors that might need changing


SAC Details Changes

Your checklist for this pull request
🚨Please review the guidelines for contributing to this repository.

[ ] Make sure you are requesting to pull a topic/feature/bugfix branch (right side). Don't request your default branch!
[ ] Make sure you are making a pull request against the default branch (left side). Also you should start your branch off default branch.
[ ] Check the commit's or even all commits' message styles matches our requested structure.
[ ] Check your code additions will fail neither code linting checks nor unit test.
[ ] I have added necessary documentation (if appropriate)

Linked Issues
Fixes
Description
Please describe your pull request.
❤Thank you!
Post merge checklist

[ ] Follow steps from the guidelines for contributing to this repository.
[ ] If you are a new contributor, ping in the thread and one of the maintainers will add you to all-contributors list.

@dibendusahani please follow conventionalcommits while making commits. You can also refer to this video why good commit messages are important.


New Feature : Generate barcode view

Is it planned to add the feature to generate a barcode image view given data and barcode format ?
Thanks in advance !

@Spierki No this is not a planned feature. Sorry. The purpose of this library is only about barcode reading :)


Implement Artifact Store functionality in RDBS version

Implemente the common functionality in the RDBS version.

Fixed in #360


Rawhide builds not shown

For some reason only stable fedora branch builds are visible in "packit status" output in the koji build listing:
$ packit status
There are no builds for branch f10
There are no builds for branch f11
There are no builds for branch f12
There are no builds for branch f13
There are no builds for branch f14
There are no builds for branch f15
There are no builds for branch f16
There are no builds for branch f17
There are no builds for branch f18
There are no builds for branch f19
There are no builds for branch f20
There are no builds for branch f27-modularity
There are no builds for branch f7
There are no builds for branch f8
There are no builds for branch f9

Downstream PRs:
  ID  Title                                         URL
----  --------------------------------------------  -----------------------------------------------------------
   4  Anaconda dirinstall gating tests (text, vnc)  https://src.fedoraproject.org//rpms/anaconda/pull-request/4
   3  Add dirinstall gating tests (text, vnc)       https://src.fedoraproject.org//rpms/anaconda/pull-request/3

Dist-git versions:
f10: <IP_ADDRESS>
f11: <IP_ADDRESS>
f12: <IP_ADDRESS>
f13: <IP_ADDRESS>
f14: <IP_ADDRESS>
f15: <IP_ADDRESS>
f16: <IP_ADDRESS>
f17: <IP_ADDRESS>
f18: <IP_ADDRESS>
f19: <IP_ADDRESS>
f20: <IP_ADDRESS>
f21: <IP_ADDRESS>
f22: <IP_ADDRESS>
f23: <IP_ADDRESS>
f24: <IP_ADDRESS>
f25: <IP_ADDRESS>
f26: <IP_ADDRESS>
f27: <IP_ADDRESS>
f27-modularity: <IP_ADDRESS>
f28: <IP_ADDRESS>
f29: <IP_ADDRESS>
f30: <IP_ADDRESS>
f7: <IP_ADDRESS>
f8: <IP_ADDRESS>
f9: <IP_ADDRESS>
master: <IP_ADDRESS>

No upstream releases found.

Latest Bodhi updates:
Update                     Karma  status
-----------------------  -------  --------
anaconda-30.25.6-3.fc30        3  stable
anaconda-29.24.7-3.fc29        0  testing
anaconda-30.25.6-2.fc30        3  stable

Latest Koji builds:
f21: anaconda-21.48.22-1.fc21
f22: anaconda-22.20.13-1.fc22
f23: anaconda-23.19.10-1.fc23
f24: anaconda-24.13.7-1.fc24
f25: anaconda-25.20.9-1.fc25
f26: anaconda-26.21.12-1.fc26
f27: anaconda-27.20.4-6.fc27
f28: anaconda-28.22.11-2.fc28
f29: anaconda-29.24.7-2.fc29
f30: anaconda-30.25.6-3.fc30

Contents of .packit.yaml:
specfile_path: anaconda.spec.in
synced_files:
  - packit.yaml
  - anaconda.spec.in
upstream_project_name: anaconda
downstream_package_name: anaconda
current_version_command: ["sh", "-c", "git describe --tags | cut -d - -f 1-2 | cut -b 10-"]
current_tarball_command: ["make", "release"]

Upstream repo URL:
https://github.com/rhinstaller/anaconda

Fixed by #447


Improvement of XML merger: resolve before merge

from #403 we discovered in sprint-review that the following issue was not really addressed:

[ ] Resolve before Merging (in Load()): moving resolve into the load method

Details: The XML templates may contain variables ($[IDE_HOME], $[M2_REPO], etc.) that should be resolved when the workspace config is updated. Doing this after the merge has several flaws:

If the workspace file (and not the template) itself has some variable this would also be resolved what is undesired. If legacy support for variables is enabled, even things like ${JAVA_HOME} could be resolved accidentally if the IDE uses it inside XML config files in the workspace.
If the variable would be used in some attribute like <myelement id="$[JAVA_VERSION]"> in the template then on the first run of configurator the workspace is empty and this would be created and resolved. On the next run, the merger would not match <myelement id="$[JAVA_VERSION]"> together with the resolved one from the workspace such as <myelement id="21"> and therefore two elements would be created and after resolving the variable they would even be duplicated if the variable value did not change. This surely is an edge-case but still indicating why the current approach is somehow wrong.

As Salim told there where quite some tests breaking if this was implemented so be prepared to reserve some time for testing and analysis when implementing this story.

This issue slighly collides with #582. As a workaround we will use override as merge strategy in the XML templates that otherwise would be affected by this "bug". When this story is implemented, as a follow up cleanup we should change the override to combine and test if all is working as expected.


Add Nomic

Added initial chain config for Nomic. Still needs persistent_peers.

Thanks for this PR! :)
Merged for now, but could you please add key_algos, slip44, fees, more peers and explorer (if you have one) @mccallofthewild ? See the Osmosis file as a reference.


Notifier.notify() should get called when API route is hit

Context:
I have built an online collaboration tool. I would like a desktop notification to appear to the other users when a message is sent. I have placed the code below in my API call for posting a new message (as a test). I am testing with ubuntu.
 notifier.notify({         'title': 'My notification',         'message' : 'Hey, its nick'     });
When hitting the API route that would fire the notifier.notify() method nothing happens. I get no errors either so I'm not quite sure the issue. Also, I was under the impression I would have to ask the users for permission to send desktop notifications, but I don't see anything regarding permissions.
Any advice/ guidance is greatly appreciated. Thanks so much!
My macOS is 10.13.

Hi, @NicolasPerl ! This package is intended for desktop-apps/server-apps (like CLI tools, Electron apps, etc) and does not run in the browser. There is a built in standard API in the browser to do that, though 👌 Check out some examples and the docs on: https://developer.mozilla.org/en-US/docs/Web/API/notification


GoogleAuthorizationCodeRequestUrl approval_prompt:force and prompt:consent

After updating of nugets from 1.13.1 to 1.32.2 version Google OAuth not return refresh_token on request. I.e. always refresh_token=null
Our GoogleAuthorizationCodeRequestUrl properties in both versions (1.13.1 and 1.32.2):
AccessType=offline,
ApprovalPrompt=force.
Looks like approval_prompt no longer supported (replaced to prompt:consent): https://github.com/google/oauth2client/issues/453
Can you confirm that our issue related to not right request attribute (i.e. we should use prompt:consent)?
If yes, please propose the workaround with minor changes from our side. Maybe we can temporarily revert nugets to more earlier version than 1.32.2 but upper than 1.13.1?
Or you can make fix of GoogleAuthorizationCodeRequestUrl?
Thank in advance!

If it's about the nuget packages, this is the right project - but I know nothing about this particular area. Will assign to Chris to investigate :)

In that case i would love to see some code for this. I am not aware of any way of passing ApprovalPrompt to this library.   I believe i have seen a few comments on stack overflow about it over the years.
I always thought ApprovalPrompt  was used internally to prompt when the client requested offline access.

@jskeet @LindaLawton yes, it’s about .net nuget package.
As I know ApprovalPrompt parameter in GoogleAuthorizationRequestUrl not changed, i.e. it still “approval_prompt” source https://github.com/google/google-api-dotnet-client/blob/b26fd3effa79808c229d5bf8814777ecc90a7193/Src/Support/Google.Apis.Auth/OAuth2/Requests/GoogleAuthorizationCodeRequestUrl.cs
I provide source example soon.

@chrisdunelm @LindaLawton my sample, works fine in nuget 1.13.1 but not in 1.32.2, i.e. in 1.32.2 refreshToken always null.
    public class CustomAuthorizationCodeFlow : AuthorizationCodeFlow
    {
        public override AuthorizationCodeRequestUrl CreateAuthorizationCodeRequest(string requestUri)
        {
            public CustomAuthorizationCodeFlow(Initializer initializer) : base(initializer)
            {
            }

            return new GoogleAuthorizationCodeRequestUrl(new Uri(AuthorizationServerUrl))
            {
                ClientId = ClientSecrets.ClientId,
                Scope = string.Join(" ", Scopes),
                RedirectUri = redirectUri,
                AccessType = "offline",
                ApprovalPrompt = "force"
            };
        }
    }

    var flow = new CustomAuthorizationCodeFlow(initializer);
    var auth = new AuthorizationCodeWebApp(flow, redirectUri, state);
    var result = await auth.AuthorizeAsync(userId, taskCancellationToken);
    var refreshToken = result.Credential.Token.RefreshToken;

looks like ApprovalPrompt = "force" should be renamed to Prompt = "consent".

Nice find!

@LindaLawton Thank you for a quick investigation. @chrisdunelm Any chance that this issue being investigated/fixed on this week?

@mgkcortyw: I'm still confused about why you're seeing different behavior between 1.13.1 and 1.32.2. That file hasn't changed for a very long time. Can you just confirm that you really do see a difference between those two versions?
Yes, we may well need to deprecate the ApprovalPrompt property and introduce a new Prompt one due to server-side changes, but 1.13.1 still had ApprovalPrompt sending the approval_prompt query parameter, so I wouldn't expect that to have changed anything.

@jskeet i think real problem in changed api urls https://github.com/google/google-api-dotnet-client/blame/364ef9d891996d36f49db3764486806a8981a41a/Src/Support/Google.Apis.Auth/OAuth2/GoogleAuthConsts.cs

Well there are new constants, all to do with OpenID, but none of the existing constants have changed.
Constants in 1.13.1: https://raw.githubusercontent.com/google/google-api-dotnet-client/v1.13.1/Src/Support/GoogleApis.Auth/OAuth2/GoogleConsts.cs
Constants in 1.32.1: https://raw.githubusercontent.com/google/google-api-dotnet-client/v1.32.2/Src/Support/Google.Apis.Auth/OAuth2/GoogleAuthConsts.cs
Again, could you check that you really see a difference in behavior between 1.13.1 and 1.32.2? Diagnosing "something that needs to change due to a server change" and "something that needs reverting to earlier code" are fairly different, and I can't currently see any change since 1.13.1 that would explain this.

I think GoogleAuthorizationCodeFlow changed from using GoogleAuthConsts.AuthorizationUrl to use GoogleAuthConsts.OidcAuthorizationUrl in PR#1003 back in May 2017.
https://github.com/google/oauth2client/issues/453 suggests this change of auth URL causes the problem.
I'll do some more work to confirm this is the case.

@LindaLawton That's probably #916. We updated to the latest URLs, but don't read from the openid-configuration.

@jskeet you are right, it’s not GoogleConsts.cs issue, but related to urls. @chrisdunelm hit the target! Thanks.
We are use GoogleAuthorizationCodeFlow.Initialize() method and pass it into CustomAuthorizationCodeFlow constructor in my code listing above.

I'll create a PR to add a Prompt property to the GoogleAuthorizationCodeRequestUrl class; and to mark the existing ApprovalPrompt property as obsolete.
This will allow you to set prompt=consent as required to retrieve a refresh token.

@chrisdunelm Thank you! It’s great?
Could you release new version of nugets after that?


fix: Remove properties, options and configurators aggregations if no filterable properties are found

1. Why is this change necessary?
Currently the properties, options and configurators aggregations are only removed if ids are found. This results into the filter panel being rendered even if empty as there still exist aggregations:
https://github.com/shopware/platform/blob/8b1d7518ef1833212a38fe939b87ab98420cbe27/src/Storefront/Resources/views/storefront/element/cms-element-sidebar-filter.html.twig#L10
2. What does this change do, exactly?
Always remove the properties, options and configurators aggregations and only add the properties aggregation if related properties are found.
3. Describe each step to reproduce the issue or behaviour.
Create a category, assign one product without any properties, assign a layout which has all filters disabled, except the property filters, the filter panel is still rendered, and on mobile one sees a filter button.
4. Please link to the relevant issues (if any).
-
5. Checklist

[x] I have written tests and verified that they fail without my change
[x] I have created a changelog file with all necessary information about my changes
[ ] I have written or adjusted the documentation according to my changes
[x] This change has comments for package types, values, functions, and non-obvious lines of code
[x] I have read the contribution requirements and fulfil them.

6. Additional commits
I additionally pushed two more commits, one is fixing the phpstan issues of the ProductListingFeatureSubscriber and the other is fixing an issue with the tests, where a wrong array shape was passed.
I noticed this as I looked into the phpstan problems of the test. But I currently have no time to add the array shapes of the data providers :-) Maybe one wants to ignore these altogether, as described here: https://github.com/phpstan/phpstan-phpunit/issues/63#issuecomment-592515871

Hello,
thank you for creating this pull request.
I have opened an issue on our Issue Tracker for you. See the issue link: https://issues.shopware.com/issues/NEXT-22970
Please use this issue to track the state of your pull request.

Hello @aragon999,
our quality assurance has found out that this behavior also occurs with other filters like 'Rating'. Maybe we can find a general solution that solves the problem for all filters.

@SimonVorgers what exactly do you mean, that the filters are shown if no rating exists for the products of this category?

@SimonVorgers I have looked into this issue and understood the problem.
To solve it each of the other filters needs filtering, but I think the exact filtering depends on the filter itself, for example the rating should be removed if max is null:

I will look into this further, but not sure when I will be able to fix this.

@SimonVorgers I added a method which filters the empty aggregations. Unfortunately I currently have no time to look into the failing ES test.
The test is failing since the rating aggregation is missing. But not sure how to properly fix that. Could you look into that?

@SimonVorgers I now fixed all tests, rebased to the current trunk and also found another issue which happend in the search, that the filter were still displayed. I therefore extended the filter procedure.
Could please you review the changes? :-)

@SimonVorgers how is the status here?

@SimonVorgers @keulinho @trandangtri what is the status here? It is quite annoying to have to wait always for such a long time, without any feedback.
There was only one feedback (over a month ago), but since than nothing. By now the phpstan issues of this PR have already been implemented, which now cause a merge conflicts of this PR.
As this is for a customer, I also need to carry around this changes as a patch and apply it to a project. I can now refactor this PR again, to take the phpstan changes of trunk into account, but I am not willing to do it a second time.

Hi @aragon999
we have re-imported this pull request and are reviewing it with our QA.
Also we are taking care of some additional things like conflicts / fixing some phpstan issues that occur in our gitlab pipeline :)

@SimonVorgers thank you :-)

Hi @aragon999
sorry for writing so late (again). Your PR was already in the merge train, but failed. We noticed that your fix is not compatible with the option Disable filter options without results under product settings. I had already looked at the problem but didn`t find a nice solution for it.. :disappointed:

To be honest, this PR becomes really annoying. In particular since I do not know how often I rebased, than some things have already been implemented, which this PR included, e.g., the fix for the tests: https://github.com/shopware/platform/commit/b43f8877752b3950ce6aad6c22164783386514ce#diff-05caeea29d8f9461d5aeba86e15cf9714c2c6b728deae62d464804397572f5a4
In future it would be for example nice, if you cannot merge the full PR, that you cherry pick the parts, which can be merged... In this case it could have been the fix for the tests, or the PHPStan issues. But otherwise rebasing and than noticing that it has already been fixed otherwise is annoying... I know this is probably some more work on your side, but otherwise being a contributor is not fun. And in the meantime you on your side have to again invest the time to fix an already fixed issue (if you do not know that this fix exists somewhere).
If for example some parts should than be put into a separate PR, fine, I can do that, just let me know... -.-
Taking so much time between answers is also not helping, as the core changes a lot, I mean the PR was opened in August and will now only land significant time later, probably only in the next major release.
Now coming back to the listing, after rebasing and cleaning up, I tried to reproduce the "issue". But to be honest I have no idea what you mean. For me everything works, as intended, i.e. with the configuration "Disable filter options without results" enabled, for example the "shoe-size" selection is disabled if only filters are selected, which do not have a corresponding "shoe-size":

So could you please elaborate a bit more on this, what exactly does not work in which configuration?

At the moment the e2e test Should disable free shipping filter in tests/e2e/cypress/e2e/storefront/listing/disabled-filter-combinations.cy.js is failing.
It fails because it cant satisfy that the shipping filter is disabled.

I will close this for now due to inactivity. Whenever you get around to fix the issue, feel free to reopen the PR.


Route switch stops working when an  sibling component reads a resource

Describe the bug
I came across this strange bug. Consider a route that renders like this,
        <div class="items-left">
            <ThingsNav/>
            <Outlet/>
        </div>

Now suppose <ThingsNav /> reads a resource using create_resource (which the sub-route also does), then none of the link clicks on that nav work at all. To workaround this bug, you have to move the <ThingsNav/> component to inside of each sub route components.
(See the linked PR below for more details)
Leptos Dependencies
Using 0.4.9 (exact version are in Cargo.lock in the PR linked below),
leptos = { version = "0.4", features = ["serde"] }
leptos_axum = { version = "0.4", optional = true }
leptos_meta = { version = "0.4" }
leptos_router = { version = "0.4" }

To Reproduce
Detailed repro steps here: https://github.com/srid/leptos-fullstack/pull/19
Expected behavior
See https://github.com/srid/leptos-fullstack/pull/19
Screenshots
See https://github.com/srid/leptos-fullstack/pull/19
Additional context
Discord link: https://discord.com/channels/1031524867910148188/1143326756334489651

This is a hydration issue — it fails silently but hydrates the wrong elements. You can see this if you open the DOM inspector, look for the <!--<ThingsNav>--> marker comment, and then try to find the <!--<Outlet>-->, which doesn't appear. Exploring some of the DOM around there shows some unhydrated marker nodes.
Either of two solutions works:

replacing the SuspenseWithErrorHandling with Suspense
upgrading to 0.5.0-beta, which significantly improves some of the Suspense/ErrorBoundary hydration issues


Handle ignorePending differently

Previously, individual templates were always marked as "Pending" if they
had an unsatisfied dependency. The ignorePending field was only
considered when calculating the overall compliance of a Policy. But that
meant that it was not possible to correctly calculate the overall state
with just the status object, which is confusing.
Now, the ignorePending field is considered when marking the compliance
of each template. This makes how the overall compliance is calculated
more clear, and the new messages may be more helpful for users.
Refs:

https://issues.redhat.com/browse/ACM-2101

Looks good! Thanks for the changes @JustinKuli


Make build_python work on mac (tested on a BigSur)

Problem
scripts/build_python.sh does not work on mac.
Summary of Changes
Update the script to work on mac:

document requiring brew install coreutils
add BigSur define to avoid whl install errors
add validation that virtualenv is available as a module and pip-install it if it is not (on mac virtualenv does not seem available, even though pigweed is already virtualenv)

Here's a version without needing coreutils:
https://github.com/project-chip/connectedhomeip/pull/5194

closing after #5194


Can't load Material Design font because of missing missing glyph

Try to load the font file called MaterialIcons-Regular.ttf located in this giant zip.
Get error Failed to find replacement characters '◻' or '?' that originates from epaint-0.23.0/src/text/font.rs:364:17.

This is the code that I used to load the font file:
        let mut fonts = FontDefinitions::default();
        fonts.font_data.insert(
            Self::FONT_ICONS.to_owned(),
            FontData::from_static(include_bytes!(
                "../../../res/fonts/material-design-4.0/MaterialIcons-Regular.ttf"
            )),
        );

        fonts
            .families
            .entry(FontFamily::Name(Self::FONT_ICONS.into()))
            .or_default()
            .insert(0, Self::FONT_ICONS.to_owned());

and this is an excerpt from the egui/epaint font.rs file:
        const PRIMARY_REPLACEMENT_CHAR: char = '◻'; // white medium square
        const FALLBACK_REPLACEMENT_CHAR: char = '?'; // fallback for the fallback

        let replacement_glyph = slf
            .glyph_info_no_cache_or_fallback(PRIMARY_REPLACEMENT_CHAR)
            .or_else(|| slf.glyph_info_no_cache_or_fallback(FALLBACK_REPLACEMENT_CHAR))
            .unwrap_or_else(|| {
                panic!(
                    "Failed to find replacement characters {PRIMARY_REPLACEMENT_CHAR:?} or {FALLBACK_REPLACEMENT_CHAR:?}"
                )
            });

I think that this font family represents an edge case, because it's icons rather than letters/numbers/symbols, which means that nobody will ever use this font without knowing exactly which code points to use, which means that the concept of a replacement glyph shouldn't apply.
Clearly, it would have been better if the authors of this font file had included a glyph for the white medium square, but since they didn't, and I'd like to use the font file as-is, I don't have a good workaround at this point.

See discussion in https://github.com/google/material-design-icons/issues/1600. It looks like glyph #0 is called .notdef and is supposed to serve as the glyph for missing glyphs. I'll see if I can update epaint to know about this.

I tried reading the code, and it looks like ab_glyph::GlyphId(0) has a special meaning in epaint to get substituted with the replacement character. I'm not sure whether ab_glyph::GlyphId is the same as a "glyph ID" in the TTF spec, so I don't know whether ab_glyph::GlyphId(0) means glyph ID 0. But it seems like if the code that calculated replacement_glyph were replaced with something that just asked ab_glyph for information about Glyph ID 0, then it would give the same functionality, but properly using .notdef rather than heuristically identifying the glyph to render when there is no glyph to render.

I've attached MaterialIcons-Regular.ttf.zip so you don't have to download that big zip to repro the bug.

Same behavior with MaterialSymbolsOutlined[FILL,GRAD,opsz,wght].ttf.

I stumbled on the same bug, well, not necessiarly a bug, anyway.
If you just want to "skip" this bug, here's my solution:

Watch a series of tutorial of FontForge on YouTube, take about half hour.
Open your icon font with FontForge.
Put whatever glyph/vector/thingy into position 0x3F which is the ? character should be, as long as the slot isn't empty. For example the > is empty but ? isn't:

From menu bar go to File -> Generate Fonts, then save it to somewhere.
Use the newly generated font in your code.
Profit.

Closed by https://github.com/emilk/egui/pull/4542


Add way to post today's entry directly to a chat client

Not sure what this would entail, but it'd be cool if there was an option for sending today's entry directly to a chat channel.

This might get the 1.0.0 label, but I'm just not sure the level of effort, and don't think it's necessary for a 1.0 release. This could change though.

Slack


refactor Instructor Embedders to be compatible with the new Document dataclass

replace Document.text with Document.content
replace Document.metadata with Document.meta

fixed by #44


Keep contributor information up-to-date automatically

problem
Embedded author information becomes stale.
There is a temporary workaround with a command to update this metadata periodically, but this is far from ideal.
Other related entities (organizations, projects) suffer the exact same problem.
proposed solution
Only store the id of related objects. On retrieval from the repository, the missing metadata is filled in. Since we have an in memory cache for these objects, performance shouldn't be too badly affected.
Contributor with id at rest:
{"person_id": "123"}

Contributor with id after retrieval:
{"person_id": "123", "person": {"first_name":  "...",  "orcid": "123", ...}}

An external contributor has the same metadata rest as after retrieval:
{"external_person": {"first_name":  "...", ...}}

This refactor allows for future new contributor types:
{"external_organization": {"name": "...",  ...}}

related issues

#1067
#1037
#1036
#480
#455

progress
work is happening here: https://github.com/ugent-library/biblio-backoffice/tree/external-relations

merged into dev


Updated Switch docs to 0.57

Switch docs didn't reflected deprecations and new props for 0.57.

Somehow the preview doesn't reflect the switch.md changes, I don't know if that affects the PR.


Support for Groth16 should be added.

Only GM17 is currently supported. To cut gas expenditures, we should add G16 support.
If we switch to G16, the proofs will be different (within the ZoKrates .code files will need to be changed to avoid the malleability security consideration of G16). As a result, I believe we will only be able to support G16 and not both G16 and GM17.
Ideas and suggestions are appreciated.

When using G16, developers should keep in mind that an attacker who sees a good proof can quickly create an alternative but still valid proof. As a result, depending on the use case, ensuring that the same evidence cannot be presented twice on the chain may not be sufficient to prevent attackers from replaying proofs. The following mechanisms can be used to address this problem:
The use of an ethereum address as a public input to the program, as well as non-malleable algorithms like GM17, are all examples of signed proofs nullifiers.

When we add batching/relayering functionality, it adds another layer of complexity. Malleability checks based on msg.sender don't always work when transmitting a proof via a relayer (since the relayer is msg.sender). The relayer could make changes to their 'customer's' proof and resubmit it.
Precaution is required.

So, to avoid malleability, we don't want the transaction to be changed by a third party, resubmitted numerous times, or submitted by someone who wasn't supposed to submit it (e.g. sniping transactions from the mempool to get the relayers compensation). The problems are the same for batches of transactions and relayed transactions, and they may be solved in the same way: The evidence is bound to all necessary criteria.
You don't assume that msg.sender is permitted to receive payments for relayed transactions, and you only want the transaction to be submitted by a specified address. The smart contract provides msg.sender as an input to the proof, and the user provides the withdraw address - the relayer cannot change the withdraw address, and no one else may submit the transaction without invalidating the proof.
The same holds true for batches of transactions: msg.sender is supplied as an input to the batch, and the batch forces it as an input to all proofs. This means that no one other than msg.sender can submit the batch, and the batch cannot contain any proofs that aren't connected to that sender.
However, a third issue that affects relayers and batches is temporal latency. You don't want a relayer to keep a transaction for a week and then submit it; you want an assurance that the transaction will be included within N blocks or not at all. You can accomplish this by including the maximum block height as a proof parameter, and the smart contract will reject those whose maximum is smaller than the current block.
You'd also like an assurance that the transaction will be included in the next batch, rather than any subsequent batch, when dealing with batches of transactions. You may do this via a hash chain, where all the public parameters from the previous batch are hashed to produce the next 'batch ID hash,' and then proofs generated for a certain batcher are connected to that, allowing them to only be included in that batcher's next batch.
The advantage of using msg.sender for anti-malleability is that it can be enforced by a smart contract without the need for any additional circuit logic to verify signatures. It implies that the arguments to the proof have been signed by the owner of that key - the smart contract then forces that as an input to the proof - then all of the proof's inputs are hashed and passed as a single public input - meaning


Cannot build gyro, no member named 'source' in struct 'std.build.Pkg'

when building gyro, I get the following error message:
zig build -Drelease-safe
./deps.zig:16:9: error: no member named 'source' in struct 'std.build.Pkg'
        .source = FileSource{
        ^

as per the readme:

Gyro follows the master branch of the zig compiler

This means you need to compile using zig master which is what I think you're running into. You can get the version of your zig compiler by running zig version. Alternatively you may download binaries from the releases.
The other case you might be running into is that there was a break in the stdlib, and my nightly CI run has not picked up on it yet.

closing as it's been two weeks and has been addressed


#438 Optimize hive table existence query

Closes #438
Added option:
hive {
  #...
  optimize.exist.query = true
}

to any Hive context. If

false (default = previous behavior), uses this query for checking Hive table existence:
SELECT 1 FROM my_db.my_table WHERE 0 = 1
true, uses this query for checking Hive table existence:
DESCRIBE my_db.my_table
(this is faster since it never touhces data, but may depend on Hive dialect)

In addition, added more logging related to Hive table existence check for easier debugging Hive issues.

Yes, it is a good suggestion, especially since Pramen already supports templates for creating and updating tables.
However, existence of Hive tables is a tiny bit of functionality I don't want users to care. So hard coded for now.
The reason I left the original (slow) behavior by default, is to make sure I don't break existing pipelines accidentally.
But if we encounter more issues regarding this, I'm indeed going to generalize the solution to make it more flexible, possibly with templates.