Use deterministic hashes to reference assets inside themself

Description
There are some specific cases where it is needed to reference. Let's take for example React server components.
Client components needs to be transformed on the server like the following:
import { transformSource } from "react-server-dom-esm/node-loader";

// ...

    async transform(code, id) {
      if (this.environment.name !== 'server') return;

      const context = {
        format: "module",
        url: id,
      };

      let { source } = await transformSource(
        code,
        context,
        (source: string) => ({ source }),
      );

      return source;
    }

The above example will transform the component into a registerClientReference call with the given url (in our case the module id). Later on, when the client will ask for the React server stream, it will use that url to fetch the client component using the provided url.
Obviously this can only work in dev mode.
Suggested solution
The goal would be to have a deterministic way to generate assets hashes using a randomly generated build id shared across environments allowing uage of getFileName() before the asset is actually emitted.
    async transform(code, id) {
      if (this.environment.name !== 'server') return;

      const url = this.environment.mode === 'dev' ? id : this.getFileName(id)

      const context = {
        format: "module",
        url,
      };

      let { source } = await transformSource(
        code,
        context,
        (source: string) => ({ source }),
      );

      return source;
    }

Alternative
There are in fact many alternatives available to us but they all require to make compromises.
Change the behavior of the third party libraries
This one does not ask much gymnastic. We can simply change the behavior on how things work and call it a day. In this case, it is possible to generate our own custom hashes used as the urls instead of module ids. Generate a manifest file that can be used by the client to determine what is the correct url to fetch the client components.
Run the "build" twice
This one looks a bit far-fetched but it does not require to change the behavior imagined by the authors of the library (in this case the React team). Making it easier to follow new releases.
The idea behind running the build twice is to first scan the sources that have been transformed, store there assets references that are then used as the url in the second build.
Additional context
No response
Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

I'm not sure if I get it, but I'll answer assuming that you want to know the file name of the chunk that will contain the module during the transform hook.
I think that's not possible. Which files are contained in which chunks will not be known until after the chunking is finished (which is after the transform hook is run). Even if the hash is determined based on something else that can be known before the transform hook, the name part of the chunk file name is not stable. So it's not possible to know the chunk file name in the transform hook.

I'm not sure if I get it, but I'll answer assuming that you want to know the file name of the chunk that will contain the module during the transform hook.

It is exactly what I am looking for. After digging in how chunking works, I think you are completely right and it is not a simple enhancement but would require to completely change the chunking behavior to provide a solution to a problem that has workarounds.

would require to completely change the chunking behavior

I don't think it's feasible even if the chunking behavior is changed.
But anyways, this feature needs to be implemented on rollup / rolldown level rather than in Vite, so please create an issue there.


Add support for dynamically updating partition sealed state.

Partition's sealed state is dependent on the sealed states of its
replicas (which is controlled individually by the respective nodes).
A partition is READ_WRITE if all its replicas are READ_WRITE. Equivalently,
a partition is READ_ONLY if any of its replicas is READ_ONLY. See the
design document for details.
Individual instances (or admin operations) are supposed to update the
InstanceConfig to mark their replicas as READ_ONLY or READ_WRITE (this is not
part of this patch). When this change is made, HelixClusterManager updates the
states of the replicas as part of the InstanceConfig change handling. Changes to
replica sealed state is tracked at the highest level and a getPartitionState()
call on a Partition will refresh and cache its state from those of its replicas
if there has been such a change since the last refresh.

@vgkholla @nsivabalan Addressed comments.

@vgkholla updated, please take a look.


"Invalid server URL. Please enter a valid URL." Error

#Hi
I'm having trouble with using my own storage space, found here: https://etherealandco.b-cdn.net/obsidian (it 404's because there are no direct files to link to), I'm not sure if I've just set this up wrong, or misunderstood what exactly I'm meant to do. The provider I'm using is bunny.net with a google login, do I need to provide any other information before I can get some help? Thank you so much for this plug-in!

Secondary path? As far as I'm aware (sorry I'm kind of new to using URLs) the link I'm using is the direct path link to my cdn & storage

The backend service address can only be a protocol plus a domain name:
https://www.xxx.com
You cannot use a secondary path, and cannot be like this:
https://www.xxx.com/xxx

Thanks so much! Fixed that so now it just ends in .net, my struggle is using the login? I put a username & a password but it says "Load failed", I read the information and it says it'll provide a password if there is none, but it's asking for a password or a valid username no matter what I do, if I'm understanding right I need to use my bunny.net login? But I'm using google login & an email isn't a valid username, so sorry to keep bothering you! And thanks in advance!

The username and password are automatically created. If a username has not been logged in before, it will be automatically created. For example, if you enter username: aaaaa and password: bbbbb, the account aaaaa will be automatically created. If an error message is prompted during login, there should be a problem with your backend service. Please verify whether your backend service is normal.

There's no problem with my service that I can see, every time I press the login button my service gives a status 404 for the path "... .net/robots.txt" (which doesn't exist on my service), but the link I gave is only to "... .net" with nothing in the end? (The only built in errors I should be seeing on my service are 403's or 405's for manually blocked root path access to my files or blocked POST requests respectively)

The plugin will not access robots.txt when logging in, it should be accessed by other clients
You can use the following command to verify whether your backend service is normal, use cmd under Windows, or iTerm under MacOS to execute the following command:
curl -X POST https://share.alter.run/api/user/login
Return result:

Sorry for responding a few days late, I was waiting on my laptop to arrive before I could try this; I'm getting a 400 bad request error, not sure why, I'm going to assume it's something bunny.net itself is doing, I'll continue to investigate everything myself and hopefully I'll find something lmao
Thank you so much for all your help!! I'll close this since it seems to be an error on my side alone 😆


Support for draw eraser, draw pen width & text rotate

Wondering if you do have any plans to support draw eraser, draw pen width and make the text rotatable?

You can use two-finger pinch and twist to scale or rotate text/image stickers.
There are no plans to support erasers and select pen widths at this time.


Resolving vulnerable dependencies in td.vue, as picked up by Trivy

Summary
The latest push to main flagged a vulnerable library in our production image.
Description for the changelog
Removes vulnerable versions of node-fetch and follow-redirects
Other info
We should consider running trivy on PRs as well (if it's not already).  It could also have been a timing issue where the vulnerabilities were disclosed between when the PR was open and when it was merged.  I will look into this later.

The CI pipeline isn't running on PRs... Taking a look!  I'll also take a swing at the browserstack nightly pipeline that failed today as well

Codecov Report

Merging #379 (5822e73) into main (659dd05) will decrease coverage by 0.33%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main     #379      +/-   ##
==========================================
- Coverage   97.85%   97.51%   -0.34%
==========================================
  Files         125       27      -98
  Lines        1768      483    -1285
  Branches      113        0     -113
==========================================
- Hits         1730      471    -1259
+ Misses         31       12      -19
+ Partials        7        0       -7

Impacted Files
Coverage Δ

src/service/threatmodelApi.js

src/components/LocaleSelect.vue

src/service/migration/data.js

src/store/actions/repository.js

src/components/report/Coversheet.vue

src/service/provider/github.provider.js

src/store/actions/loader.js

src/service/x6/shapes/store.js

src/service/x6/graph/graph.js

src/service/migration/nodes.js

... and 88 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 659dd05...5822e73. Read the comment docs.

Trivy is still unhappy, I may have jumped the gun requesting review.  Looking a bit deeper!

We just need to fix the Trivy Scan prolly need to update a few more deps! Lgtm

Same dep was in td.server (which already had an override for 1.14.7, same as td.vue did) :face_exhaling:
I'm building and testing locally first, then will push the update :sweat_smile:


Supporting loading the subtitles from remote server

Castnow would become more perfect if it can load the subtitles from remote servers.
I have tested that remote .vtt subtitle files can be successfully loaded, but .srt subtitle files can't.
Please consider adding this feature, thanks.

I decided to auto download the files and provide castnow with local path
thanks


[proxy] dynamically mount -H dirs into the proxy when launching

This lets you use -H unix://*.sock outside /var/run/weave.
The giant gotcha is that if you use -H /var/run/foo.sock, you will reintroduce the bug with
namespaces not being GCed. We could make passing a socket in /var/run, an error...
Still working on the test...
Fixes #1579

We could make passing a socket in /var/run, an error

warning.


[Bug] After loading a task bundle, every included task is automatically run

As the title says. How could this behaviour be avoided?
I mean, when I load a task bundle, I would like to keep those tasks in a stopped state by default.

I've added an option to the API method that will allow you to disable the autostart. If you're using the OverseerLoadBundle command, you can get this behavior by adding using a bang: OverseerLoadBundle!

Great work 👍.
I wonder if there is a way to enable autostart only for certain tasks when loading the bundle.
Maybe by setting an option in the register_template struct?

I think this is getting a little too specific. If you want some tasks to autostart and others to not, your best bet is to load the bundle with autostart = false and then iterate through the tasks yourself calling start on the ones you want started. If you want to designate it when you register the template, try adding a flag to the metadata of the task that you can read later.

If you want to designate it when you register the template, try adding a flag to the metadata of the task that you can read later.

I've not understood that bit, could you kindly clarify, please 🙂.

When you register the template you can make it create a task with some specific metadata
overseer.register_template({
  name = "Some Task",
  builder = function(params)
    return {
      cmd = {'echo'},
      args = {"hello", "world"},
      metadata = {
        autostart_when_loaded = true,
      },
    }
  end,
})

Then after you load a bundle you could run a function like this
for _, task in ipairs(overseer.list_tasks({status = overseer.STATUS.PENDING})) do
  if task.metadata.autostart_when_loaded then
    task:start()
  end
end


Added support for adding PGVector (Vector database for Postgres)

Added support for adding PGVector (Vector database for Postgres)
PGVector
PGVector Python
The integration test can be run using
poetry run pytest tests/integration_tests/vectorstores/test_pgvector.py

Make sure you setup a postgres database with pgvector extension installed PGVector Installation Steps before running the integration tests.

@hwchase17 I had to add
pgvector = {version = "^0.1.6", optional = true}
psycopg2-binary = {version = "^2.9.5", optional = true}

as optional dependencies to pyproject.toml, which led to change in the poetry.lock on running
poetry lock --no-update

causing the conflict. Should i remove the dependencies and revert these two files to the master branch or how should i resolve them?

@hwchase17 I had to add
pgvector = {version = "^0.1.6", optional = true}
psycopg2-binary = {version = "^2.9.5", optional = true}

as optional dependencies to pyproject.toml, which led to change in the poetry.lock on running
poetry lock --no-update

causing the conflict. Should i remove the dependencies and revert these two files to the master branch or how should i resolve them? I don't have much experience with poetry, so can you please guide here?

i can work to resolve!

will take a look at imports tmrw if i ahve time!

Thanks for the merge! Any timeline on when it will be available in the PyPI release?


Change the default value for warnings on element type changes to 'false'

This change reflects our growing uncertainty about this being a valid warning in the first place.
Checklist before submitting:

[x] Added entry to CHANGELOG.md
Added/updated relevant tests
Added/updated documentation

Coverage remained the same at 90.084% when pulling 077f4f7a8007c276a87d2237a8db0a718555bd53 on ZoteTheMighty:warn-on-type-change-default-off into 96c0f8efcfba277a98437596590576bf300bfe5e on Roblox:master.


The internationalize works only in localhost

The example project with internatilanize works only in localhost, but in cloud server works only in local language (spanish), i use CentOS7 with apache as reverse proxy and run directely with java -jar.

Hey @WHK102 - what do you mean exactly? The functionality should be entirely unrelated to the way you deploy the application.


Support of FBO readPixels

Hello,
On AsmJS platform, i need to perform a render to texture and then read back pixels.
bgfx::readTexture does not work on AsmJS because read back texture is not supported as glGetTexImage does not exists.
The solution is to use glReadPixels on the FBO used to perform the render to texture.
I can not find any bgfx::readFrameBuffer method.
Did i missed something ?
Regards,
Harold

It's not implemented because glReadPixels is slow for real-time use. There could be some way to optimize it, if there are multiple FBOs and read is not used against one that was used for the writing in the same frame, but I haven't tried it.

In my case, i should use glReadPixels to get pixels of one-shot RTT. The end-user does not car about its screenshot request freezing the rendering for some milliseconds.
Maybe it should be nice to support it with a big performance warning in the documentation ? :)

@hcomere did you find a solution for that? I'm trying to implement RenderTexture as well and have no luck yet.

Yes and no, i have patched bgfx with readPixels functionality but i cannot make a pull request as it is only implemented for renderer_gl.
It works but atm i have some problems with FBO depth component on asmjs.
I am actually working on that but still hope @bkaradzic will do it properly soon :)

Hi @hcomere ...
can you please share your bgfx patch for readPixels ? My requirement is only for openGL. I tried using blit to read pixels but that is working slow for me.

Hi @Raki,
Sorry but i have no patch to perform directly a readPixels call.
I had splitted my issue into 2 parts :

Patch for bgfx::readTexture allowing it to work even if glGetTexImage is not supported. This patch had been integrated in bgfx master.
Requirement to read screen back buffer to perform screenshots.

The second part required too much changes in bgfx.
I solved this part by rendering the same scene in a texture then reading this texture through bgfx::readTexture.

@hcomere Thank you so much for the quick reply. I understood how you achieved it. I'm trying to port my texture painting code to bgfx, where I will read pixel color below my mouse coords using readPixels. I will try to find if any alternative is there.


[Bug]: Model does not split in multiple Gpus instead it occupy same memory on each GPU

Your current environment

The output of `python collect_env.py`
Vllm version : 0.5.5
Nccl=2.20.5
Gpu : Telsa V100-sxm2-32GB
Cuda version : 12.6
Driver version : 560.28.03

Model Input Dumps
No response
🐛 Describe the bug
I have a node with 4 GPUs, each having 32 GB of memory. I’m loading the granite8B-code-instruct model with float16 precision, which requires approximately 16 GB of memory (8B parameters × 2 bytes). Using a tensor parallel size of 2, I plan to split the model across 2 GPUs for more efficient and faster serving.
My expectation is that each GPU should load 8Gb,
But Actual , it is loading 27GB on each gpu .
The nvidia-smi commands shows
GPU           Memory-usage
0             28333MiB/32768MiB
1              28222MiB/ 32768MiB
Code :
vllm serve granite-8B-code-instruct --tensor-parallel-size 2 --dtype float16
Before submitting a new issue...

[X] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

By default, vLLM will use 90% GPU VRAM regardless of model size (the extra memory is used for KV cache). If you want to use less memory, please specify the --gpu-memory-utilization argument.

By default, vLLM will use 90% GPU VRAM regardless of model size (the extra memory is used for KV cache). If you want to use less memory, please specify the --gpu-memory-utilization argument.

for me, still gives memory error. trying to split 40 gig model on 4 gpus eash with 40 gigs of vram.

Can you show the command used and the stack trace?


Add ability to pass a custom style

Fairly self-evident. Project of mine required inline styling directly on the svg element.

This looks good to me, what's stopping it from going in?

I tried to merge it manually because it had conflicts but apparently didn't work.
Added in 332b4a6579c898e478a3c27445276d01207be185


Librespot not shown on network and connecting with credentials does not work. Where to start looking?

I am using a librespot plugin in Kodi/LibreELEC and cannot get it to work. The person who made the plugin also has no clue what is happening. The thread can be found here: https://forum.libreelec.tv/thread/8438-librespot-addon/?pageNo=35
I was wondering what I can do to trace the root of the problem.
What would cause librespot not to be shown on the network (discovery mode on)?
When discovery mode is off, which credentials should be used (what is "username" in this case)?
Where to look when putting credentials also does not lead to librespot showing in the devices?

You'll probably have to run librespot directly on its own, because the problem might not even be with librespot.  If you get the same problem running it directly then you'll have eliminated the plugin and other media player as causing the problem, and you'll also be able to post full librespot debugging logs.


Enable CliRunner to echo output to stdout/stderr

I'm driving my CliRunner testing from pytest. pytest will capture stdout/stderr by default, and display them when an assertion fails. As CliRunner captures all of the output itself, I don't get any guidance as to what my application has done on failure, without adding explicit debugging code to each assertion.
I'd like to be able to do something like CliRunner(echo=True) so that I get the output of its run both in its attributes and actually written to stdout/stderr.

Would also love to see this.
I haven't dug deep into the internals for CliRunner.isolation() here, so I'm probably not doing this in the most correct way, but here's my workaround for anyone else interested:
import functools

import pytest

@pytest.fixture
def cli():
    """Yield a click.testing.CliRunner to invoke the CLI."""
    class_ = click.testing.CliRunner

    def invoke_wrapper(f):
        """Augment CliRunner.invoke to emit its output to stdout.

        This enables pytest to show the output in its logs on test
        failures.

        """
        @functools.wraps(f)
        def wrapper(*args, **kwargs):
            echo = kwargs.pop('echo', False)
            result = f(*args, **kwargs)

            if echo is True:
                sys.stdout.write(result.output)

            return result

        return wrapper

    class_.invoke = invoke_wrapper(class_.invoke)
    cli_runner = class_()

    yield cli_runner

...

def test_basic(cli, package):
    result = cli.invoke(package, ['--help])
    assert result.exit_code == 0

This feature would be very useful if available directly in click.

I also came up against this. For the record my solution was the following helper method:
def invoke(cmd, expect_exit=0, catch_exceptions=False, **kwargs):
    runner = click.testing.CliRunner()
    result = runner.invoke(cli, cmd, catch_exceptions=catch_exceptions, **kwargs)
    if expect_exit is not None:
        assert result.exit_code == expect_exit
    return result

Hi, any updates on this ?

2024 and click still swallows all testing output or requires an annoying wrapper that can only print output at the very end of the test. Both of these realities stink.
I don't understand the apprehension for adding an option to pipe/duplicate the stdout stream instead of completely hijacking it.

You have time to delete my comment but not address this 7+ year old issue?
Wow.


ImportError: cannot import name 'make_q4' from 'exllama_ext' (unknown location)

Hello!
Working through a small issue. I just installed this fork of exllama via github clone and installing the reqs.txt. When I try to run the example script, I'm getting this traceback:

Did I miss a step somewhere? Sorry, if this is a dumb ask. I'm super new at this :(

Make sure to install the exllama package with: python -m pip install .
This fork's purpose is to provide a separate package for the exllama code that does not rely on JIT compiling at runtime and allows for distributing the code and kernels so that others do not need to build it themselves.

Ah thank you!


Exclude test files and snapshots folders from copy task

Currently tests that run after copy to packages or dist throw the following error:
 10 obsolete snapshot files found, run npm run test:build:packages -- -u to remove them..

As paths in tests are very specific and wouldn't work for consumers, we can avoid packaging them.

yeah, not related to the inciddent


Add a NAT performance test with web server serving very big file

To test very high network speeds it is necessary to serve content from memory. Otherwise several gig files take too much space on disk.

This has been done using https://github.com/gshimansky/httpperfserv/ project which implements a web server that serves arbitrary number of bytes from memory.


When selecting a unit in version 5.0 the unit does not stay in focus when leaving the map.

In version 5.0 modern interface when a unit is selected from the devices menu - the unit does not continue to track of the unit leaves the map boundaries displayed.  Reselecting the unit recenters focus.
There is no option in either the web interface  the mobile to pin a unit for continuous tracking as the unit moves (or i'm unable to locate it).

There's an option in preferences called "follow" or something like that. It works the same way as the old app.


Russian translation(Doesn't work now because fonts)

The translation mostly works, but it doesn't display the font. (I don't know what to do to make it display the font lol)

Yay that's working


How to prevent images from loading?

Hey there, I'm coming from a Nightmare / Electron background. Within Nightmare I was able to set preferences like this to disable images from loading on the page:
webPreferences: {
  "images": false
}

I was wondering if there is a way to do this as well with puppeteer instead of the block images example in the repo. Thanks!

I was wondering if there is a way to do this as well with puppeteer instead of the block images example in the repo.

So far there's no such option. Why doesn't the approach described in block-images work for you?

It doesn't work for me because the request.resourceType is undefined in every request that is made 😢

@sean-hill request.resourceType hasn't made it to a release yet. It was added 3 days ago :)
In the meantime, you can nom install from master or use the demo's old approach  for checking image requests.

Thank you!!!

I made a plugin for resource blocking, In case anyone else want's to avoid boilerplate as well:
puppeteer.use(require('puppeteer-extra-plugin-block-resources')({
  blockedTypes: new Set(['image', 'stylesheet'])
}))


chore: Add migration to update indexes for DB collection to include branchName param

Description

After git feature is introduced resources are uniquely identified by combining defaultResourceId and branchName. This PR creates the compound indexes for newPage, newAction, actionCollection and application collections to include branchName which will optimise the find query.

Type of change

Enhancement

How Has This Been Tested?

Manual

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes

/ok-to-test sha=d5f7e2a

/ok-to-test sha=bff43e4

/ok-to-test sha=83e189f


unix: work around glibc semaphore race condition

Hack around https://sourceware.org/bugzilla/show_bug.cgi?id=12674
by providing a custom implementation for glibc < 2.21 in terms of other
concurrency primitives.
The glibc implementation on these versions is inherently unsafe.
So, while libuv and Node.js support those versions, it seems to make
sense for libuv in its functionality as a platform abstraction library
to provide a working version.
Fixes: https://github.com/nodejs/node/issues/19903
/cc @bnoordhuis

LGTM

Landed in https://github.com/libuv/libuv/commit/fbd6de31be38ba362daa56ccfd5be2a8f5d3b2b1. Thanks!

@rvagg thanks! this one was definitely fun to debug :)

@vcunat I mean, the point is more or less that libuv started using the broken code itself and ran into trouble on already-released versions. Yes, people should upgrade their systems, but doing so isn’t always feasible, and it’s not an option for Node.js or libuv to raise the glibc requirement in the middle of a release line.

Hmm, I see that the fact of libuv starting to use glibc semaphores internally might be perceived as a regression, and it would be much harder to suggest to "users" (distributions) that they apply a patch to glibc instead.

We support rhel/centos 6 and 7 and those ship with glibc 2.12 and 2.17 respectively.  RedHat supports 7 until mid 2024 so our options basically are:

Work around the glibc bug in libuv, or
Tell people to fix their glibc (not an option for most users), or
Tell people to suck it up for the next 6 years.

I like working around bugs in other people's software as much as the next guy (that is, I don't) but 1. seems like the least-bad option out of the three.

Oh, and this PR also paved the way for a better semaphore implementation on z/os, see #1805.

rhel/centos have roughly 200 patches on the vanilla glibc, most of them bugfixes, so I think those could easily be persuaded.  (They also have maintainers working upstream and backport patches to other older branches.)


docs: Enhancement Proposal: PVCs < 1 GB in TopoLVM

Related to https://github.com/topolvm/topolvm/discussions/646
Motivation
For most users of TopoLVM, local nodes are the focus.
While TopoLVM was designed to be run in datacenters where the lowest provisional unit might not be an issue, edge use cases require us to create many volumes in a small amount of storage which may only be a few gigabytes.
To allow TopoLVM for proper use on edge devices and small form-factor use cases of Pods, we need to allow storage capacities that can supply storage lower than 1GB.

@toshipp @satoru-takeuchi @pluser would it be possible to get a general thumbs-up on this on a high level within this week? we would like to collect feedback and start implementing this and are eagerly waiting for you to take a look at it. I know that some details on the scheduler might still be TBD but the rest should be fine, right?

@jakobmoellerdev
I have no major objections to the high-level design.
@satoru-takeuchi does not work on this, could you remove him from reviewers?

@jakobmoellerdev
Sorry for the delay in responding.
I have read through the discussion and proposal so far. As for me, I have no particular questions.

@pluser Would you mind checking a final time over this? With your approval this can go in I think. I would start working on that as soon as we have an agreement


use todo stack when traversing GC heap

collect all live objects in a todo stack
to be marked later

remove outer anychanges loop and scan GCBits

only mark todo items after current mark(p1, p2) is done
to benefit from cache locality, prefetcher and tail recursion

Updated
There is now a fanout limit to limit stack usage and I preserved left-to-right order for marking children.

Could you explain how this changes the marking compared to the stack-only version? Apart from the LTR ordering, isn't it just moving a limited number of entries to the stack?

Is your last graph comparing the new version to the version without stack entries or to the original GC? I cannot match "old" results to any of the previous graphs.

Have you tried using the stack entries as a ring buffer? This could avoid copying the range entries to the scan-stack at all. The dlist/slist tests should stay within the buffer for all the nodes.

The dlist/slist tests should stay within the buffer for all the nodes.

Right they do, that's why those benchmarks are so much faster than the original recursion code.
By marking up FANOUT_LIMIT pointers of a range before recursing, we can finish most data types (except arrays) in one batch. That's better for cache locality and avoids unnecessary stack activity.
The price is that we require some memory (at maximum depth of tree * (fanout of tree - 1)), because each recursion step might push fanout pointers before popping one to recurse.

Have you tried using the stack entries as a ring buffer? This could avoid copying the range entries to the scan-stack at all.

Yeah, might try that.

Yeah, might try that.

When I keep filling the local stack buffer during recursion I need to process it in RTL order to maintain a depth-first order traversal. Let's see how that works out.

It's slightly better and avoids using the global toscan stack in most benchmarks (except tree1 and testgc3), so that was a good idea.

It's slightly better and avoids using the global toscan stack in most benchmarks (except tree1 and testgc3), so that was a good idea.

Sounds promising. It seems you didn't commit this version yet, did you?
BTW: improving marking speed for the conservative GC raises the bar for acceptance for the precise version ;-) The "alternative" version is even incompatible with this change as it recurses into mark functions that call back into the GC for further scanning. This could just remember the new addresses, though.

Yes I did, it's this commit. https://github.com/MartinNowak/druntime/commit/8cc59070103325aae44fda9d20536d61a762c7ac

This could just remember the new addresses, though.

Right, it should be compatible. But I would assume that a precise mark function does all of the graph traversal on it's own. Why does it need to push ranges into the GC?

done?

Ready to go from my side.

But I would assume that a precise mark function does all of the graph traversal on it's own. Why does it need to push ranges into the GC?

A generic mark function might do this, but if there are custom mark functions specified by the RTInfo, they cannot traverse the graph without recursing (or pushing detected pointers into some array). The "alternative" precise GC uses these functions to scan dynamic arrays and AA nodes.

@rainers since you've been the main reviewer here I'm leaving approval to you. Thanks!

It's not a circular buffer though, see #1073 (comment).

I still believe it could be used to avoid some copying. The actual order is a bit random anyway (depending on where the stack buffer overflow happens), but we can try this as a later optimization.

I still believe it could be used to avoid some copying

No I can't process the pointers in FIFO order because that would be mainly a breadth-first order traversal requiring a much bigger stack for broad data structures like arrays.
Let's go on with this please.

I can't process the pointers in FIFO order

Ok, I start to get it. I hope I'll find some time to do some experiments, though.

Let's get this merged please.

LGTM modulo the recent line notes. Could you please comment on these?

Unless it's changed in the last couple of weeks while I wasn't looking, the largest problem with D's GC isn't in the speed of it's collection passes, but rather with how slow the allocations are currently. Attempt to allocate an AST with the GC as it was last I looked, and most of the time won't be spent in collections, instead that time will be spent in allocation. In those cases, even adding a backoff mechanism to the collection, or disabling collection entirely doesn't result in much of a performance boost in those situations.
If I remember correctly from when I hooked up a profiler to the GC, and assuming that it's not simply overly biased, most of the time spent allocating is not spent in determining where to do the allocations, but instead in setBits where the large number of branch mis-predictions is the cause.
My attempt at optimizing it via cmov instructions resulted in no real performance gain due to data caching issues.
The best way I know of to prevent the need for that is to simply reduce the amount of data being accessed, which I believe would be best done by having the pools distinguished between the possible bits that are set (excluding marked of course, and, for practical reasons, also excluding the finalization flags). All allocations in the same pool should result in the exact same flags from getBits. (again, excluding the marked and finalization flags) This would result in less memory overall being used, and it would result in the primary misprediction (setting the appendable bits) no longer being an issue.

the largest problem with D's GC isn't in the speed of it's collection passes, but rather with how slow the allocations are currently.

That depends on the application. An interactive application can live with not-so-fast allocations, but freezing for a second or more during a collection is inaccepible.

would be best done by having the pools distinguished between the possible bits

I tried allocating separate pools for scan and no_scan allocations, but that didn't change performance much. That experiment was in context of a concurrent GC, though.

Unless it's changed in the last couple of weeks while I wasn't looking, the largest problem with D's GC isn't in the speed of it's collection passes, but rather with how slow the allocations are currently.

I have quite a solid plan for improving GC performance, now that I really digged into the architecture.
One of the points is replacing freelists with bitmaps, it's less of a priority and more complex than other optimizations, but it should speed up allocations, because currently we write all over a complete 4K page when allocating it (see here).
Though the simpler and more important optimization for faster allocations is to use thread local caches to amortize the cost of locking. Also we don't currently do anything to avoid false sharing.

but instead in setBits

Never seen that, sounds weird, but you're welcome to contribute a benchmark if you have a real-world problem that suffers from slow GC of any sort.

All review points addressed.

Auto-merge toggled on

Thanks @rainers for the good code review.

Very happy to see this in! @MartinNowak care to celebrate by adding more detail and benchmark results to the changelog?

Very happy to see this in! @MartinNowak care to celebrate by adding more detail and benchmark results to the changelog?

I'm planning for a few more GC performance improvements.

optimize growth strategy
thread local caches for allocations
speeding up find pool
replacing free lists with bitmaps
using fixed size pools (maybe introducing a huge size class)

And yes, I planned to make a detailed summary for the changelog.
Let's see where the total numbers end up, but it really looks good.


module Vs tool .toml files

Right now the tool .toml files (MQ, AlphaPept...) contain information on how to parse the input tables (run name, column headers...) AND information about the module (expected ratio).
We should have:

toml files that contain only parsing information
one toml file per module that contain the module information

Also, right now the toml are named with the tool name, maybe we also want to include information of the input file (evidence, peptidegroup etc... for MQ for example) that it corresponds to. Because in the end there will be several toml files for MQ.

Maybe we should also add in .toml file the name of the parsed file (evidence.txt, peptides.txt...)

BTW, here is the list of the input files that we want to parse for each module:
https://docs.google.com/spreadsheets/d/1J_SVlD879Z8fG2SRwCmvwZ7-b3qXGn3kz3aPuW23RFY/edit#gid=0

Also, in all the toml files we kept "LFQ_Orbitrap..." in the run_mapper. I guess it is because it was copied/pasted from MQ. We should only keep "Condition_A_Sample_Alpha_03" etc...

Fixed in https://github.com/Proteobench/ProteoBench/pull/112


Bump cborg version to <IP_ADDRESS> (in line with stack)

ouroboros-consensus doesn't build otherwise.

bors r+


Jasper not listening

I managed to get Jasper to work however when it is running is won't listen to me saying 'jasper' and won't say what it is supposed to at startup, I have tried 3 different headphones and 2 different mics and none of them are working.
I ran /jasper/jasper.py --debug http://pastebin.com/3C9HCBHc
Just found running jasper.py without debug has started giving this error message
pi@raspberrypi ~ $ /home/pi/jasper/jasper.py

        JASPER - THE TALKING COMPUTER           *

(c) 2015 Shubhro Saha, Charlie Marsh & Jan Holthuis *

ALSA lib pcm_direct.c:980:(snd1_pcm_direct_initialize_slave) unable to install hw params
ALSA lib pcm_dmix.c:1030:(snd_pcm_dmix_open) unable to initialize slave
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm.c:2217:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm_direct.c:980:(snd1_pcm_direct_initialize_slave) unable to install hw params
ALSA lib pcm_dmix.c:1030:(snd_pcm_dmix_open) unable to initialize slave
ALSA lib pcm_dmix.c:957:(snd_pcm_dmix_open) The dmix plugin supports only playback stream
ALSA lib pcm_direct.c:980:(snd1_pcm_direct_initialize_slave) unable to install hw params
ALSA lib pcm_dmix.c:1030:(snd_pcm_dmix_open) unable to initialize slave
ERROR:apscheduler.executors.default:Job "Notifier.gather (trigger: interval[0:00:30], next run at: 2015-06-06 23:32:08 UTC)" raised an exception
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/apscheduler/executors/base.py", line 108, in run_job
retval = job.func(*job.args, **job.kwargs)
File "/home/pi/jasper/client/notifier.py", line 39, in gather
[client.run() for client in self.notifiers]
File "/home/pi/jasper/client/notifier.py", line 18, in run
self.timestamp = self.gather(self.timestamp)
File "/home/pi/jasper/client/notifier.py", line 43, in handleEmailNotifications
emails = Gmail.fetchUnreadEmails(self.profile, since=lastDate)
File "/home/pi/jasper/client/modules/Gmail.py", line 61, in fetchUnreadEmails
conn = imaplib.IMAP4_SSL('imap.gmail.com')
File "/usr/lib/python2.7/imaplib.py", line 1148, in init
IMAP4.init(self, host, port)
File "/usr/lib/python2.7/imaplib.py", line 163, in init
self.open(host, port)
File "/usr/lib/python2.7/imaplib.py", line 1159, in open
self.sock = socket.create_connection((host, port))
File "/usr/lib/python2.7/socket.py", line 553, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
gaierror: [Errno -2] Name or service not known
WARNING:apscheduler.scheduler:Execution of job "Notifier.gather (trigger: interval[0:00:30], next run at: 2015-06-06 23:32:38 UTC)" skipped: maximum number of running instances reached (1)
WARNING:apscheduler.scheduler:Execution of job "Notifier.gather (trigger: interval[0:00:30], next run at: 2015-06-06 23:33:38 UTC)" skipped: maximum number of running instances reached (1)

There is a problem with the gmail notifications. Try removing your email address from profile.yml and see if that helps.

Oh ok thanks, I haven't tested this yet because I have been busy Ill try later or tomorrow


Provide Implementations of Common Play/Twirl Interfaces

Functions like Messages and routes.Assets.at() are heavily used. Implementing some way to mimic these with Spring MVC would allow a more seamless transition between the two frameworks.

Instead of attempting to create a straight-up implementation of Play's generate routes package, it might be an idea to create a version of JSP's <c:url>, e.g. @Route("/path/to/resource") returns "/Spring-MVC-Twirl/path/to/resource.


Add Style to foods & recipes

General requirements
Make sure that there are [no linter errors].
Make sure that you used correct  gitflow.
Make sure that you documented your work [in a professional way.

Install bootstrap
add styles

@AlyAtalla You did the requirements nothing more than merge it


Laravel 8 migration

[x] Update Laravel => v8

[x] Set required PHP version to 7.3

[x] ~Switch from jwt-auth to Laravel Sanctum?~

Done in vue3 branch


Update release notes

Minor changes
Signed-off-by: Smriti S<EMAIL_ADDRESS>

Preview environment: https://ci.pr.harness.io/pr-363


resource_cloudflare_record search mismatch when allow_overwrite=true

Confirmation

[X] My issue isn't already found on the issue tracker.
[X] I have replicated my issue using the latest version of the provider and it is still present.

Terraform and Cloudflare provider version
cloudflare 3.15.0
Affected resource(s)

resource_cloudflare_record

Terraform configuration files
locals {
  fqdn = "value.domain.com."
}

# the first record is created
resource "record" resource_cloudflare_record {
  name=local.fqdn
  allow_overwrite=true
  type="A"
}

# the second record fails with the message
# attempted to override existing record however didn't find an exact match
resource "record" resource_cloudflare_record {
  name=local.fqdn
  allow_overwrite=true
  type="A"
}

Debug output
See line 5629:

original certificate requested: *.testtest.perium.nl
domain created by ACM certificate resource _795c354dbbac980dbca68aae23851205.testtest.perium.nl.
domain lookup by Clouldflare provider: _795c354dbbac980dbca68aae23851205.testtest.perium.nl..perium.nl

I0714 08:36:46.389837   31336 eventsink.go:59] GET /client/v4/zones/5cca90f59df588c7b60c9c30009e668c/dns_records?name=_795c354dbbac980dbca68aae23851205.testtest.perium.nl..perium.nl&page=1&type=CNAME HTTP/1.1

tflog.Debug
https://github.com/cloudflare/terraform-provider-cloudflare/blob/390201e830bc6b4a9ab41210a87734464ff3dbc1/internal/provider/resource_cloudflare_record.go#L140-L143
Panic output
No response
Expected output
both resources created
Actual output
error message: attempted to override existing record however didn't find an exact match
Steps to reproduce

Create 2 Record resources and use FQDN as a name
add allow_overwrite flag
terraform apply
observe that the 1st record is created while the 2nd one produces an error

Additional factoids
I'm using pulumi bridge provider https://github.com/pulumi/pulumi-cloudflare version 4.7.0 with TF provider version 3.15.0 however the bug is present in current master branch.
I believe that this piece of code is responsible for the problem.
https://github.com/cloudflare/terraform-provider-cloudflare/blob/390201e830bc6b4a9ab41210a87734464ff3dbc1/internal/provider/resource_cloudflare_record.go#L134-L136
Namely line 135 Name: d.Get("name").(string) + "." + zone.Name which appends zone.Name to the name argument.
Given:
zone: example.com
record: www.example.com
The code will produce a malformed string example.com.www.example.com causing the mismatch.
I found out about this problem when creating ACM certificate resource.
ACM Certificate resource outputs a CertificateOptions which contains the attribute 'resource_record_name', that is FQDN eg. '_ca595e19a694c3bf2dfa20123456.example.com.'
References
#pulumi/pulumi-cloudflare/issues/264

@jacobbednarz Could you please remove the needs-information label - I believe I provided enough information in case there is anything missing, please let me know.

looks like we're still missing the TF_LOG=DEBUG output from the command. irrespective of what you think is enough context, it is requested to confirm the internals and API calls so if you can provide that, we can have a look into the issue.

@jacobbednarz I updated the issue and attached the TF log file.

is the trailing period for the domain variable intentional? a cursory look at the debug logs suggests you're hitting this because the values (name and API zone name) don't match resulting in this code path. removing the trailing period should address this (also, you can use @ instead here too).

Had a look at this cause I was having the same issue and I believe this should be a bug rather than support.
The issue lies in https://github.com/cloudflare/terraform-provider-cloudflare/blob/master/internal/provider/resource_cloudflare_record.go#L132-L142
It detects if the name is either @ or matches the zone.name, if so then it uses the zone.name which is all fine.
If the above is not true then the request is made using d.Get("name").(string) + "." + zone.Name but name could be partial (i.e. subdomain) in which case this makes sense, or it could contain the zone.Name (i.e. subdomain.example.com).
I think there needs to be an else if in that block to detect if the zone.name is contained in the name and possibly also remove any trailing .

i don't want to extend the allow overwrite functionality any further as it is already a pretty decent sized foot gun.
if you're attempting to use a subdomain in DNS records, you want to only include the subdomain as part of the name value. i.e. name = "www". this should make the functionality here work as expected without yet another code path to evaluate.


Fixed elementSelected and Reply issues

Coverage decreased (-0.008%) to 99.01% when pulling e29030969423761df1199ae10b4a3fceba05e866 on omenocal:dev into add792791bed6fe993608089a1d8ff055e6a05ff on mediarain:dev.


Added method matching support.

Sorry, I only just saw this after doing a big rewrite. But good idea, and I'd be happy to merge if you're able to resolve the new conflicts with master. The main difference in the code is the move to es6 (or, rather, the subset of it supported in nodejs4)

@wheresrhys I've rebased it

Excellent thanks


Failed to load Emojis when using Twemoji renderer

There seems to be a problem when loading up Picmo when Twemoji renderer is enabled. The getTwemojiKey function fails and results in "Failed to load emojis". Pressing try again doesn't resolve the issue.
This can be replicated in the Playbook.

@traone Have you found any workaround for this issue?
@joeattardi What could suddenly be causing this?

@joeattardi Have you found a fix?

We are getting in the browser console:
Uncaught (in promise) TypeError: a is undefined [renderer-twemoji.js:2]

What might be causing this error? Our users are really annoyed, please find a fix as soon as possible!

Hello!
I have the same problem. Do you get a decision for this error "Uncaught (in promise) TypeError: a is undefined [renderer-twemoji.js:2]"?

Our users are really annoyed, please find a fix as soon as possible!

I'm sorry that your users are really annoyed and I hope to be able to find the cause of this soon.
However you need to manage your expectations. I'd like to remind you that maintaining PicMo is something I do on the side in my spare time. I have never made a single cent from my work (despite companies like Tinder and Starbucks benefitting from the work).
I have a full time job that takes up a huge chunk of my time, I am also writing a book right now which takes up another huge chunk.
I appreciate you using PicMo but keep in mind that you assume a certain amount of risk when using open source software, including the risk that bugs might not be fixed on a timeline that you would prefer.
I hope to have some time soon to look into this, but for now I'd recommend using the native renderer instead.

It looks like Twemoji is in bad shape right now. A while back Twitter all but abandoned the open source project. There is a fork now, but they have not yet finished adding all of the new emojis for Emoji 15.
I can't draw emoji images so it's effectively blocked for the time being.
However! If you still want to use Twemoji and are ok with using Emoji 14 only, you can specify emojiVersion: 14 in the picker options and this should hopefully resolve the issue for now.

@joeattardi I appreciate your prompt reply!
We are OK with using Emoji Version 14 as there seem to be no issues at the moment.

Failed to load emojis is coming for me only on iOS when using and viewing the emoji picker from my laptop that's windows it's working but when viewing the emoji picker from my iPhone on Safari it's giving Failed to load emojis

What options are you passing to the picker?
Is there a stack trace?

What options are you passing to the picker?
Is there a stack trace?

This is my config for createPop:
autoFocus: 'search',
            initialCategory: 'custom',
            theme: darkTheme,
            className: 'my-picker',
            maxRecents: 10,
            categories: [
                'recents',
                'custom',
                'smileys-emotion',
                'people-body',
                'animals-nature',
                'food-drink',
                'travel-places',
                'activities',
                'objects',
                'symbols',
                'flags',
            ],
            renderer: new TwemojiRenderer(),
            custom: emotes,
            i18n: {
                'categories.activities': 'Activities',
                'categories.animals-nature': 'Animals & Nature',
                'categories.custom': '<%=guild.name%>',
                'categories.flags': 'Flags',
                'categories.food-drink': 'Food & Drink',
                'categories.objects': 'Objects',
                'categories.people-body': 'People & Body',
                'categories.recents': 'Recently Used',
                'categories.smileys-emotion': 'Smileys & Emotion',
                'categories.symbols': 'Symbols',
                'categories.travel-places': 'Travel & Places',
                'error.load': 'Failed to load emojis',
                'recents.clear': 'Clear recent emojis',
                'recents.none': 'You haven\'t selected any emojis yet.',
                'retry': 'Try again',
                'search.clear': 'Clear search',
                'search.error': 'Failed to search emojis',
                'search.notFound': 'No results found',
                'search': 'Search emojis...'
            }
        },{
            referenceElement: trigger,
            triggerElement: trigger,
            position: 'bottom',
            showCloseButton: false
        }```

Please see above https://github.com/joeattardi/picmo/issues/280#issuecomment-1594541854
Twemoji is broken for the latest emoji version, it's out of my control. If you use emoji version 14 it should work again.

Please see above #280 (comment)
Twemoji is broken for the latest emoji version, it's out of my control. If you use emoji version 14 it should work again.

Oh it's due to the above one only? beacuse i thought it's opening on laptop and not mobile then it should be due to something else.
Btw got this in console from mobile browser:

My mistake, looks like this is something different, sorry about that.

My mistake, looks like this is something different, sorry about that.

But when I changes the version of Twemoji the issue got fixed, weird


Integrating Apple Pay

Hi,
I have been trying to implement Apple Pay for days but I can't succeed. I am asking for help here because Stripe support does not seem to know much about this package.
Here is the flow I use :
import { useApplePay, useStripe } from '@stripe/stripe-react-native';

const MyComponent = () => {
  const {
    presentApplePay,
    confirmApplePayPayment
  } = useApplePay({
    onShippingMethodSelected: (shippingMethod, handler) => {
      // Handling shipping methods selection here, works fine.
    }
  });

  const stripe = useStripe();

  const handleApplePay = () => {
    const { error, paymentMethod } = await presentApplePay({ // params here work fine, not important });

    if (!error) {
      // Call to my custom API. Params are the total amount, the currency code and the address
      const paymentIntent = createPaymentIntent(params);

      // Always fails, returns {"error": [Error: Payment not completed]}
      const confirmation = await confirmApplePayPayment(paymentIntent.clientSecret);

      // I have never tested this because I never get to this point
      const confirmPayment = await stripe.confirmPayment(clientSecret);
  }

  return (
    <Pressable onPress={handleApplePay}>
      <Text>Apple Pay</Text>
    </Pressable>
  )
}

I have several questions :

Is my flow right?
Why does the confirmApplePayPayment method keeps failing?
How does Stripe knows what PaymentMethod to use ? I is done automatically with the confirmApplePayPayment method or should I do something else?
How do I get the shipping address the user selected so I can save it to my database (actually using Shopify)?
Depending on the shipping country, the shipping methods are not the same. Is there a way to update shipping methods ?

If you need more context I will provide it to you, but I don't really know what else to say for now.
Thank you.

The useApplePay hook needs to be called from a functional component (i.e. inside MyComponent). Your ESLint setup should catch that. Otherwise, this flow looks good to me, but let me know if that doesn't fix things 👍
I'll answer some of your other questions:

How does Stripe knows what PaymentMethod to use ? I is done automatically with the confirmApplePayPayment method or should I do something else?

It's done automatically, you'll use the payment method you just created with Apple Pay

How do I get the shipping address the user selected so I can save it to my database (actually using Shopify)?

Using the callbacks you supply to the useApplePay hook: https://stripe.dev/stripe-react-native/api-reference/interfaces/UseApplePayProps.html

Depending on the shipping country, the shipping methods are not the same. Is there a way to update shipping methods ?

Same answer as above :)

Thanks a lot for your help. I made a mistake writing this issue, my hooks do get called from the component. So this is not the issue...
As for the shipping method, I can't get it from the hooks because it is incomplete. See https://github.com/stripe/stripe-react-native/issues/1120. It is a security measure from Apple. Is there any way to get the entire address after Apple Pay confirmed the payment ?
And regarding the shipping methods, I can update cart items with the handler of the callback, but can I update the actual shipping methods ?

We are adding the functionality to edit the actual shipping methods, this will be coming soon

Hi @aurelien-robineau
I am stuck at the exact same problem, can you please let me know how did you resolve your issue?
Thanks.

Hi @EmTeaZy, I am sorry I see your comment only now. You have probably found a solution by now but this may help others.
The issue I had was in the onShippindMethodSelected method. As you can see it takes a handler param, which absolutely needs to be called. If not, Apple Pay will fail.

hi @aurelien-robineau ! you should be able to do this now with the Platform Pay APIs


DBZ-3779: Import Pattern-fly CSS from @patternfly/patternfly

Importing patternfly css only from @patternfly/patternfly.
using MiniCssExtractPlugin.loader.

@indraraj I got the following WARNING upon running yarn start after yarn install and yarn build
WARNING in chunk src_app_shared_index_ts-src_i18n_index_ts-webpack_sharing_consume_default_react-dom_react-dom-2016c9 [mini-css-extract-plugin]
Conflicting order. Following module has been added:
 * css ../../node_modules/css-loader/dist/cjs.js??ruleSet[1].rules[1].use[1]!../../node_modules/postcss-loader/src/index.js??ruleSet[1].rules[1].use[2]!./src/app/components/ApplicationErrorPage.css
despite it was not able to fulfill desired ordering with these modules:
 * css ../../node_modules/css-loader/dist/cjs.js??ruleSet[1].rules[1].use[1]!../../node_modules/postcss-loader/src/index.js??ruleSet[1].rules[1].use[2]!./src/app/layout/layout.css
   - couldn't fulfill desired order of chunk group(s)

Probably you could add ignoreOrder: true under MiniCssExtractPlugin({ignoreOrder: true, ... to resolve.

Thanks for catching, update the PR with a fix.

LGTM. Thanks.


Allow for overlapping Aggregates

Issue type:
Feature Request

Python version:
NetBox version:

We would like to see support for overlapping Aggregates.  We use Aggregates to show available subnets within the defined Aggregate.  We allocate address space hierarchically so we'd like the flexibility of scoping our output so the most appropriate aggregate for the task.  For instance:
<IP_ADDRESS>/8 is used for customer networks
<IP_ADDRESS>/9 is used in DC1
<IP_ADDRESS>/9 is used in DC2
<IP_ADDRESS>/16 is allocated to Customer X in DC1
etc...
If I need a new /16 for a new customer in DC1 we'd like to look at the 10/9 Aggregate.
If we need to allocate a new /24 network to Customer X in DC1 we'd like to look at the 10.1/16 Aggregate.

This would defeat the purpose of aggregates. Aggregates define the top level of your IP space hierarchy. Use prefixes to organize space within aggregates.


Something wrong with this waypoint???

Challenge Waypoint: Manipulate Arrays With shift has an issue.
User Agent is: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.86 Safari/537.36.
Please describe how to reproduce this issue, and include links to screenshots if possible.
My code:
var ourArray = ["Stimpson", "J", ["cat"]];

removedFromOurArray = ourArray.shift(); // removedFromOurArray now equals "Stimpson" and ourArray now equals ["J", ["cat"]].

var myArray = ["John", 23, ["dog", 3]];

// Only change code below this line.

removedFromMyArray = myArray.shift();

var removedFromMyArray = ["John"]

// Only change code above this line.

(function(y, z){return 'myArray = ' + JSON.stringify(y) + ' & removedFromMyArray = ' + JSON.stringify(z);})(myArray, removedFromMyArray);

It doesn't have a pass or fail mechanic that other waypoints have.

This has been fixed by PR #4870 and is pending deployment to the production website.
Thanks and happy coding!


Optimize job poll interval for Slurm computer for production

https://github.com/aiidateam/aiida-core/issues/5638#issue-1369918831

Hmm, interestingly it seems in AiiDA environment the default min job interval is always 10 seconds?
verdi shell
 load_computer("slurm").get_minimum_job_poll_interval()

https://aiida.readthedocs.io/projects/aiida-core/en/latest/topics/performance.html?highlight=benchmark workflow#benchmarks

I noticed that the daemon worker takes 50% CPU when waiting for ORCA job. That seems very excessive. Not sure if it is related to the issue here, or is something else...


DOP-4552: Get facet names programmatically in Query/index.ts

Ticket
DOP-4552
Notes
I wanted to leverage the existing getFacetKeys function in Search Index, but obtaining the taxonomy and passing it into this function outside AtlasAdmin proved a bit cumbersome.
This solution is the main one that comes to mind, but open to any suggestions for better code structure or locally storing the facet key names into a member of SearchIndex, similar to insertTaxonomyIntoSearchIndex but to my knowledge the results of this function are only being uploaded online and doesn't exist locally/in a way that's easily accessible to Query.
README updates

[ ] This PR introduces changes that should be reflected in the README, and I have made those updates.

[x] This PR does not introduce changes that should be reflected in the README

removed the function i added. stored taxonomy and result of getFacetKeys in SearchIndex and passing that into the search functions


Enable guava-beta-checker and fix findings

PR Description
guava-beta-checker is Google's EP checker that identifies usage of beta Guava APIs. This checker is limited to the main source tree and doesn't produce warnings for test code. There weren't many findings.

Such APIs should never be used in library code that other projects may depend on; using the Beta Checker can help library projects ensure that they don't use them.

When removing Streams.zip I was worried that the new implementation might be slower, so I added a benchmark function for the BLS.aggregateVerify method. In the process of adding this, I made a some changes to whole benchmark class; things could be simpler with a setup function.
Before
Benchmark                              (sigCnt)   Mode  Cnt    Score   Error  Units
BLSBenchmark.aggregateVerifySignature         1  thrpt   10  926.271 ± 1.448  ops/s
BLSBenchmark.aggregateVerifySignature         2  thrpt   10  710.323 ± 1.462  ops/s
BLSBenchmark.aggregateVerifySignature         4  thrpt   10  481.074 ± 5.471  ops/s
BLSBenchmark.aggregateVerifySignature         8  thrpt   10  295.479 ± 2.709  ops/s
BLSBenchmark.aggregateVerifySignature        16  thrpt   10  163.657 ± 1.735  ops/s
BLSBenchmark.aggregateVerifySignature        32  thrpt   10   86.401 ± 0.738  ops/s
BLSBenchmark.aggregateVerifySignature        64  thrpt   10   44.565 ± 0.155  ops/s
BLSBenchmark.aggregateVerifySignature       128  thrpt   10   22.638 ± 0.093  ops/s

After
Benchmark                              (sigCnt)   Mode  Cnt    Score   Error  Units
BLSBenchmark.aggregateVerifySignature         1  thrpt   10  927.673 ± 1.370  ops/s
BLSBenchmark.aggregateVerifySignature         2  thrpt   10  712.032 ± 0.952  ops/s
BLSBenchmark.aggregateVerifySignature         4  thrpt   10  485.891 ± 2.164  ops/s
BLSBenchmark.aggregateVerifySignature         8  thrpt   10  294.099 ± 1.980  ops/s
BLSBenchmark.aggregateVerifySignature        16  thrpt   10  162.698 ± 1.052  ops/s
BLSBenchmark.aggregateVerifySignature        32  thrpt   10   86.303 ± 0.847  ops/s
BLSBenchmark.aggregateVerifySignature        64  thrpt   10   44.676 ± 0.309  ops/s
BLSBenchmark.aggregateVerifySignature       128  thrpt   10   22.720 ± 0.018  ops/s

So things are pretty much the exact same. The new version might actually be a hair faster.
Documentation

[x] I thought about documentation and added the doc-change-required label to this PR if updates are required.

Changelog

[x] I thought about adding a changelog entry, and added one if I deemed necessary.

On second thought, Teku isn't a library and this restriction doesn't really apply; we control the version of Guava we use. If a beta API is modified, we will know. So unless this is something you guys want, I'll close this PR.

The Streams.zip is fairly neat, so we might stick with the current implementation :)

Yup I agree. Sorry to waste your time. Thanks!

I'd never say anything like this is a waste of time :) thanks for your continued efforts!


How to insert a NULL value for slug

I am creating a 5 level deep menu for my website. 3 levels always need a unique slug, but 2 levels do not need a slug.
How can I insert a row in my database without executing the Sluggable event, when, for example, my $request->level == 2.
`public function store( StoreMenu $request ) {
if( $request->level == 2) {
//Insert without sluggable
}else{
insert with sluggable
}
}
`

Untested, but I think this would work:

Set the source of your slug to be a custom method on your model via config:

public function sluggable()
    {
        return [
            'slug' => [
                'source' => 'customSource'
            ]
        ];
    }

Configure that function to handle the logic, returning NULL in the case where you don't need a slug and the real field when you do (e.g. the "title" of your model):

public function getCustomSourceAttribute()
{
    if (\Request::get('level') == 2) {
        return null;
    }

    return $this->title;
}

Let me know if that works!

It works exactly how you described it! Thanks!


After shutting down a node, pods on other nodes cannot access the keepalived VIP.

Expected Behavior
After shutting down a node, pods on other nodes can access the keepalived VIP.
Actual Behavior
I have three master nodes, the IP addresses are: <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>. There is also a VIP created using keepalived: <IP_ADDRESS>.
But when <IP_ADDRESS> is shut down, the pod on <IP_ADDRESS> cannot access VIP <IP_ADDRESS>, but can access <IP_ADDRESS>, and vice versa.
<IP_ADDRESS>/16 is the cidr of my ovn-default, <IP_ADDRESS>/16 is the cidr of the join, and there is a route of <IP_ADDRESS>/16 via <IP_ADDRESS> dev ovn0 on each node.
I found that manually deleting this route from <IP_ADDRESS> will affect the pod on 103's access to VIP.
Another thing is that if you shut down the <IP_ADDRESS> node, the remaining 102 and 103 are normal, and you can access VIP without being affected; shutting down any node 102 or 103 will only affect the other party, and the pods on 101 will not be affected.
I don’t know where the problem is that affects 102 and 103. Can you provide some ideas?
Steps to Reproduce the Problem

Additional Info

Kubernetes version:
Output of kubectl version:
v1.25.4

kube-ovn version:
v1.11.8

operation-system/kernel version:
Output of awk -F '=' '/PRETTY_NAME/ { print $2 }' /etc/os-release:
Output of uname -r:
"Ubuntu 22.04.3 LTS"

5.15.0-78-generic

Did you set the gatewayType to centralized and gatewayNodes to <IP_ADDRESS>

Did you set the gatewayType to centralized and gatewayNodes to <IP_ADDRESS>

The subnet is ovn-default, and the gatewayType is distributed.
spec:
  cidrBlock: <IP_ADDRESS>/16
  default: true
  excludeIps:
  - <IP_ADDRESS>
  gateway: <IP_ADDRESS>
  gatewayNode: ""
  gatewayType: distributed
  natOutgoing: true
  private: false
  protocol: IPv4
  provider: ovn
  vpc: ovn-cluster

@oilbeater
I tried to set the gatewayType to centralized and gatewayNodes to <IP_ADDRESS>, but the pod kube-ovn-controller reported errors:
E1019 09:38:19.081378       1 subnet.go:182] error syncing 'ovn-default': failed to add ecmp policy route, no gateway node exists, requeuing
I1019 09:38:24.201913       1 subnet.go:315] format subnet ovn-default, changed false
I1019 09:38:24.517695       1 ovn-nbctl-legacy.go:414] set logical switch: ls ovn-default, lr ovn-cluster, protocol IPv4, subnet <IP_ADDRESS>/16, gw <IP_ADDRESS>
I1019 09:38:24.536404       1 ovn-nbctl-legacy.go:2919] delete dhcp options for switch ovn-default protocol Dual
I1019 09:38:24.542358       1 ovn-nbctl-legacy.go:2964] set lrp ovn-cluster-ovn-default
E1019 09:38:24.624419       1 subnet.go:1825] gw node <IP_ADDRESS> does not exist, node "<IP_ADDRESS>" not found
E1019 09:38:24.624455       1 subnet.go:1301] failed to init centralized gateway for subnet ovn-default, no gateway node exists
E1019 09:38:24.624472       1 subnet.go:934] reconcile OVN route for subnet ovn-default failed: failed to add ecmp policy route, no gateway node exists
E1019 09:38:24.624514       1 subnet.go:727] reconcile subnet for ovn-default failed, failed to add ecmp policy route, no gateway node exists

The vip <IP_ADDRESS> does not belong to the ip of the cluster nodes.
Is there any other way to troubleshoot this problem? It has been bothering me for a long time.

Does the VIP also runs on the master nodes?
For Pod running in kube-ovn network, the VIP address will be treated as an external address, and the packet will be send to the ovn0 on the node the pod runs. And then it should be snated to the node address to visit the VIP. I recommend to use tcpdump to trace the packet to see if something goes wrong.


Slate is rendering without any formattting when I publish to HTML

Hello,
I published my slate docs to HTML and then moved them to another server... now all the sidebars and font formatting have disappeared.  It looks like this:

not sure what happened, so I would appreciate it if anyone had any ideas!  Thanks
Edit: It's a standard LAMP server if that helps!

Got any page errors in the inspector? Somehow it's just not finding your javascript or CSS files.

nope, no page errors at all...  when I check the net section of the console, everything loaded fine.  I've got everything just the way middleman generated it with the "javascripts" and "stylesheets" folders in the "build" folder.  Thanks in advance for any insight you might have!

Wat. Hmm. Is it a private server, or do you have a link I can look at?

unfortunately, I'm documenting something proprietary for my company, so I don't think I can share the link right now... At least I know now that it seems like an issue with it not loading the CSS or javascript!  Thanks for that!  If there's anything you can think of off the top of your head, let me know

Will do! Difficult to debug without more and others aren't having this issue, so I'm closing it for now, but I'll let you know if I think of anything. And if you encounter it on a server I can look at, let me know!

I figured it out!  turns out ad block plus was blocking the css and javascript haha.... weird...

haha, nice! yea, that's really bizzare. does it happen on other Slate sites like http://lord.github.io/slate/?

nope lol.  just seems like my site in particular for whatever reason... weird

Cool, well good to hear it's not a wider issue. Thanks for following up!

sure thing!  Thanks for your help!


how to run the spatial-LDA on all cells

I'd like to run the spatial-LDA on all the cells in my dataset. I tried to set all cells as index cells by setting is_anchor_col argument in the featurize_samples function to a column where all values are True, but that produces an error.
if os.path.exists(PATH_TO_TUMOR_MARKER_FEATURES_PKL):
    with open(PATH_TO_TUMOR_MARKER_FEATURES_PKL, 'rb') as f:
        tumor_marker_features = pickle.load(f)
else:
    neighborhood_feature_fn = functools.partial(neighborhood_to_marker,
                                              markers=markers)

    tumor_marker_features = featurize_samples(patient_dfs, neighborhood_feature_fn, 100, 'is_index',
                             'x', 'y', n_processes=N_PARALLEL_PROCESSES)
    with open(PATH_TO_TUMOR_MARKER_FEATURES_PKL, 'wb') as f:
        pickle.dump(tumor_marker_features, f)

ERROR:
RemoteTraceback                           Traceback (most recent call last)
RemoteTraceback:
"""
Traceback (most recent call last):
File "/home/kidzik/miniconda3/envs/spatial-lda/lib/python3.7/multiprocessing/pool.py", line 121, in worker
result = (True, func(*args, **kwds))
File "/home/kidzik/miniconda3/envs/spatial-lda/lib/python3.7/site-packages/spatial_lda-0.0.3-py3.7.egg/spatial_lda/featurization.py", line 51, in _featurize_sample
is_anchor_col, x_col, y_col, z_col=z_col)
File "/home/kidzik/miniconda3/envs/spatial-lda/lib/python3.7/site-packages/spatial_lda-0.0.3-py3.7.egg/spatial_lda/featurization.py", line 38, in _featurize_cells
neighborhood_kdTree = KDTree(neighborhood_cells[coord_cols].values)
File "/home/kidzik/miniconda3/envs/spatial-lda/lib/python3.7/site-packages/scipy/spatial/kdtree.py", line 239, in init
self.maxes = np.amax(self.data,axis=0)
File "/home/kidzik/miniconda3/envs/spatial-lda/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 2505, in amax
initial=initial)
File "/home/kidzik/miniconda3/envs/spatial-lda/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 86, in _wrapreduction
return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
ValueError: zero-size array to reduction operation maximum which has no identity
"""
The above exception was the direct cause of the following exception:
ValueError                                Traceback (most recent call last)
 in
7
8     tumor_marker_features = featurize_samples(patient_dfs, neighborhood_feature_fn, 100, 'is_index',
----> 9                              'x', 'y', n_processes=N_PARALLEL_PROCESSES)
10     with open(PATH_TO_TUMOR_MARKER_FEATURES_PKL, 'wb') as f:
11         pickle.dump(tumor_marker_features, f)
~/miniconda3/envs/spatial-lda/lib/python3.7/site-packages/spatial_lda-0.0.3-py3.7.egg/spatial_lda/featurization.py in featurize_samples(sample_dfs, neighborhood_feature_fn, radius, is_anchor_col, x_col, y_col, z_col, n_processes)
87             all_sample_features = list(tqdm(pool.imap(featurize_sample_fn,
88                                                       sample_dfs.items()),
---> 89                                             total=total))
90     else:
91         for i, sample_df in sample_dfs.items():
~/miniconda3/envs/spatial-lda/lib/python3.7/site-packages/tqdm/_tqdm_notebook.py in iter(self, *args, **kwargs)
221     def iter(self, *args, **kwargs):
222         try:
--> 223             for obj in super(tqdm_notebook, self).iter(*args, **kwargs):
224                 # return super(tqdm...) will not catch exception
225                 yield obj
~/miniconda3/envs/spatial-lda/lib/python3.7/site-packages/tqdm/_tqdm.py in iter(self)
1003                 """), fp_write=getattr(self.fp, 'write', sys.stderr.write))
1004
-> 1005             for obj in iterable:
1006                 yield obj
1007                 # Update and possibly print the progressbar.
~/miniconda3/envs/spatial-lda/lib/python3.7/multiprocessing/pool.py in next(self, timeout)
746         if success:
747             return value
--> 748         raise value
749
750     next = next                    # XXX
ValueError: zero-size array to reduction operation maximum which has no identity
I think it's because the neighborhood_cells becomes an empty arrey if I set all cells as index cells.
Can you kindly advise on how to run the pipeline on all the cells, please?
Thanks a lot!
Magda

Fixed by #13


Update batch-compute-node-environment-variables.md

Adds documentation for an existing environment variable

@bgklein : Thanks for your contribution! The author(s) have been notified to review your proposed change.


String to entity conversion inconsistencies

Using v 2.03 of the SDK I experienced inconsistent handling of entities by the SDK.
For instance, when importing an account ID from a .env file:
const myAccountId = process.env.MY_ACCOUNT_ID
And then using myAccountId to configure the client:
client.setOperator(myAccountId, myPrivateKey);
setOperator seems to convert the string to an Account ID entity.
However, when performing other calls, specifically TopicMessageSubmitTransaction() the SDK manages it differently; producing a type error when passing a string into the following:
const myTopicId = process.env.MY_TOPIC_ID
const MessageSubmit = await new TopicMessageSubmitTransaction({
      topicId: myTopicId,
      message: "Hello there!",
    }).execute(client);
  }

This produces a type error while I anticipated the string to entity conversion being consistent across all SDK calls.

Thanks for pointing this out, what you have should definitely work; we'll fix this with the next release.

#400 was very close to being merged and released with https://github.com/hashgraph/hedera-sdk-js/milestone/2 However, there were some mirror issues within the PR which prevented it from being merged. Moving this issue to milestone https://github.com/hashgraph/hedera-sdk-js/milestone/3


Fix incoming links in medium article (sruffilli)

TL;DR
We recently change Fabric's repo structure. Please update your medium article links to point to the right location.
Expected behavior
No response
Observed behavior
No response
Terraform Configuration
NA

Terraform Version
NA

Additional information
Affected articles:

Resource Factories: A descriptive approach to Terraform

Thanks Julio - Fixed

@sruffilli in the article, this links points to a non-existing page: https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/tree/master/examples/factories
Can you please have a look? Thanks!

Eagle eye!
Fixed.


Add Philips hue Calla outdoor lamp

Can you add the Calla outdoor Lamp?
https://www2.meethue.com/nl-nl/p/hue-white-and-color-ambiance-calla-sokkellamp/1742030P7
Product id: LCF002
Service id: 528
profile id: 49246
after try to add to says to set it up as a simple zigbee device where i only can turn it on or off.
Homey user

Thanks for the information, I will add this to the list of devices to be added.

Now in test version.
https://homey.app/sv-se/app/com.philips.hue.zigbee/Philips-Hue-ZigBee/test/


add browser test suite

Some additional plugins to consider using:

https://github.com/bahmutov/cypress-react-unit-test could possibly port Mocha tests from JSDOM to this
https://docs.cypress.io/plugins/#visual-testing something for visual regression testing (png snapshots)
https://github.com/cypress-io/code-coverage code coverage (requires Istanbul/NYC manual setup for instrumentation)

@zenamaro try rebasing off master or cherry-picking the last 2 commits

Wow, this is pretty cool indeed.
I guess this can replace not only all of the JSDOM and Enzyme tests, but potentially any future unit test as well (eg. jest). No point in running them on node anyway.
I suggest we port the unit tests (except maybe things like "calls componentDidMount") and ideally we can cover everything around regeneration (ie #317 and things like restoring content on dirtyProp change, etc).

Yeah, I think the majority of the IO tests can go through Cypress. There may be a few unit tests left, but ideally we’d want those to be generic enough that they would work as regression tests for a hooks version, instead of calling instance methods directly.

@zenoamaro I ported over most of the old Enzyme unit tests to browser tests and added a "test harness" set up instead of using the demo page in the tests.
Also discovered that because we were using chai and mocha, we can pretty much run the old tests directly inside Cypress just by importing the file (!!).

@zenoamaro it looks like the Travis integration stopped reporting in, can you re-enable it?

Merging this. I think there is enough to go on that we (or someone else) can write failing test cases for outstanding bugs now.


Feature: exceptions for nonzero codes for specific tools?

Check https://github.com/asottile/add-trailing-comma/issues/83 for some information about this issue.
In a nutshell: some tools return 1 just to indicate something was modified (the case with add-trailing-comma) but autoformat only applies the formatting if the status code is 0. I was thinking about a variable that you can check for different status codes for different tools. The default is still 0.

Looking at the sourcecode (https://github.com/asottile/add-trailing-comma/blob/master/add_trailing_comma.py) I would say that the exitcodes used in that tool are badly designed. It is not possible to distinguish between e.g. unicode decode error and a successful run with changes being made.
Secondly, I would say that tool is conceptually not a formatter. Because for a formatter a successful run would involve formatting the code, whereas in that tool, formatting is seen as a failure (due to exitcode > 0). So it's more of a weird checker+formatter mashup.
I'm inclined to oppose the introduction of complexity for dealing with conceptually invalid cases. However, if this problem starts to occur for more tools, then the complexity may be justified and I will change my mind.
For now, the right thing seems to me to patch that tool with a commandline option that let's it function as a either checker or a formatter. If the maintainer is unwilling to support that, you can wrap that tool in a shell script that always gives exitcode 0.
I will close this issue for now, but feel free to continue the discussion or bring up any thoughts.

For this specific tool, add-trailing-comma, a new parameter was added, --exit-zero-even-if-changed.


[FLINK-22721][ha] Add default implementation for HighAvailabilityServices.cleanupJobData

In order to not break the HighAvailabilityServices interface, this commit adds a default
implementation for the HighAavilabilityServices.cleanupJobData method which does nothing.
Users of this interface are recommended to override this method and add a specific
implementation in order to clean up job specific HA services after a job completion.

Thanks for the review @XComp. Merging this PR now.


Factor out build version to separate file

This request has 3 components:

Some more cleanup of files removing inappropriate executable bits and trailing whitespace
Structuring the the build version so that it can be specified with a compiler option or in a version file (which can then be auto-generated)
Formatting the hex dump of the EEPROM so that each byte is specified by two hex digits

Good Work, thanks!


Incompatible with MoreTags

https://pastebin.com/0nzgSSjK

Issue is not on BCLib side

Update: looks like issue is with redirect in both libraries. Strange thing is that it worked fine for a long period of time

Should work fine since 1.4.4


IPhone 8 Plus IOS 12.1 uncover 3.0.0b37 line:1188

#pinned
Need help
And info
Please
https://cdn.discordapp.com/attachments/349251947192254466/550934863813804032/image0.png
When trying to jailbreak and trying to restore RootFS on an iPhone 8 Plus  @ iOS 12.1 unc0ver fails because RootFS is already mounted.
There is no OTA file on the iPhone
"shapshotSystemVersion"?
Also updates doesn’t work never used a ota blocker

Previously I had iOS 11.3.1 with Electra accidentally updated to iOS 12.1 stayed on it till a jailbreak came out and that happened

@Gerardo323 You didn't follow the guidelines, one of thing from the check list said to check for if the issue already exist before posting. Please read the guidelines next time #627.
Duplicate of #629 , so closing.


add loadStale cache option to load already expired cache data

Closes #250 with an additional cacheOptions.loadStale to load the current cached state even if it is expired.

This is an excellent idea, @blackjid, thank you!

Publishing now under<EMAIL_ADDRESS>


[site] Generate complete DIF description

Currently the text produced for the DIF description stops at the first
subelement. This is apparent if the description contains a variable
enclosed in backticks. The text up to this variable is returned but not
the rest.
Instead of using the text member of the element, which only contains
the text up to the first subelement according to the ET documentation,
create the string manually.
Create the string for the XML element and process the result for usage
in HTML.
Another option to create the string would be to use itertext to get
the text of all subelements, but properties would be lost so no code
highlighting would be possible.
Fixes #5001
Signed-off-by: Tobias Wölfel<EMAIL_ADDRESS>

Modified the regex to reflect the discussion to make it more robust, just in case this will be merged.

Close in favor of #5251


CloudFromation yaml is no more accessible

Hello,
When I want to deploy the Cloudformation we have the following error on the provide file https://chriskingpartnershare.s3.amazonaws.com/PersonalizePOC.yaml:
S3 error: Access Denied For more information check http://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html

Template is inaccessible


Support procedural for loops

Feature Request
Is your feature request related to a problem? Please describe:

Describe the feature you'd like:

I'm using kcl to generate configuration with docker-compose for etcd cluster, i want to set a param of cluster node count and then generate a cluster, and the kcl configuration like this:
for _i:=1;i<5;i++{
        _composeService+={...}
}

This is very helpful for quickly generating a docker-compose file for a stateful service cluster! and seems like it's not support now:  https://www.kcl-lang.io/docs/next/user_docs/support/faq-kcl#46-procedural-for-loop
The case2 is dynamically generate kind cluster configuration files for multiple kind nodes. https://github.com/kcl-lang/kcl/issues/1410
you can see that now i just use if count==3 / count ==5 for it (:
Describe alternatives you've considered:

Teachability, Documentation, Adoption, Migration Strategy:

Thanks for quickly reply and i will try the range in tomorrow, seems like it's what i want!

closing.. range is working for me.


643 Provide tril([out] Matrix, [in] Matrix, int, Phase)

Closes #213
~Requires #231~
~Requires #241~
Generalisation of this feature in #271

If we have have select in v0.9 (#271) then this issue should be resolved as an algorithm (maybe grb::algorithms::matrices<>::tril or grb::algorithms::tril).

Given we have #271 slated for v0.8, should this MR still be retained actually?

Given we have #271 slated for v0.8, should this MR still be retained actually?

I think it should be discarded


pm: remove pm global, make per domain isolated

Summary
Remove the PM gloabal variables. make callback register to the domain directly.
To keep compatible, make default pm_register behaviour only register to the PM_IDLE_DOMAIN.
Impact
When CONFIG_PM_NDOMAINS==1, no impacts.
When CONFIG_PM_NDOMAINS > 1, the default pm_register will not able to get notification from domain 1 .. any more, have to change to the new pm_domain_register API.
Benefit
Can make each power domain isolated. less confused when we have more powerdomain and do register action.
Also make each domain only take lock once is enough, don't have to care about global pm callback reg lock.
For more pm domain case, and SMP case, make more optimize possible.
Testing
CI & specific product code R&D self test done.

@jasonbu I think these modifications could be reflected in our documentation: https://nuttx.apache.org/docs/latest/components/drivers/special/power/pm/index.html

Agreed, update the document as a append commit, will it work?


OnMembersRemoved event is not supported

How do I simulate user leaving conversation?

Hi @turbozapekanka
To answer your question, you can send activities to your bot in an Emulator conversation using the Direct Line API.
Step 1: In Emulator
Make a note of the first two addresses that appear in the log panel, and pick one of them to use as the host for your Direct Line API calls (they both worked for me, so I'm not sure what differences you might encounter between them):

Once the Emulator connects to a conversation with your bot, click on the "POST" button on this line:
[10:51:28] POST 200 directline.conversationUpdate

This will show you the channel account objects that you can put in your activity's JSON

Now click on the "201" button on this line:
[10:51:28] POST 201 directline.startConversation

This will show you a conversation ID that you can use to form the path of your request. You will also see a token like "2" but the token isn't actually necessary because you can make unauthenticated requests to the Emulator's endpoint.
Step 2: In Postman
Make a POST request to this endpoint:
<YOUR_HOST>/v3/directline/conversations/<YOUR_CONVERSATION_ID>/activities

The JSON body should look something like this:
{
  "type": "conversationUpdate",
  "membersRemoved": [
    {
      "id": "ec4ed115-68cc-472e-ae03-08c32f8f6fab"
    }
  ]
}

Does this answer your question? I'm going to unassign myself from this issue so the team will see it in case you meant this as a feature request rather than a how-to question.

Yes this answers my question, thanks!


All xml invalid with 1.12.0

XML that worked with 1.11.0 is invalid with 1.12.0.
settings.idp_sso_target_url is now nil where it was set previously.
Tried XML from Okta and Azure.

git blame tells me I should use idp_sso_service_url instead.

In that commit, I introduced the idp_sso_service_url and idp_slo_service_url
but if settings.idp_sso_service_url is not set, it should be used the value of settings.idp_sso_target_url
      def idp_sso_service_url
        val = nil
        if @idp_sso_service_url.nil?
          if @idp_sso_target_url
            val = @idp_sso_target_url
          end
        else
          val = @idp_sso_service_url
        end
        val
      end

how the parameters of your settings look like?

Have the same issue:
IDP_METADATA = <<~XML
  <?xml version="1.0"?>
  <EntityDescriptor xmlns="urn:oasis:names:tc:SAML:2.0:metadata" entityID="https://app.onelogin.com/saml/metadata/100500">
    <IDPSSODescriptor xmlns:ds="http://www.w3.org/2000/09/xmldsig#" protocolSupportEnumeration="urn:oasis:names:tc:SAML:2.0:protocol">
      <KeyDescriptor use="signing">
        <ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
          <ds:X509Data>
            <ds:X509Certificate>MIICWDCCAcGgAwIBAgIBADANBgkqhkiG9w0BAQ0FADBIMQswCQYDVQQGEwJ1czET
  MBEGA1UECAwKQ2FsaWZvcm5pYTERMA8GA1UECgwIVGFsa2FibGUxETAPBgNVBAMM
  CFRhbGthYmxlMCAXDTE3MTAxODEwNTEwMloYDzIyOTIxMjE1MTA1MTAyWjBIMQsw
  CQYDVQQGEwJ1czETMBEGA1UECAwKQ2FsaWZvcm5pYTERMA8GA1UECgwIVGFsa2Fi
  bGUxETAPBgNVBAMMCFRhbGthYmxlMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKB
  gQDSPK+387wuuvtUfboKA08upd/FGQk6raq8Z2XbJ898241e2plVrpayhd8yqxvI
  RZfgo+PKXDjctx71IG/nYE9sH1i0TwPsg0Darh/l9g+sevSE7XPGuXk+tSSAg31k
  bHm0u2fuKK9tKudJwVOC2M0NGiuIu4LJLSYtk1GUcGUhxQIDAQABo1AwTjAdBgNV
  HQ4EFgQUd94eOMal7K/1VZJL9EK8LAvmUIowHwYDVR0jBBgwFoAUd94eOMal7K/1
  VZJL9EK8LAvmUIowDAYDVR0TBAUwAwEB/zANBgkqhkiG9w0BAQ0FAAOBgQAzpO35
  AvCiikfh6OYTkSPRdAWwq6A7j1CWYAj9mjAnO/meR2/z6DUlmb4v5S2mKirwDoFc
  iUjIuk8Ja2PHUArYilDVMeH+Hmss+Eb3AYpPHTFqnvvlyKMdtSQJr9I34ozz68HA
  0TF5cOW11slBJwgIGDh+Nvw9bx62G3BWoAkSFw==</ds:X509Certificate>
          </ds:X509Data>
        </ds:KeyInfo>
      </KeyDescriptor>
      <SingleLogoutService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect" Location="https://talkable-dev.onelogin.com/trust/saml2/http-redirect/slo/100500"/>
      <NameIDFormat>urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress</NameIDFormat>
      <SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect" Location="https://talkable-dev.onelogin.com/trust/saml2/http-redirect/sso/100500"/>
      <SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST" Location="https://talkable-dev.onelogin.com/trust/saml2/http-post/sso/100500"/>
      <SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:SOAP" Location="https://talkable-dev.onelogin.com/trust/saml2/soap/sso/100500"/>
    </IDPSSODescriptor>
  </EntityDescriptor>
XML

parser = OneLogin::RubySaml::IdpMetadataParser.new
settings = parser.parse_to_hash(IDP_METADATA)
puts [OneLogin::RubySaml::VERSION, settings[:idp_sso_target_url]].inspect

=> ["1.11.0", "https://talkable-dev.onelogin.com/trust/saml2/http-redirect/sso/100500"]
=> ["1.12.0", nil]

so the use of settings.idp_sso_target_url should still be compatible
how the parameters of your settings look like?

Seems to be a breaking change somewhere.
With 1.11.0 idp_sso_target_url was set, nothing else changes other than moving ruby-saml to 1.12.0 idp_sso_target_url is nil. (idp_sso_service_url returns the correct value though).
I guess the issue is you never set idp_sso_target_url when parsing (https://github.com/onelogin/ruby-saml/commit/6f8046ed9e8077beb882e1342c6a7283fc007286#diff-bb70d9aec12c5d4cf09ccead0b97fc3b31add1ad737d0eb5f1554079436be68bL213)

I gonna investigate

The parse_to_hash method now does not return the SSO URL inside the idp_sso_target_url and instead, returns it as idp_sso_service_url.
But the parse method that uses parse_to_hash will build the settings with this dict and initialize the Settings properly and the toolkit work as before, since now accepts IDP SSO URL endpoint value to be provided using idp_sso_target_url or idp_sso_service_url
@baelter, @zhuravel, @JCB-K  were you using directly parse_to_hash and then initializing the settings in a way that you were trying to retrieve directly the idp_sso_target_url index from the dict?

idp_metadata_parser = OneLogin::RubySaml::IdpMetadataParser.new
settings = idp_metadata_parser.parse saml_xml
settings.idp_sso_service_url.nil? # => false
settings. idp_sso_target_url.nil? # => true

Yes, it fills idp_sso_service_url which is the parameter that will be used by the toolkit with preference over idp_sso_target_url
https://github.com/onelogin/ruby-saml/blob/master/lib/onelogin/ruby-saml/settings.rb#L79

Yea, so breaking change. behaviour of idp_sso_target_url changed.

The toolkit keeps working with the change, so I don't consider there was a behavior change at the toolkit level.
If you are using only the IdpMetadataParser class, I agree there was a behavior change and you need to use
idp_sso_service_url and not idp_sso_target_url on 1.12.0
I could implement parse_to_hash to retrieve both, idp_sso_service_url and idp_sso_target_url, but that also opens the possible issue of having not the expected final value on the Settings, if a change is done after building the dict on idp_sso_target_url and not idp_sso_service_url which has preference.

Okey, all good. Closing this now that we have got it documented for others how run into this.

This change wasn't mentioned in the changelog :S


Upgrade visibility of classes, members and fields to allow a custom parquet reader implementation (Issue #296)

Fixes
Issue #296
Description
Expected behaviour
Certain classes, enums, methods and properties are required to be used if a custom Parquet Reader class needs to be implemented. Some of them are as follows
Thrift/FileMetaData.cs
Thrift/ColumnChunk.cs
Thrift/RowGroup.cs
Parquet/ParquetActor.cs
Parquet/File/ThriftFooter.cs
Parquet/Data/Schema/Field.cs
Parquet/Data/Schema/MapField.cs
Parquet/File/Values/Primitives/Interval.cs
Currently, these are marked as default or internal and need to be made public for them to be accessed outside of the dll.
Actual behaviour
A custom parquet reader cannot be implemented as the required classes are not public.
Test
Added a simple test to demonstrate the use

In the parquet loader implementation that we are working on we don't want / can't store the entire DataSet in memory but want to read the data incrementally, row by row. We also need to buffer an optimum amount of each column chunk so that we don't access the file everytime we access a new row. As such we need access to internal classes. Would you prefer removing the thrift internal classes and adding a reference to a Thrift Nuget package?

@dwaijam there is no way to access parquet data by rows as it's a columnar format. At the moment you can chunk data by passing Offset and Count in ReaderOptions, will that work?
With regards to Thrift, there is no package available which won't introduce tons of conflicting and unused dependencies on asp.net core, therefore we had to embed only requred parts of Thrift code. Unfortunately Apache community doesn't care/doesn't have enough skills in .NET to confirm PR from us to make only core parts of Thrift available.
With regards to optimal read performance, we are moving from the idea of row-based DataSet in next major version and introduce column streaming instead. We'll still keep it easy to use by allowing to serialise classes directly to Parquet format with some MSIL code generation which makes it super fast, but it's still a few months away from now.

Yes, ofcourse its a columnar format but we need to implement a RowCursor interface that has a MoveNextRow() method which will be called from upstream for working with parquet files. This needs to be very fast and hence we are caching each rowgroup i.e. all its columnar chunks for only a subset of the columns. Using the ReaderOptions on ParquetReader will be a very heavy operation as it calls Read() method which instantiates a bunch of classes to begin reading. For my implementation, I am instantiating only ColumnarReader object for each column chunk as I move from 1 row group to the next and that's about it. This helps to make it run faster.
I have used a reference to the Apache Thrift nuget package that you had created - https://www.nuget.org/packages/Apache.Thrift/ and that works for me.

@dwaijam the biggest overhead in this version is boxing/unboxing which is what we are working on at the moment. This will be solved by better data structures and generating MSIL on the fly.  ParquetReader doesn't actually create many classes, it will be the least concern in performance.  You can also limit the columns it reads by passing their names in ReaderOptions. I'm not sure opening internals is a good path for this library, it's opening a potential worm hole of instabilities. Is it possible to create something like a Read overload that returns only the data you need?

@dwaijam I think @aloneguid is right. Opening the internals is a bad idea. Parquet is mainly used for big data and columnar distribution. Primary use case is to forego unwanted columns in the execution plan. This implementation is the right one.


Switch to PSR-4 and remove autoload

Unless I don't see the real use of src/Identicon/autoload.php?

Can you please squash your commits. Thx

Can you please rebase this PR and squash your commits.

Rebased and squashed @blaugueux

Good job 👍