Failed to load configured propagator tracecontext error when using Opentelemetry

Nuitka version, full Python version, flavor, OS, etc. as output by this
command.

python -m nuitka --version

1.4rc12
Commercial: None
Python: 3.8.10 (tags/v3.8.10:3d8993a, May  3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)]
Flavor: CPython Official
Executable: C:\Soultion-Applications\test\.venv\Scripts\python.exe
OS: Windows
Arch: x86_64
WindowsRelease: 10
Version C compiler: C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30037\bin\HostX64\x64\cl.exe (cl 14.2).

How did you install Nuitka and Python

pip installed the actory version in a venv.

The specific PyPI names and versions

python -m pip freeze

anyio==3.6.2
asgiref==3.6.0
click==8.1.3
colorama==0.4.6
Deprecated==1.2.13
fastapi==0.89.1
h11==0.14.0
idna==3.4
Nuitka @ https://github.com/Nuitka/Nuitka/archive/factory.zip
opentelemetry-api==1.15.0
opentelemetry-instrumentation==0.36b0
opentelemetry-instrumentation-asgi==0.36b0
opentelemetry-instrumentation-fastapi==0.36b0
opentelemetry-instrumentation-logging==0.36b0
opentelemetry-sdk==1.15.0
opentelemetry-semantic-conventions==0.36b0
opentelemetry-util-http==0.36b0
orderedset==2.0.3
pydantic==1.10.4
PyYAML==6.0
sniffio==1.3.0
starlette==0.22.0
typing-extensions==4.4.0
uvicorn==0.20.0
wrapt==1.14.1

Also supply a Short, Self Contained, Correct, Example

import asyncio
import platform

from fastapi import FastAPI
from opentelemetry import trace
from opentelemetry.instrumentation.fastapi import FastAPIInstrumentor
from opentelemetry.instrumentation.logging import LoggingInstrumentor
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider

from typing import List, Optional

import fastapi
from fastapi.middleware.cors import CORSMiddleware

def initialize_telemetry_for_fast_api(app: FastAPI, service_name: str) -> None:
    resource = Resource(attributes={"service.name": service_name})
    trace.set_tracer_provider(TracerProvider(resource=resource))
    LoggingInstrumentor().instrument()
    FastAPIInstrumentor.instrument_app(app)

def create_app(cors_origins: Optional[List[str]] = None):

    app = fastapi.FastAPI(
        title="Example API",
    )

    app.add_middleware(
        CORSMiddleware,
        allow_origins=cors_origins or [],
        allow_credentials=True,
        allow_methods=["*"],
        allow_headers=["*"],
    )

    @app.get("/health")
    async def health():
        return "The API server is healthy."

    return app

cors_origins = ["*"]
app = create_app(cors_origins)
host = "<IP_ADDRESS>"
port = 8000

kwargs = {
    "app": app,
    "host": host,
    "port": port,
    "reload": "True",
}

if platform.system() == "Linux":
    import uvicorn

    uvicorn.run(**kwargs)
else:
    from asyncio.windows_events import ProactorEventLoop

    from uvicorn import Config, Server

    class ProactorServer(Server):
        def run(self, sockets=None):
            loop = ProactorEventLoop()
            asyncio.set_event_loop(
                loop
            )
            asyncio.run(self.serve(sockets=sockets))
    config = Config(**kwargs)
    server = ProactorServer(config=config)
    server.run()

Provide in your issue the Nuitka options used

nuitka --standalone .\test_otel.py

Failed to load configured propagator `tracecontext`
Traceback (most recent call last):
  File "C:\SOULTI~1\test\TEST_O~1.DIS\opentelemetry\propagate\__init__.py", line 137, in <module opentelemetry.propagate>
StopIteration
Traceback (most recent call last):
  File "C:\SOULTI~1\test\TEST_O~1.DIS\test_otel.py", line 6, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "C:\SOULTI~1\test\TEST_O~1.DIS\opentelemetry\instrumentation\fastapi\__init__.py", line 181, in <module opentelemetry.instrumentation.fastapi>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "C:\SOULTI~1\test\TEST_O~1.DIS\opentelemetry\instrumentation\asgi\__init__.py", line 205, in <module opentelemetry.instrumentation.asgi>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "C:\SOULTI~1\test\TEST_O~1.DIS\opentelemetry\instrumentation\utils.py", line 26, in <module opentelemetry.instrumentation.utils>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "C:\SOULTI~1\test\TEST_O~1.DIS\opentelemetry\propagate\__init__.py", line 137, in <module opentelemetry.propagate>
StopIteration

Does not work in stable release of nuitka as well.
Works fine when running using python normally.
This might be of some help: https://github.com/orgs/pyinstaller/discussions/6853
Made this sample code based on something I am working on. Need to use opentelemetry in it.

If you are available to do a PR, this can be fixed by adding to the yaml configuration, described here: https://nuitka.net/doc/nuitka-package-config.html
And here is more of a walk through style post: https://nuitka.net/posts/nuitka-package-config-kickoff.html
Here is a similar PR for another package: https://github.com/Nuitka/Nuitka/pull/1993

I tried fix it. But my nuitka version and versions opentelemetry are slightly different
Nuitka
1.5rc12
Commercial: None
Python: 3.10.9 (main, Jan 22 2023, 21:45:35) [Clang 14.0.0 (clang-14<IP_ADDRESS>)]
Flavor: pyenv
Executable: /Users/yragantron/Library/Caches/pypoetry/virtualenvs/test-fastapi-sA9Sk0hI-py3.10/bin/python
OS: Darwin
Arch: x86_64
Version C compiler: /usr/bin/clang (clang).

My dependencies
python = "^3.10"
fastapi = "^0.85.0"
uvicorn = "^0.18.3"
nuitka = "^1.4.8"
opentelemetry-api = "^1.10.0"
opentelemetry-sdk = "^1.10.0"
opentelemetry-exporter-jaeger = "^1.10.0"
opentelemetry-instrumentation = "^0.29b0"
opentelemetry-instrumentation-redis = "^0.29b0"
opentelemetry-instrumentation-fastapi = "^0.29b0"
opentelemetry-instrumentation-httpx = "^0.29b0"
opentelemetry-instrumentation-sqlalchemy = "^0.29b0"
opentelemetry-instrumentation-botocore = "0.29b0"

I use same code and I got same error.
In the module opentelemetry.propagate there is such a piece of code
environ_propagators = environ.get(
    OTEL_PROPAGATORS,
    "tracecontext,baggage",
)

for propagator in environ_propagators.split(","):
    propagator = propagator.strip()
    try:
        propagators.append(  # type: ignore
            next(  # type: ignore
                iter_entry_points("opentelemetry_propagator", propagator)
            ).load()()
        )

It imports this class opentelemetry.trace.propagation.tracecontext.TraceContextTextMapPropagator and this opentelemetry.baggage.propagation.W3CBaggagePropagator
I pulled nuitka from the develop and added to the config standard.nuitka-package.config this code
- module-name: 'opentelemetry.propagate'
  implicit-imports:
    - depends:
        - 'opentelemetry.trace.propagation.tracecontext'
        - 'opentelemetry.baggage.propagation'

But it didn't solve the problem, I get the same error.

The entry points are just not included. Currently Nuitka only does it when it can statically optimize the code. The above code is not like that. There is also no option to force including it. I guess, wiritng it out, such that Nuitka can see it might help. Your yaml code is good we can can it.
This is from Nuitka changelog:
-  Compile time optimize ``pkg_resources.iter_entry_points`` too, such
   that these can be used to resolve plugin modules, which helps with
   adding support for ``agscheduler`` package plugins. Note that these
   still need to be manually included with ``--include-module`` but now
   that works.

If you used append_code and put ```
iter_entry_points("opentelemetry_propagator", "tracecontext")
iter_entry_points("opentelemetry_propagator", "baggage")

there, it would work. But I am not entirely sure. If it does not, that would be a bug by itself in Nuitka, since the contained entry points from that code should be exposed to the generic search for sure.

I didn't succeed with this
iter_entry_points("opentelemetry_propagator", "tracecontext")
iter_entry_points("opentelemetry_propagator", "baggage")

But I solved it by directly importing these classes.
Here my config https://github.com/Nuitka/Nuitka/issues/2108

@YraganTron that's great. I think I will be able to condense that. Apparently it's not the only issue, but it's great progress. I will continue in #2108 then.

I am trying this example right now out, aiming to include this in a 1.7 hotfix maybe, this is all sorted out by you already, or so I hope.

So, I got this to work. We added a feature --include-distribution-metadata that in conjunction with hidden imports would solve it, but I enhanced the Yaml format to do it, and added it there, so it now works out of the box. Need to check the related issue now, if there is anything different there.

This is part of the 1.7.1 hotfix I just made.


Remodel Hello messages as version negotiation and transform messages based on negotiated version

This is an attempt at relaxing the version requirements of our wire protocol. What we have currently in master does not work  (for 0.4.7, sorry for not testing this better first :grimacing: ).
Here is screenshot of this branch in action:

bors r+


concat" not yet implemented

https://qlever.cs.uni-freiburg.de/wikidata/U8lVrR

The CONCAT expression has been implemented for a while now.

I think this issue needs a positive example then and might need to be reopend for the time being. The example query above still fails.

Just an observation: The query now fails for other reasons: 1. For the use of Blazegraph's label service 2. For trying to group by a variable defined via SAMPLE.

Ok, the example item has been merged. The query works if its id is updated: https://qlever.cs.uni-freiburg.de/wikidata/nWiLr6


Reinvestigate support for OpDebugTypePointer

SPIRV recently adds an extension to relax restrictions for forward declarations in ExtInst, this should allow us to define forward pointer types in debug insts.
We should reinvestigate if this is now allowed, and report back remaining blockers for us to generate debug info for general pointer types.
https://github.com/KhronosGroup/SPIRV-Headers/pull/434
https://github.com/KhronosGroup/SPIRV-Tools/pull/5698

It appears that the implementation for the forward declaration hasn't been released.
On Vulkan-Header side, the commit, "ff2afc3", was submitted on May-30-2024.
And the latest release was on Mar-27-2024, which was "vulkan-sdk-1.3.283".
On Vulkan-Tool side, the commit, "6a2bdee", was submitted on Jun-4-2024.
And the latest release was on Apr-22-2024, which was "vulkan-sdk-1.3.283".
I was able to get ToT and build them locally, which unblocks me for the task.
But I wonder if this is little too early to make changes based on a feature not released on vulkan-sdk side.

We can consider it released as long as the feature is available from tot.

Well.. it turned out the OpTypeForwardPointer is for something else.
And I also found that OpExtInstWithForwardRefs can be used with DebugTypeComposite from one of test cases.
%33 = OpExtInstWithForwardRefsKHR %void %1 DebugTypeComposite %5 %uint_0 %31 %uint_3 %uint_7 %32 %5 %uint_0 %uint_3 %34

It seems like there was a build break on spirv-header repository and spirv-tools.
In order to use OpExtInstWithForwardRefs, we need little more commits than include fixes.
Spirv-header:

commit 2acb319af38d43be3ea76bfabf3998e5281d8d12 (HEAD, origin/main, origin/HEAD, main)
Author: Kévin Petit<EMAIL_ADDRESS>Date:   Wed Jun 12 16:41:14 2024 +0100
SPV_ARM_cooperative_matrix_layouts (#433)

Spirv-Tools:

commit ce46482db7ab3ea9c52fce832d27ca40b14f8e87 (HEAD)
Author: Nathan Gauër<EMAIL_ADDRESS>Date:   Thu Jun 6 12:17:51 2024 +0200
Add KHR suffix to OpExtInstWithForwardRef opcode. (#5704)
The KHR suffix was missing from the published SPIR-V extension.
This is now fixed, but requires some patches in SPIRV-Tools.

Currently this task is blocked because I cannot figure out how to build the executables for "aarch64-linux" platform.
Waiting for a help from @expipiplus1 for the problem described on the PR comment.
https://github.com/shader-slang/slang/pull/4527#issuecomment-2204091088


Add select multiple for dropdown form field

It would be very helpful to have either the ability to toggle the drop down list into a multi-select, or another form element representing multi-select.
Some forms I'm trying to create need multi-selects (and have too many choices to be represented as checkboxes). I ended up doing this manually, but it would be a helpful addition.

@dwasyl I agree. I know there's a component out there in a different repository, but why not just use the existing chooserpanels? What if you want to add several snippets/pages/images or some such in an association table? You already have a great interface to deal with this. For now it just works exclusively with one to many and one to one relationships. I think this is related https://github.com/wagtail/wagtail/issues/2203. It discusses a workaround of placing a chooserpanel in an inlinepanel. Imo the multi select widgets are not scalable enough out of the box to work with large sets of options. The Chooser Panels, however, are very useful for this. So I'd like to suggest allowing users to take advantage of the chooser panels in many to many relations like they can in regular foreign key relationships. Especially seen as how the bulk of the work (the chooser panels themselves) are already present in the CMS.

@timmysmalls If I'm reading correctly, this issue is about forms created with the wagtail.contrib.forms form builder, not forms within the Wagtail admin.

I believe #3020 solved it for forms, so it could be closed?

@zerolab Ah, good spot!


Fix exaggerated terrain normals.

Bug found on the forum: https://groups.google.com/forum/#!topic/cesium-dev/hQ58SIkZHhI
Another reason we need to add tests #3902.

@kring since you were part of the forum discussion, can you merge this?  Thanks.


Status of this Project?

Can anyone share thoughts on the state of this project?
This project contains a lot of amazing work and I'd love to be able to use it in a bunch of Ruby/Rails projects. However, there are a few issues:

This project looks somewhat abandoned
Lack of support for Ruby > 2.4. Is a blocker for us #505
Several dependencies on projects that also appear to be somewhat abandoned.

Questions:

Is there a team interested in maintaining this project?
Is there a plan to fix dependencies?
Is there a plan to support Ruby > 2.4?

I'd be happy to contribute ideas and time on plotting a path forward but would love to be part of team that is driving towards that.

@v0dro  @zverok I think you'd be best placed to answer this. Fair to say neither of you are working on this project anymore?

To the best of my knowledge, the answer is as follows.
Short answer: yes, seems nobody to work on the project actively at the time.
Long answer: The library was created by @v0dro and since some point, evolved by contributions of SciRuby's students mostly. I was involved with the library development at some point (mostly as a SciRuby mentor), but since I am retired from SciRuby, I am not anymore.
Unfortunately, I lost touch with Sameer (@v0dro) since then, the last I know he was working on Rubex, which also seems dormant now. Considering his latest blog entries, it seems that he (like me before that) became disillusioned in prospects of gaining attention to Ruby from the scientific community, and switched to Python, but it is just my assumptions.

Hey @zverok, thanks for the update, and all of your work on SciRuby.
Since it sounds like Daru is in a holding pattern, I put together a take on data frames for Ruby called Rover. It uses Numo internally for performance (similar to how Pandas uses NumPy), and I've moved Prophet to it, as well as added support to XGBoost and LightGBM. I think there's a lot more to do, but hopefully it provides a good starting point. Would love to hear others' thoughts on the topic - feel free to open an issue to discuss.

@ankane I'm curious, would you mind giving any more background - where you see the rover project going, why the start from scratch approach and how you think it compares to Daru (and Pandas) currently?

@baarkerlounger The end goal is to make data analysis and machine learning as enjoyable as possible in Ruby. The main difference from Daru is its based on Numo, for both performance and easy integration with Rumale.

@ankane,
following your work I'd like to ask if it's any feasible to join forces and leverage Daru's codebase in your Rover project?

@arbox I'm happy to chat on specific features / ideas, but want to give this a fresh look (and think there's limited benefit to reusing the codebase).

My favorite Ruby's fancy DataFrame looks like this.
It's a bit like Daru at first sight. but, it's much faster because it uses numo-narray and Apache Arrow on the backend. The features are simple and it looks clean.
Let's consider modifying Daru. In this case, you need to remove a lot of Daru's code.
That decision may be difficult for a non-founder to make because it may destroy compatibility.
So it's reasonable that someone starts new projects.
Ruby is object-oriented and I think it's hard to work with functions.
There should be a variety of works and ideas.
The mountains won't get higher if they don't have a wider base.

Here is my take. There is a lot to like about this project. The code is very clean and has some nice abstractions. However, as I stated i the initial question there are some challenges today in terms of dependencies and support for Ruby > 2.4.
@ankane I'd disagree with you that there is little to be benefit from the existing code base. There are a lot of features you don't have in Rover yet. Plus the implementation you have at the moment is strongly coupled to Numo.
@kojix2 We can solve the problems you mention. I don't think its removing a lot of code (I share a desire for this direction but don't believe its necessary: https://github.com/kojix2/chai) Also I'm sure we can solve 'non founder' ownership and breaking changes it happens all the time.

The mountains won't get higher if they don't have a wider base.

I agree with this but lets make sure we build a mountain and don't end up with a bunch of half finished hills 😄
IMO there is value in a Ruby implementation of DataFrames. For me its a convenient way to transport and mutate 'matrix' like data. It's not required that it be blazingly fast of have amazing scale in the data sets it runs. Nor IMO is it a requirement that you be able to do everything in a Jupyter notebook. At a certain point if you want all those features the simpler leap is to go you Python and Pandas.
However, I'd love to see this project get a bit more life into it and to update it to make it more usable. My wish list would be:

Add a configuration mechanism for the underlying "Array" storage.

Remove the core dependencies on NMatrix & GSL. Perhaps we publish them as separate Gems for backwards compatibility. But frankly they're so old and unsupported I'm not convinced
Add an implementation of Array storage backed by Numo.

Add a configuration mechanism for plotting library

Remove the core dependencies on gruff publish them as gems

Update the DataFrame API to much more closely match Pandas

Rename some functions
Add missing features (I have some as monkey patches)

Cleanup the Ruby backports and other hacks

I don't think any of this is particularly difficult. There is a pretty long list of features to close the gap to Pandas but they can be easily done one at a time with easy to review PRs.
I'd be happy to contribute I'd be happy to lead but I don't have the bandwidth do it alone. If there are others willing to help we can do it. @kojix2 @ankane @arbox @baarkerlounger and others are you up for giving it a go?

Just to add one more variety to already existing list of opinions!
I believe there are two, not really related questions about "dataframe implementation" in Ruby:

Interface (as in API)
Implementation (including performance and most of the "features")

My interest was always around (1): how may we create a generally useful dataframe, which would be obviously helpful for everyone in the community (in fact, my involvement with Daru started with this 4-year old blog post, and that was the direction of my work in a maintainer role). "Generally useful", for me, also implies "naturally playing" with all Rubyist intuitions, emerging from Enumerable, Hash, Array etc. I once dreamed of ever-present "dataframe API" that could've suite "backends" as different as SQL db, CSV file or Apache Arrow. And I imagine that this work might also lead to accepting some useful "micro-metaphors" as useful idioms for other collections and libraries.
For what I see, most of the other people involved in Daru's or other dataframe libraries' development mostly focused on datframes that would be a) fast for large data processing; b) include wast set of features (with emphasis on "it is possibe/done one method" even at a price of API consistency and knowledgeability); c) frequently, look familiar rather for "those who already worked with Pandas" than "those who already worked with Ruby Array/Hash".
I'd imagine some ideal several layers project, with:

one layer focused on relatively small and consistent Ruby DataFrame API,
another on "just features", lots of additional methods that can be implemented in terms of said API,
even another on backends for said API (even as different as "DataFrame interface to PostgreSQL table", which will convert all API calls into SQL statements)

I might see that there are some parts of Daru that might be "scraped" for "layer 2" (various features), but in terms of API, I never really liked it, and don't believe it can be "gradually improved" (I had tried it for several monthes in 2018 and finally just gave up).
TBH, I even have my own (unfinished and unpublished) "completely new DF project" with API I really love, but ... I don't see an audience for it anymore.

@zverok Thanks for sharing your perspective. I share a very keen desire for #1 too. That seems fundamental. As I said if you really care about performance/scale then IMO you're going to get better luck by using Pandas. It's likely to always be faster etc. That doesn't stop us from having "fast" options in Ruby but that speed shouldn't come at the cost of API clarity.
I think there is a balance to be struck between having a similar API to Pandas and feeling like Ruby.  There is huge value in a Ruby DataFrame that at a minimum has the same features, with the same names as Pandas. Perhaps this is a simple as naming:

Panda Series vs Daru Vector
Pandas concat vs Daru union
Some of these are probably historical accidents of Daru mimicking an older version of the Pandas api and not being updated. I'd hope that simple naming things would be easy to solve (and provide backwards compatibility within reason)

Having used Daru DataFrames in a project in anger I haven't felt the same sense of "its not Ruby" that you suggest. I'd like to understand that more. So two followup questions:

Can you share some examples of "it is possible/done one method" in Daru? I agree they exist but I'm curious which ones stand out to you.
Can you share some places where Daru "doesn't feel like Ruby" and a sketch of how we could make and API that is better suited to Rubyists?

@jonspalmer It is hard for me to answer the questions in details: both technically, because I haven't been involved with Daru for ~2 years now, and ethically, as a lot of its features either was implemented by Sameer, who is the initial author or by students of SciRuby we were mentoring, so pointing at any particular feature would be also pointing at a person who invented/implemented it.
Also, I have changed my mind about "what's best" several times, and I am currently even not sure whether "really useful" solution will be one class (DataFrame) or some family of classes (like, "table which is first and foremost for navigation/presentation" and "table that is first and foremost for complex math").
I suspect that the base API design decisions are very fundamental and it is hard to change them once settled. Between those, I'd say: indexing/enumerating; what are "indexes" (one or two of them, nesting of indexes and other behaiors), desired level of "matematicity" of the DF (whether df1 + df2 is "sum of each pair of elements" or more like Array#+), what's the policy of supported data types, copy-on-write semantics, etc., etc.

@zverok

Also, I have changed my mind about "what's best" several times, and I am currently even not sure whether "really useful" solution will be one class (DataFrame) or some family of classes (like, "table which is first and foremost for navigation/presentation" and "table that is first and foremost for complex math").

IMO the answer on this is very clear - one class. The Panda's api is strong in this regard. From my perspective there isn't any need for more than two main objects DataFrame and Vector (and perhaps we call it Vector because that's what Ruby's Matrix class uses vs Panda's Series). Having the arithmetic and statistical functions as first class methods on DataFrame and Vector is simple and doesn't get in the way if you don't need it.
For things like df1 + df2 it sort of doesn't matter. Provided that there is a clear method with a good name that this + operator is an alias for its just syntactic sugar. I don't believe that any particular choice is more Ruby like than any other. We should pick one, ideally Panda's like, and document it. Similarly things like copy-on-write you just need to decide, document and provide an option to change the default. This isn't really a Ruby or not-Ruby like concern it's just picking an API.
WRT indexing - I'm not sure there is a Ruby right or wrong answer. You could argue that we should mimic the dual index treatment in the Ruby Matrix class but honestly its not obviously right either. Consistency with that API doesn't bother me too much.
To summarize I don't see these as big challenges. We could just decide and start moving things to a better place.

@jonspalmer I believe you seriously underestimate the design space (and how design decisions affect the library's usability). Let's stick just to one example of two parts (it is just to illustrate the point). Imagine you have data shaped this way:
         Q1   Q2   Q3 Total
      ---------------------
Smith | 100  100  150   450
Jones | 200  180  200   580
Khan  | 150  180  180   510
Total | 450  470  530  1450

Question 1 (a more simple one, but very first in design, which every DF designer handles somehow): how do you address "Total" column and "Total" row? Everybody tends to start with...
df['Total'] # column or row?

...and there are many different ways to handle it :)
Now, to dive into some details. (For the sake of simplicity, let's say we decided that df.col['Total'] would be a way to address columns.)
How do you express "add 10 to (each value of) column Total"?

df.col['Total'] += 10 is "obvious" choice, but it is performed as df.col['Total'] = df.col['Total'] + 10, e.g. "produce intermediate array, then replace the column in dataframe", which, without some clever internal optimizations, could be very ineffective.
df.col['Total'].map! { |val| val + 10 } is "just regular Ruby", but (again without some complicated tricks) it is bound to perform row-by-row, effectively prohibiting "backend optimization" (e.g. if internal Vector is some fast C numeric array which has optimized operation "increase each value"), or complicated backends (like those that really will convert it into UPDATE table SET Total=Total+10 instead of performing literally)
some df.col['Total'].increase(10) kinda "solves" the two problems above, but now it stops to look "elegant" (implying that there would be a custom method for every math operation, and something like ((value + 10) / 2).round(3).clamp(0..100) will be inexpressable at all)
...and so on :)

I am honestly not sure the "whatever, let's design it someway and that will be it" is the approach that will lead anywhere useful. In fact, several existing (and incompatible) dataframe libraries that Ruby has (besides Daru and new kid Rover, there are ... some: 1, 2, 3 (at one point endorsed by Ruby Association), 4, 5, etc.) clearly demonstrate that.

Question 1 how do you address "Total" column and "Total" row?
df['Total'] # column or row?

Answer:
a) you just have to choose. There is no right or wrong.
 That decision should inform the answers. So for Pandas and Daru the right answer is "Column" because Columns are the first class object and rows are not. It's therefore natural for the 'first index' to represent columns (which also encourages efficient data access)

"add 10 to (each value of) column Total"

df.col['Total'] += 10 is typically syntactic sugar for df.col['Total'] = df.col['Total'] + 10 which it turn is syntactic sugar for df.col['Total'] = df.col['Total'].add(10). So now there are two things to consider:

Is the column assignment efficient?
If columns are the first class object then yes this should be a cheap and simple operation.
Is adding 10 to the column efficient?
You would certainly expect the Column/Series/Vector class to make sure that it is.

df.col['Total'].map! { |val| val + 10 }

a) most APIs would expect df.col['Total'] to produce a copy of the column. So this example doesn't fix the assignment problem
b) the column methods map!(&block) and add(value) are both great Ruby methods but they are not the same. map and map! assume you're going to do something different with each element of the column as such it can't/shouldn't be optimized to to be efficient if you happen to do the exact same thing to each element. If you want that use the add method.

df.col['Total'].increase(10)

we have the column copy problem again. You're increase method is just the existing add method on Daru::Vector. Seems fine to me.
to your larger example:
col = df.col['Total']
newCol = ((col + 10) / 2).round(3).clamp(0..100)
# or
newCol = col.add(10).div(2).round(3).clamp(0..100)
# is this more efficient?
newCol = col.map { |v| ((v +10)/2).round(3).clamp(0..100) }

I don't know which would be faster/more efficient. Whatever the answer it would need to be carefully measured. The DataFrame API can't anticipate all the use cases it can only provide reasonable, well named building blocks to allow options for the consumer to use to solve their particular problem.
Are there cases where you really, really want to do things 'in place'? Perhaps but the use cases are going to be very specific and the 'right way' to optimize them is going to be very subtle.
To take your example of wanting to "add 10 to Total". You could argue that we really need to manipulate "Total" in place because it's more efficient. However, its more likely that the situation is something like this:
         Q1   Q2   Q3
      ---------------
Smith | 100  100  150
Jones | 200  180  200
Khan  | 150  180  180

df.col['Total'] = df.col['Q1'] + df.col['Q2'] + df.col['Q3']

So now you could say "Hey I really want to add 10 to "Total" in place. It sucks that this is 'so inefficient'"
df.col['Total'] = df.col['Total']  + 10

but that's the wrong problem to go after. It would be way better to simply fix it when you generate 'Total' the first time. (it's user error not API error)
df.col['Total'] = df.col[['Q1', 'Q2', 'Q3']].sum(axis: :column) + 10
df.row['Total'] = df.sum(axis: :row)

Which doesn't require inefficient intermediate columns/rows.
My argument isn't that we should blindly build an API and hope it works out. Instead we should carefully build something that is clear, flexible and consistent that allows consumers to solve their problems. We cannot nor should not expect the API to solve every corner case cleanly or naturally. Specific problem will require specific solutions. The more specific the problem the less likely the solution will be "elegant" but that's an entirely normal and expected tradeoff.
From my perspective a lot of work and real world use cases have gone into the Pandas API and it is powerful, feature rich and natural to use. The choices and design there would be very natural to replicate in Ruby (with the exception perhaps being the Python slice operator being a bit more flexible than Ruby ranges). Daru has made a great initial set of steps to get close to replicating that API. IMO we should continue that work and bring it up to date with the current state or Ruby and Pandas. I'm not totally clear what you are proposing as an alternative?

Status update on daru.
CC: @zverok  @jonspalmer @kojix2
I had a brief e-mail conversation with @jonspalmer and have promised him to update daru on the following points:

Make the gem >2.4 compatible.
Close or merge pending issues and PRs.
Remove the nmatrix dependency and make it a separate plugin.
Release a new gem version.

However, I am currently in grad school and using a lot of low level C++/C/FORTRAN for my research and have therefore lost touch with data analysis. I will happy if someone else is willing to take over the project. As @kojix2 says, having a dataframe with support for Arrow would be great. Relying on Ruby for speed is a bad idea. However all this will require a central point of contact (i.e. a maintainer willing to commit a few hours a week).
My take on the future direction of daru is that we should forget the scientific computing audience and let them be happy with Julia/Python/R. Our real audience should lie in the Ruby community (web dev etc.). This was pointed out by @zverok much earlier and I agree that his course of action would have been appropriate.
BTW Numo has some speed issues due to the data representation that it uses (last I checked was more than 6 months ago) and I'm not sure if they've been resolved. @prasunanand can fill in on this better, I believe.

New daru version has been released and all old PRs have been merged/closed.

@v0dro Following up on this what is the new version that has been released? I don't see any new tags here.

My bad I forgot to tag it. You can see it on rubygems here: https://rubygems.org/gems/daru


Add a list of validators that meet the official GTFS requirements

BlockTripsWithConsistentTypeValidator

Validates that trips that belong to the same block have consistent route types (e.g., bus does not transfer to rail).
Generated notice: BlockTripsWithInconsistentRouteTypesNotice.

FareAttributeAgencyIdValidator

Checks that fare_attributes.agency_id field is defined for every row if there is more than 1 agency in the feed.
Generated notice: MissingRequiredFieldNotice.

FeedHasLanguageValidator

Checks that a feed has language defined in either feed_info.feed_lang or agency.agency_lang. Note that feed_lang is a required field: if there is a feed_info entity, it must have feed_lang set. agency_lang, instead, is optional. The recommended way is to provide feed_lang and omit agency_lang.
Generated notice: FeedHasNoLanguageNotice.

StationUsageValidator

Checks that every station (location_type=1) in stops.txt has child platforms ( location_type=0). Note that a station that has child entrances (location_type=2) or generic nodes (location_type=3) is also reported.
Generated notice: StationWithoutPlatformsNotice.

StopLatLngRequiredValidator

Checks that conditionally required fields stop_lat and stop_lon are set depending on location_type.

stop_lat and stop_lon are required for stations (location_type=0), stops (location_type=1) and entrances (location_type=2) and optional for other types.

Generated notice: MissingRequiredFieldError.

StopTimeInSequenceOrderValidator

Validates that fields arrival_time, departure_time and shape_dist_traveled are non-decreasing along each trip.
Generated notices:

TripWithOutOfOrderArrivalTimeNotice
TripWithOutOfOrderDepartureTimeNotice
TripWithOutOfOrderShapeDistTraveledNotice

TransferFrequencyBasedTripValidator

Validates that trip-to-trip transfers do not involve frequency-based trips.
Generated notice: TransferForFrequencyBasedTripNotice.

TripShapeDistTraveledValidator

Validates that trips specify stop_times.shape_dist_traveled values properly. Please see generated notices for details.
Generated notices:

TripWithShapeDistTraveledButNoShapeNotice
TripWithShapeDistTraveledButNoShapeDistancesNotice
TripWithPartialShapeDistTraveledNotice

Thanks for the ping @asvechnikov2! (and.. wow @aababilov 🎉).
I will have a look this week, looking forward to it 😊

The LGTM.  However, you do need to update RULES.md with the new validation notices you've added.

I'd also call out that issue #1304 is probably still blocking us from getting a clean acceptance test run.

Hello! Sorry it took a while. A few things here:

FeedHasLanguageValidator
Is this mentioned in the spec somewhere? I couldn't find it. Adding this to the spec looks like a breaking change. Maybe something to add in the best Practice and flag as a warning here? Where is this coming from?
StationUsageValidator
Reading the spec, I think stations without any children are okay? Please tell me if you think otherwise
BlockTripsWithConsistentTypeValidator, TransferFrequencyBasedTripValidator and TripShapeDistTraveledValidator
These new validators make a lot of sense! That being said, so explicit spec mention. I suggest we first amend the spec and then merge them. Happy to propose the changes.
FareAttributeAgencyIdValidator, StopTimeInSequenceOrderValidator, StopLatLngRequiredValidator -> LGTM
One question though: would TripWithOutOfOrderShapeDistTraveledNotice replace the existing DecreassingOrEqualStopTimeDistance?

I suggest we split this PR between what's explicitly mentioned in spec VS what is not.
What is explicitly mentioned: we just need the documentation update and we're good to merge.
What isn't: add the clear statements in the spec itself, and then merge


Forward port cleanup

Changes:

Updated build instructions
Added package dependencies
Gazebo version logic

Was this tested on harmonic?

@Ryanf55 What warnings are you getting, can you please open an issue?


Add the CSS scrollbar Properties

Related Issues: #1121, #1449
There are a bunch of scrollbar properties that work with IE9 and Opera1:

scrollbar-arrow-color: Specifies the color of the arrow elements of a scroll arrow - https://msdn.microsoft.com/en-us/library/ms531154(v=vs.85).aspx
scrollbar-base-color: Specifies the base color of the main elements of a scroll bar - https://msdn.microsoft.com/en-us/library/ms531155(v=vs.85).aspx
scrollbar-face-color: Specifies the color of the scroll box and scroll arrows of a scroll bar - https://msdn.microsoft.com/en-us/library/ms531157(v=vs.85).aspx
scrollbar-highlight-color: Specifies the color of the slider tray, and the top and left edges of the scroll box and scroll arrows of a scroll bar - https://msdn.microsoft.com/en-us/library/ms531158(v=vs.85).aspx
scrollbar-track-color: Specifies the color of the track element of a scroll bar  - https://msdn.microsoft.com/en-us/library/ms531160(v=vs.85).aspx

There are also a few vendor prefixed properties for IE9:

-ms-scrollbar-3dlight-color: Specifies the color of the top and left edges of the scroll box and scroll arrows of a scroll bar - https://msdn.microsoft.com/en-us/library/ms531153(v=vs.85).aspx
-ms-scrollbar-darkshadow-color: Specifies the color of the gutter of a scroll bar - https://msdn.microsoft.com/en-us/library/ms531156(v=vs.85).aspx
-ms-scrollbar-shadow-color:  Specifies the color of the bottom and right edges of the scroll box and scroll arrows of a scroll bar - https://msdn.microsoft.com/en-us/library/ms531159(v=vs.85).aspx

Footnotes

1: I haven't been able to find this in Opera's official documentation.

AFAIK, this only works in Presto-Opera, so versions up to and including 12.5, not in versions 15 and up. I also doubt this works on any platform other than Windows, or on Windows 8+. Could easily be tested though :)

-1

What about ::-webkit-scrollbar?

Now available at http://caniuse.com/#feat=css-scrollbar


Cannot find 'aws-crt-cpp-config.cmake' from the 'crt/aws-crt-cpp/cmake' directory

I want to build C++ S3Client using yocto. I am trying to build aws-sdk-cpp using meta-aws. I followed following steps -

I copied the aws-sdk-cpp_1.11.162.bb into my recipes directory.
Ran the following command - $ MACHINE=machine-name bitbake aws-sdk-cpp

I encountered following error in do_configure -
| CMake Error at cmake/AwsFindPackage.cmake:19 (find_package):
|   By not providing "Findaws-crt-cpp.cmake" in CMAKE_MODULE_PATH this project
|   has asked CMake to find a package configuration file provided by
|   "aws-crt-cpp", but CMake did not find one.
|
|   Could not find a package configuration file provided by "aws-crt-cpp" with
|   any of the following names:
|
|     aws-crt-cppConfig.cmake
|     aws-crt-cpp-config.cmake
|
|   Add the installation prefix of "aws-crt-cpp" to CMAKE_PREFIX_PATH or set
|   "aws-crt-cpp_DIR" to a directory containing one of the above files.  If
|   "aws-crt-cpp" provides a separate development package or SDK, be sure it
|   has been installed.
| Call Stack (most recent call first):
|   src/aws-cpp-sdk-core/CMakeLists.txt:586 (aws_use_package)

I also tried to change the SRC_URI from git://github.com/aws/aws-sdk-cpp.git;protocol=https;branch=main \ to     gitsm://github.com/aws/aws-sdk-cpp.git;protocol=https;branch=main \ to account for submodules but that did not help and I got the same error message again.
Another thing I tried was as per the suggestion in error message -
EXTRA_OECMAKE += "\
    -DCMAKE_PREFIX_PATH=crt/aws-crt-cpp/cmake \
"

But this also did not work.
Please let me know how to build aws-sdk-cpp using Yocto.
Please let me know if you need anything for this.

Please ignore this issue for now. I think the above way of using recipe files is wrong.

Hi,
you need to include the whole meta-aws layer in your bblayers.conf not just copy one recipe. If you copy need to copy ALL dependent recipes as well but will lack the comfort of easy upgrades (https://docs.yoctoproject.org/overview-manual/concepts.html#layers).
Let me know if you still need help.
Will close this issue now.


[WIP]Support ENUNUServer0.4.0

Features have been added following the update of SimpleENUNUServer0.4.1 (SimpleENUNU0.3.1).

ENUNU type and feature type have been added. (EnunuConfig)
Enabled f0 to be changed even when the feature type is ‘melf0’. (EnunuRenderer)
Enabled port number to be changed. Enabled the port number to be changed in the preferences in future PRs. (EnunuRenderer,EnunuPhonemizer,EnunuEnglishPhonemizer,EnunuClient,EnunuUtils).

EnunuKoreanPhonemizer now uses a different version of ENUNUServer with complex changes,
Not affected by updates. *Will be integrated in future PRs.
It means that it is backwards compatible.
Test environment.
SimpleENUNUServer0.4.1
SimpleEnunuServer-0.3.0+0.local.9.1.
EnunuPhonemizer.
EnunuEnglishPhonemizer.
ENUNU_KanadeShia_NightScene_SiFiGAN_MEL
ENUNU_Kanadeshia_Dimiwse_typeN
ENUNU_夏目悠李_RMDN_v0.0.3

SimpleENUNU has been updated and will be withdrawn.


Move some of the transaction errors as warnings

Move some of the transaction errors to just warnings, ex:
ERROR crunch::crunch] Subxt error: Rpc error: RPC error: ErrorObject { code: ServerError(1010), message: "Invalid Transaction", data: Some(RawValue("Transaction is outdated")) }

Ideally the transaction should be retried at least one more time, if it fails again with the same error just discard it and continue

Hey @paulormart, this looks like it could be a good first issue for one of the Polkadot Blockchain Academy to work on during the Kudos Carnival. What do you think?


Add Indonesia GCP course

What does this PR do?
Add resource(s)
For resources
Description
Why is this valuable (or not)?
Add new google cloud course in indonesian language
How do we know it's really free?
its from udemy
For book lists, is it a book? For course lists, is it a course? etc.
course list
Checklist:

[ ] Read our contributing guidelines.
[ ] Search for duplicates.
[ ] Include author(s) and platform where appropriate.
[ ] Put lists in alphabetical order, correct spacing.
[ ] Add needed indications (PDF, access notes, under construction).
[ ] Used an informative name for this pull request.

Follow-up

Check the status of GitHub Actions and resolve any reported warnings!

not free


Add HW version mask/pos

Description
Please provide a detailed description of what was done in this PR.
(And mention any links to an issue docs)
Changes include

[ ] Bugfix (non-breaking change that solves an issue)
[ ] New feature (non-breaking change that adds functionality)
[ ] Breaking change (change that is not backwards-compatible and/or changes current functionality)
[ ] Tests
[ ] Documentation
[ ] Other (for changes that might not fit in any category)

Breaking changes
Please complete this section if any breaking changes have been made, otherwise delete it.
Additional comments
Please post additional comments in this section if you have them, otherwise delete it.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Please upload report for BASE (API_LEVEL_5@a175f73). Learn more about missing BASE report.

Additional details and impacted files
@@              Coverage Diff               @@
##             API_LEVEL_5     #717   +/-   ##
==============================================
  Coverage               ?   85.96%
==============================================
  Files                  ?        8
  Lines                  ?      399
  Branches               ?        0
==============================================
  Hits                   ?      343
  Misses                 ?       56
  Partials               ?        0

Flag
Coverage Δ

unittests
85.96% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


precompiled: remove it all

This avoids having to hack some check/fix format stuff in our private
build.

@htuch


Add new function to isolate desired ROI

This branch adds a new function isolateROI() which lets you isolate a sub-ROI of an smld structure.

I've added an example to README.md and updated documentation.  I don't know how to make a meaningful generic method for the SMLD type so for now I've only changed the documentation for the SMLD2D method.


Remove numpy dependency

The numpy dependency is very large and unnecessary for this project. It is only used in one short block of code in j1939_22.py.
The dependency to numpy has been removed in favor of built-in Python list operations.

PR #46 might be useful to look at?

PR #46 might be useful to look at?

Wow, I can't believe I didn't see this beforehand! Thanks for pointing it out. Their PR should do the trick.


Bluetooth: Mesh: Add read callback for ccc in provisioning service

This patch adds read permission for client characteristic configuration
descriptor. This is required by MESH/NODE/MPS/BV-06-C,
MESH/NODE/MPS/BV-07-C PTS tests.
Change-Id: I56b77e5ad6a92ec225e28c59c58b59f3ffed3679

Wouldn't this also be needed for proxy_ccc? In fact I think it'd be good to name the variables prov_ccc_val and proxy_ccc_val to avoid confusion.


Calling rs.status immediately after rs.initiate throws auth error

I have put all the steps to initiate a replica-set in a shell script. Basically, I am trying to convert a standalone instance A  to replica-set that includes one arbiter B and one secondary node C. These are the following steps in the script:

Start the standalone instance without auth parameter.
Create an admin user for mongodb admin database with the following role:
Successfully added user: {
"user" : "mongodb_admin",
"roles" : [
{
"role" : "root",
"db" : "admin"
}
]
}
Stop the standalone instance A.
Start all the three nodes(A, B,C) with replica-set option and key-file access control.
Initiate replicaset on instance A by calling rs.initiate.
Call rs.status to check if the health of all members is 1 (currently there is only A in replicaset)
Add the arbiter node B and secondary node C to replicaset.
Wait until all the nodes are healthy.

Evrything is fine upto step 5. I am able to successfully initiate a replicaset. But step 6 throws the following error:
2018-08-27T11:43:47.080+0100 E QUERY    [thread1] Error: auth failed : DB.prototype._authOrThrow@src/mongo/shell/db.js:1608:20 @(auth):6:1 @(auth):1:2
When I put a sleep of 15 seconds in between steps 5 and 6, it works perfectly fine.
But I still want to make sure that I am doing things correctly. If yes then for how much longer should I sleep between steps 5 and 6 to ensure that the above mentioned error doesnt occur.
Also I am unable to understand why its giving an auth error even though I used the same user and password for initiating the replica-set?

Apologies. Just realised this replository is for mongodb  using kubernetes. Closing it.


what is timeline to officially support package signing

Hi team,
What is anticipated release timeline when nugget package signature feature will be supported? Is that work still in progress?

The work is still in progress, we expect it to become public on nuget.org after Visual Studio 2015 RTM ships. There is no committed timeline yet, we will reassess priorities after RTM.

This issue is being closed because it has not been updated in 3 months.
We apologize if this causes any inconvenience. We ask that if you are still encountering this issue, please log a new issue with updated information and we will investigate.


Localization Support?

Hi there, thank you for making this. I've noticed it doesn't appear to work with Localization. The strings aren't read in the Localizable.strings files since they're not Text() but rather Markdown ()
Any ways around this?

Sorry for the late response. MarkdownUI does not support LocalizedStringKey at the moment, but I might consider adding support in the future. As a workaround, you can use the good old NSLocalizedString:
Markdown(Document(NSLocalizedString("yourLocalizedStringKey", comment: "")))

I also need this function.
At present, I read the String file according to the language, and then directly pass the parameters to the Markdown method. If there is a better idea, it should be realized as soon as possible. thank you
PS：
I like this open source project very much and hope to stick to it. Thank you for your contribution to open source. Thank you very much

Unfortunately, I can't add support for LocalizedStringKey because its properties are not accessible. Maybe Apple opens this API in a future SwiftUI release. For the moment, we have to stick to the workaround that I mentioned previously.


simulated_stack.py

Dear Scott,
I want to simulate the deformation time series using simulated_stack.py. Unfortunately, the expected deformations (used in https://github.com/alpha-tango/insar-deformation-extraction/blob/main/model-replication.ipynb) were not produced. I want to know how to use simulated_stack.py to generate the synthetic deformation. I look forward to your reply (my email address: chenglong-li@ies.ac.cn).  If you have any difficulties, please let us know.  Anyway, we will thank you.

Dear Scott, I want to simulate the deformation time series using simulated_stack.py. Unfortunately, the expected deformations (used in https://github.com/alpha-tango/insar-deformation-extraction/blob/main/model-replication.ipynb) were not produced. I want to know how to use simulated_stack.py to generate the synthetic deformation. I look forward to your reply (my email address: chenglong-li@ies.ac.cn). If you have any difficulties, please let us know. Anyway, we will thank you.

hello, I have the same problem with you, have you fixed it? My email<EMAIL_ADDRESS>


havok 2018 (Elden Ring)

Do you plan to support havok 2018 (Elden Ring)?

I don't have any plans to expand the scope of this project at the moment.


Initial pass at Raspberry Pi support.

All the tags/roles work now
Started a small 'rpi' role
resolves pigmonkey/spark#51

Thanks for the review and sorry for the delay in responding.  I will make some updates and reapply all roles and get back to you soon!

Stale. Closing.


quint test run produces itf traces without the last state

quint test run produces itf traces without the last state
module myTest {
  var x: int
  action init = x' = 0
  run myTest = init.then(x' = 1).then(x' = 2).then(x' = 3)
}

and then
$ quint test --output {}.itf.json myTest.qnt
$ jq -c .states < myTest.itf.json
[{"#meta":{"index":0},"x":{"#bigint":"0"}},{"#meta":{"index":1},"x":{"#bigint":"1"}},{"#meta":{"index":2},"x":{"#bigint":"2"}}]

Reported by @rnbguy
It is reproducible on HEAD.
@konnov noted this is probably a regression from #1133

is this resolved? I see that my example still produces wrong output on latest main.

Whoops! I closed the wrong issue. Thanks for checking :)


Adding description to some of the v1beta3 objects.

Partially fixes #3628
cc @bgrant0607

I just commented on Pod and PodList, but please use the same pattern for all objects.

@bgrant0607: Addressed comments. PTAL.

LGTM

@vishh Please fix the ListMeta SelfLink description in your next PR. Also, the documentation of which fields cannot be updated has been merged, so please update the descriptions with that info.


Empty record data on chrome

I following this simple step to recording my video
JS
var player = videojs('video', {
    controls: true,
    loop: false,
    width: 334,
    height: 251,
    plugins: {
        record: {
            audio: true,
            video: true,
            maxLength: 60,
            debug: false
        }
    }
});

player.on('deviceError', function()
{
    console.log('device error:', player.deviceErrorCode);
});

player.on('startRecord', function()
{
    console.log('started recording!');
});

player.on('finishRecord', function()
{
    var data = new FormData();

    data.append('video-filename', 'video-filename.webm');
    data.append('video-blob', player.recordedData);

    console.log(data);
    console.log(player.recordedData);

    $.ajax({
        type: 'POST',
        url: '/api/upload/video',
        data: data,
        dataType: 'json',
        processData: false,
        contentType: false
    }).done(function(data) {
        if (data.code == 200) {
            return window.location.href = '/share'
        };

        console.log(data);
    });
});

Work fine in firefox, but has empty data in chrome. .

So, what did i miss ?
Chrome 50.0.2661.102
Firefox 46.0.1
Latest version of RecordRTC.js & videojs.record.js

Test using blob
var blob = new Blob([player.recordedData]);

console.log(blob.size); = 15
And sure, working fine in firefox

@adrianug in older versions of the Chrome browser you would need to use recordedData.audio and recordedData.video and stitch these 2 blobs together on the server-side using something like FFmpeg. With recent Chrome browsers >= 49 (inc. the one you're using) a single blob is created and stored into recordedData. What is the Video field is required?

Also, before uploading etc, can you reproduce this issue with the simplest example, eg. https://collab-project.github.io/videojs-record/examples/audio-video.html?

Video field is required is response from ajax post because when post from chrome, $_FILES is empty
Both firefox and chrome has correct data in blob:https%3A//site.com/4ec97744-994a-4f5e-8c1b-a9fd3ecfa8d8
But, when i try using new FormData(); to post data with ajax, player.recordedData is missing in chrome
var data = new FormData();
data.append('video-filename', 'video-filename.webm');
data.append('video-blob', player.recordedData);

And wrong size when using new Blob();
var blob = new Blob([player.recordedData]);
console.log(blob.size);

Sadly, everything works fine in firefox

Solved with new Blob([player.recordedData.video], { type: 'video/webm' }); in chrome and new Blob([player.recordedData], { type: 'video/webm' }); in firefox

Solved with new Blob([player.recordedData.video], { type: 'video/webm' }); in chrome and new Blob([player.recordedData], { type: 'video/webm' }); in firefox

Still, i dont understand why different, player.recordedData.video in chrome and just player.recordedData in firefox

It's something (left over) from RecordRTC.js, take a look at that source code if you really want to know ;)


dropdown navbar should render as active if one of its items are also active

Have you read the Contributing Guidelines on issues?

[X] I have read the Contributing Guidelines on issues.

Prerequisites

[X] I'm using the latest version of Docusaurus.
[ ] I have tried the npm run clear or yarn clear command.
[ ] I have tried rm -rf node_modules yarn.lock package-lock.json and re-installing packages.
[ ] I have tried creating a repro with https://new.docusaurus.io.
[ ] I have read the console error message carefully (if applicable).

Description
The title is self-explanatory.
Reproducible demo
No response
Steps to reproduce
module.exports = {
  themeConfig: {
    navbar: {
      items: [
        {
          type: 'dropdown',
          label: 'Community',
          position: 'left',
          items: [
            {
              type: 'doc',
              label: 'Docs',
            },
            {
              type: 'doc',
              label: 'Social',
              docId: 'social',
            },
          ],
        },
      ],
    },
  },
};

Then open either Docs or Social.
Expected behavior
The navbar dropdown should also be rendered as active to indicate that the user is browsing a page from within that dropdown.
Actual behavior
The navbar parent dropdown never renders as active.
Your environment
No response
Self-service

[ ] I'd be willing to fix this bug myself.

I would be willing to fix this bug by myself, but I'm really bad at CSS. But if you agree with the bug, and want me to fix it, I would appreciate some brief high level instructions.

Hey
Your steps to reproduce are invalid since a doc type requires a docId attribute. Creating a good working repro always saves time for everybody.
I understand your point, but I'm not sure this is a desired default behavior, considering a dropdown might contain arbitrary items, and that we'll eventually enable you to provide your own custom items later. The concept of a subitem being "active" may not be known by Docusaurus itself in the future, and depend on your own definition of what "active" means.
Also, it's not clear how to represent the dropdown as active, in both mobile and desktop modes. You will notice that pre-made dropdowns (versioning, i18n) do not have that concept currently.

See for example on the React-Native website, the dropdown does not become "active" once you press any subitem, and I'm not sure it should.
https://reactnative.dev/docs/next/getting-started

Note that very soon, once Firefox enables it, you'll be able to achieve this in userland quite easily with plain CSS. You can already use this pseudocode inspiration as a progressive enhancement:
.dropdown:has(.dropdown__link--active) {
  border-bottom: solid blue;
}

Hi @slorber,

Your steps to reproduce are invalid since a doc type requires a docId attribute. Creating a good working repro always saves time for everybody.

I am sorry about that. Will make sure not to make the same mistake in the future.

I understand your point, but I'm not sure this is a desired default behavior, considering a dropdown might contain arbitrary items, and that we'll eventually enable you to provide your own custom items later.

Having it render as active when one of its items are also active sounds like a better default to me. In either case, the user should be able to enable or disable this behavior, so we don't need to argue about what should be made the default, I think.

The concept of a subitem being "active" may not be known by Docusaurus itself in the future

That's a technical detail I can't comment, I think. If it's impossible to do, then there is nothing to be done indeed.

and depend on your own definition of what "active" means.

Well, my definition is pretty generic: if there is at least one child item active. Whatever renders these items to active is of user's choice, like activeBaseRegex option for type: default.

Also, it's not clear how to represent the dropdown as active, in both mobile and desktop modes.

I would expect the same styling of the doc navbar item, when it's active. For example, in Docusaurus Website, it's green when it's active, and white otherwise (in dark mode).

You will notice that pre-made dropdowns (versioning, i18n) do not have that concept currently.

Right, but do you think there is any reason why they shouldn't?

Note that very soon, once Firefox enables it, you'll be able to achieve this in userland quite easily with plain CSS. You can already use this pseudocode inspiration as a progressive enhancement: [...]

That's very interesting, thanks for letting me know.

All that said, another approach would be to extend the activeBaseRegex option from type: default to type: dropdown, so that I could write my regex of paths matching the paths of the dropdown items by myself and it would render the same result (just with a little more configuration repetition).

Actually, just checked at the code and we have a concept of containsActiveItems, and it's used only to decide if the mobile collasible menu is expanded by default:

Note: the dropdown already supports activeBaseRegex  to decide if the above menu is expanded by default, but it not used in any way to highlight the parent item/link.

That's a technical detail I can't comment, I think. If it's impossible to do, then there is nothing to be done indeed.

Nevermind I think I have an idea of how it could work in the future.

I would expect the same styling of the doc navbar item, when it's active. For example, in Docusaurus Website, it's green when it's active, and white otherwise (in dark mode).

Indeed that probably makes sense to reuse that logic and just apply a color to the dropdown. Considering on mobile the menu subitem would be visible by default, I'm not sure it's worth it to highlight the menu parent collapsible item? (Menu and Dropdown are 2 different UIs)

On my subjective human opinion I believe it should be highlighted on mobile as well, which would only appear once the user has clicked Back to main menu. The same way it is highlighted today for the doc navbar/menu:
https://github.com/facebook/docusaurus/assets/29582865/a4fbb593-ded0-4b6b-a1a8-c12ec7647dff

@felipecrs your demo does not show a dropdown so it's not super relevant to me.
I hacked the version dropdown with my devtools locally to emulate it.
What I understand is that you want this:

What we currently have is this:

Is it really useful to highlight the dropdown "parent" menu, when it's always expanded by default, and the child is already highlighted?
I mean, by seeing the subitem being highlighted, it's clear that it's inside the parent item. Highlighting both would maybe be more confusing because you don't know anymore which item to focus your eyes on?

Ok, you are right. I agree with you. I did not realize that the "drop-down" is always expanded in mobile view.
I would go even further then: when the dropdown is expanded on desktop view (and thus the subitem is already highlighted), maybe the highlight of the parent should disappear?

Something generic that applies to both mobile and desktop:

dropdown expanded: no highlight in parent
dropdown collapsed: highlight the parent when one child is active

In mobile it would always fall into the first condition.

Something generic that applies to both mobile and desktop:

dropdown expanded: no highlight in parent
dropdown collapsed: highlight the parent when one child is active

In mobile it would always fall into the first condition.

Except when the user manually collapses the menu item in mobile, in that case, I would say it should be highlighted if a child item is active.

Those mobile/dsktop components are different, and I'm not sure using the same ux makes sense here.
On desktop the dropdown expands while you hover it. Wouldn't it be weird if the dropdown "parent" becomes unhighlighed once you hover it? 🤔

Wouldn't it be weird if the dropdown "parent" becomes unhighlighed once you hover it? 🤔

It seems so.
Alright, I trust your judgment :)

Is this issue open ??

I would say so.

@thats-sarthak Again, please stop asking—all activities are apparent from this thread. Just send a PR if you know how to fix it.

This is working for me in "@docusaurus/core": "2.0.0-beta.17", Here is example.
themeConfig:
/** @type {import('@docusaurus/preset-classic').ThemeConfig} */
({
autoCollapseSidebarCategories: true,
navbar: {
title: "CREATACHAIN",
logo: {
alt: "creata Logo",
src: "img/logo1.png",
},
items: [
{
// type: "doc",
// docId: "Introduction/high-level-overview",
// position: "left",
// label: "Learn",
        type: "dropdown",
        position: "left",
        activeBaseRegex: 'docs/zenith',
        label: "Learn",
        items: [
          {
            label: "Zenith Chain",
            to: `/docs/zenith/Introduction/high-level-overview`,
            activeBaseRegex: 'docs/zenith',
          },
          {
            label: "Catena (CIP-20) Chain",
            to: `/docs/catena/Introduction/high-level-overview`,
            activeBaseRegex: 'docs/catena',
          },
        ],
      },
      ],
         prism: {
    theme: require("prism-react-renderer/themes/github"),
    darkTheme: require("prism-react-renderer/themes/dracula"),
  },
}),

This is working for me in "@docusaurus/core": "2.0.0-beta.17", Here is example.
// In your docusaurus.config.js or siteConfig.js file

module.exports = {
  // ... other configurations ...

  themeConfig: {
    // ... other theme configurations ...

    // Auto-collapse sidebar categories for better user experience
    autoCollapseSidebarCategories: true,

    // Navbar configuration
    navbar: {
      title: "Title", // Change this to your desired title
      logo: {
        alt: "Logo",
        src: "img/logo1.png", // Replace with your logo path
      },
      items: [
        {
          // Dropdown menu in the navbar for "Learn" section
          type: "dropdown",
          position: "left",
          activeBaseRegex: 'docs/zenith', // Highlight if in the Zenith section
          label: "Learn",
          items: [
            {
              label: "Zenith Chain",
              to: `/docs/zenith/Introduction/high-level-overview`,
              activeBaseRegex: 'docs/zenith',
            },
            {
              label: "Catena (CIP-20) Chain",
              to: `/docs/catena/Introduction/high-level-overview`,
              activeBaseRegex: 'docs/catena',
            },
          ],
        },
        // ... other navbar items ...
      ],
    },

    // Prism code highlighting configuration
    prism: {
      theme: require("prism-react-renderer/themes/github"), // Use GitHub theme
      darkTheme: require("prism-react-renderer/themes/dracula"), // Use Dracula theme for dark mode
    },
  },
};

Hi, the above fix isn't working with Docusaurus version v3.6.3. Is there an alternative way to implement this?


change staging link to spee.ch instead

staging is a speech test server, and i think it will shut down in the future, because test server. and the best use is spee.ch ( live version) and never turned off in the future..

Thanks Chris!


Manual v1.28.0 version bump

Backstage release v1.28.0 has been published, this Pull Request contains the changes to upgrade to this new release
Please review the changelog before approving, there may be manual changes needed to enable new features:

Changelog: v1.28.0
Upgrade Helper: From 1.28.0-next.2 to 1.28.0

Note: This version bump was completed manually due to version bump errors.

Thanks @freben 🚀


Add Color to Force Directed Graph

Colorize the Force Directed nodes, according to a common grouping.

+1

Notice: this issue has been closed because it has been inactive for 447 days. Feel free to comment and request for this issue to be reopened.


Support multinode training on GPU

I don't have a node with 8 gpus. I have two nodes each with 4 gpus. So is it possible to train a model on multiple nodes?

This is actually something that does work right now but it's still experimental. There's also no real public-facing API for it yet; you have to type in some obscure and fairly magical things to set it all up correctly.
We should polish it off and document it!

Can you say a bit more about your model, though? Would gradient all-reductions across multiple nodes suffice?

@hawkinsp Technically, I'm training a reformer model using Trax library.

And I assume you're just looking for data parallelism, i.e., partitioning a minibatch across GPUs, not partitioning in any other way (e.g., model parallelism)?

@hawkinsp yeah my concern is data parallelism

@hawkinsp Can you please share your notes on this (don't need a stable api) ? We are trying some hybrid data/model/pipeline parallelism  so it is a little different from @py4 but would love to get started with data parallelism

Data parallelism would of value to other projects that use XLA as well (eg https://www.tensorflow.org/swift).  Exposing this functionality in a standardized way would help drive progress in the broader ecosystem!

I don't have a node with 8 gpus. I have two nodes each with 4 gpus. So is it possible to train a model on multiple nodes?

Hello py4, I am meeting the same problem, have you found some solutions?

This is actually something that does work right now but it's still experimental. There's also no real public-facing API for it yet; you have to type in some obscure and fairly magical things to set it all up correctly.
We should polish it off and document it!

Hello hawkinsp, Could you please provide more details about how to run data parallel with multi node GPUs?

@hawkinsp We are also interested in running JAX code on multiple nodes. Anything (hacky or not) that you can share would be appreciated. Thanks!

I really enjoyed Jax during my DM internship and wanted to use it on my university SLURM cluster, but the lack of a clear (official) data parallel solution is a huge blocker to increasing Jax adoption outside of Google where you cant just grab a TPU pod. A single 8 (GPU) replica setup can barely train a Resnet50 imagenet classifier. Training SimCLR or any other large SOTA model is currently impossible without multi-node data parallelism.

I would love this feature! I enjoy Jax, but I've been largely using DeepSpeed due to its ability to distribute across clusters.

Any progress on this issue ? Using JAX to train a model on multi-node, multi-GPU is becoming a very important features for us.

This is a significant bottleneck for scaling on multi-node GPU clusters. Is there any update on this issue?
Also, there was a recent pjit tutorial that explains multi-node TPU scaling but doesn't mention about GPUs. Is that planned to be updated in the future?

@sudhakarsingh27 I constantly monitoring the jax releases, and there is something WIP that you might be interested in https://github.com/google/jax/pull/8364

See also: #9582

Yes indeed. We haven't advertised it that much yet, but (a) you need to initialize the cluster using that API, and (b) you need to follow the same rules of multi-host programming that also apply on TPU, documented here: https://jax.readthedocs.io/en/latest/multi_process.html
I suspect we can consider this issue closed when we've documented (a) in the document (b).

@hawkinsp @zhangqiaorjc
Multinode (or multiprocess) doesn't seem to work with the following jax(lib) versions:
jax                           0.3.13
jaxlib                        0.3.10+cuda11.cudnn82

Ran the attached minimal code on single node with 8 V100 GPUs as follows (2 processes with 4 GPUs each):
CUDA_VISIBLE_DEVICES="0,1,2,3" python jax_multi_node_experiment.py 0 &
CUDA_VISIBLE_DEVICES="4,5,6,7" python jax_multi_node_experiment.py 1


tip: Indentation

You want to be sure to follow consistent indentation in your code, it'll make it a lot easier to read!
I would write this:
https://github.com/HannahAurand/hannahaurand.github.io/blob/588dcd3c9ab3d6fc2ac224b8646061d63ffe30ce/index.html#L29-L39
like this:

Who am I?

I am constantly learning and applying new knowledge to all
     areas of my life. Implementing innovative strategies for
     problems is my passion--even if it means I fail along the way.
     Failure is a path to success.

Got it, will do.


ValueError: The filepath provided must end in .keras (Keras model format)

Hi!
I have been trying to use AiZynthTrain to train AiZynthFinder with some personal reactions and reaction template.
I have mapped and cleaned the reaction and template files with my own protocols and my goal is to retrain AiZynthFinder without running any additional cleaning/preparation step.
I have used the expansion pipeline with the following config file:
expansion_model_pipeline:
  python_kernel: aizynthtrain
  file_prefix: test
  nbatches: 200
  training_fraction: 0.9
  random_seed: 1689
  selected_ids_path: "lookup_templates.json"

And I got the following errors during training:
2024-09-16 13:44:07.814 [1726483142464316/model_training/206 (pid 3123416)] Task is starting.
2024-09-16 13:44:08.591 [1726483142464316/model_training/206 (pid 3123416)] 2024-09-16 13:44:08.591729: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-09-16 13:44:08.604 [1726483142464316/model_training/206 (pid 3123416)] 2024-09-16 13:44:08.604123: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-09-16 13:44:08.607 [1726483142464316/model_training/206 (pid 3123416)] 2024-09-16 13:44:08.607848: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-09-16 13:44:13.767 [1726483142464316/model_training/206 (pid 3123416)] <flow ExpansionModelFlow step model_training> failed:
2024-09-16 13:44:13.873 [1726483142464316/model_training/206 (pid 3123416)] Internal error
2024-09-16 13:44:13.875 [1726483142464316/model_training/206 (pid 3123416)] Traceback (most recent call last):
2024-09-16 13:44:13.875 [1726483142464316/model_training/206 (pid 3123416)] File "/data/users/carespos/conda/envs/aizynthtrain/lib/python3.10/site-packages/metaflow/cli.py", line 1134, in main
2024-09-16 13:44:13.875 [1726483142464316/model_training/206 (pid 3123416)] start(auto_envvar_prefix="METAFLOW", obj=state)
2024-09-16 13:44:13.875 [1726483142464316/model_training/206 (pid 3123416)] File "/data/users/carespos/conda/envs/aizynthtrain/lib/python3.10/site-packages/metaflow/tracing/__init__.py", line 27, in wrapper_func
2024-09-16 13:44:13.875 [1726483142464316/model_training/206 (pid 3123416)] return func(args, kwargs)
2024-09-16 13:44:14.668 [1726483142464316/model_training/206 (pid 3123416)] File "/data/users/carespos/conda/envs/aizynthtrain/lib/python3.10/site-packages/metaflow/_vendor/click/core.py", line 829, in __call__
2024-09-16 13:44:14.669 [1726483142464316/model_training/206 (pid 3123416)] return self.main(args, kwargs)
2024-09-16 13:44:14.669 [1726483142464316/model_training/206 (pid 3123416)] File "/data/users/carespos/conda/envs/aizynthtrain/lib/python3.10/site-packages/metaflow/_vendor/click/core.py", line 782, in main
2024-09-16 13:44:14.669 [1726483142464316/model_training/206 (pid 3123416)] rv = self.invoke(ctx)
2024-09-16 13:44:14.669 [1726483142464316/model_training/206 (pid 3123416)] File "/data/users/carespos/conda/envs/aizynthtrain/lib/python3.10/site-packages/metaflow/_vendor/click/core.py", line 1259, in invoke

2024-09-16 13:44:14.669 [1726483142464316/model_training/206 (pid 3123416)] return _process_result(sub_ctx.command.invoke(sub_ctx))
2024-09-16 13:44:14.669 [1726483142464316/model_training/206 (pid 3123416)] File "/data/users/carespos/conda/envs/aizynthtrain/lib/python3.10/site-packages/metaflow/_vendor/click/core.py", line 1066, in invoke
2024-09-16 13:44:14.669 [1726483142464316/model_training/206 (pid 3123416)] return ctx.invoke(self.callback, ctx.params)
2024-09-16 13:44:14.669 [1726483142464316/model_training/206 (pid 3123416)] File "/data/users/carespos/conda/envs/aizynthtrain/lib/python3.10/site-packages/metaflow/_vendor/click/core.py", line 610, in invoke
2024-09-16 13:44:14.669 [1726483142464316/model_training/206 (pid 3123416)] return callback(args, kwargs)
2024-09-16 13:44:14.669 [1726483142464316/model_training/206 (pid 3123416)] File "/data/users/carespos/conda/envs/aizynthtrain/lib/python3.10/site-packages/metaflow/_vendor/click/decorators.py", line 21, in new_func
2024-09-16 13:44:14.669 [1726483142464316/model_training/206 (pid 3123416)] return f(get_current_context(), args, kwargs)
2024-09-16 13:44:14.669 [1726483142464316/model_training/206 (pid 3123416)] File "/data/users/carespos/conda/envs/aizynthtrain/lib/python3.10/site-packages/metaflow/cli.py", line 468, in step
2024-09-16 13:44:14.669 [1726483142464316/model_training/206 (pid 3123416)] task.run_step(
2024-09-16 13:44:14.669 [1726483142464316/model_training/206 (pid 3123416)] File "/data/users/carespos/conda/envs/aizynthtrain/lib/python3.10/site-packages/metaflow/task.py", line 650, in run_step
2024-09-16 13:44:14.669 [1726483142464316/model_training/206 (pid 3123416)] self._exec_step_function(step_func)
2024-09-16 13:44:14.669 [1726483142464316/model_training/206 (pid 3123416)] File "/data/users/carespos/conda/envs/aizynthtrain/lib/python3.10/site-packages/metaflow/task.py", line 62, in _exec_step_function
2024-09-16 13:44:14.669 [1726483142464316/model_training/206 (pid 3123416)] step_function()
2024-09-16 13:44:14.669 [1726483142464316/model_training/206 (pid 3123416)] File "/data/users/carespos/conda/envs/aizynthtrain/lib/python3.10/site-packages/aizynthtrain/pipelines/expansion_model_pipeline.py", line 83, in model_training
2024-09-16 13:44:14.670 [1726483142464316/model_training/206 (pid 3123416)] training_runner([self.config_path])
2024-09-16 13:44:14.670 [1726483142464316/model_training/206 (pid 3123416)] File "/data/users/carespos/conda/envs/aizynthtrain/lib/python3.10/site-packages/aizynthtrain/modelling/expansion_policy/training.py", line 83, in main
2024-09-16 13:44:14.670 [1726483142464316/model_training/206 (pid 3123416)] callbacks = setup_callbacks(
2024-09-16 13:44:14.670 [1726483142464316/model_training/206 (pid 3123416)] File "/data/users/carespos/conda/envs/aizynthtrain/lib/python3.10/site-packages/aizynthtrain/utils/keras_utils.py", line 76, in setup_callbacks
2024-09-16 13:44:14.670 [1726483142464316/model_training/206 (pid 3123416)] checkpoint = ModelCheckpoint(
2024-09-16 13:44:14.670 [1726483142464316/model_training/206 (pid 3123416)] File "/data/users/carespos/conda/envs/aizynthtrain/lib/python3.10/site-packages/keras/src/callbacks/model_checkpoint.py", line 191, in __init__
2024-09-16 13:44:14.670 [1726483142464316/model_training/206 (pid 3123416)] raise ValueError(
2024-09-16 13:44:14.670 [1726483142464316/model_training/206 (pid 3123416)] ValueError: The filepath provided must end in `.keras` (Keras model format). Received: filepath=test_keras_model.hdf5
2024-09-16 13:44:14.670 [1726483142464316/model_training/206 (pid 3123416)]
2024-09-16 13:44:14.674 [1726483142464316/model_training/206 (pid 3123416)] Task failed.
2024-09-16 13:44:14.679 Workflow failed.
2024-09-16 13:44:14.679 Terminating 0 active tasks...
2024-09-16 13:44:14.679 Flushing logs...
    Step failure:
    Step model_training (task-id 206) failed.

where the final error is:
2024-09-16 13:44:14.670 [1726483142464316/model_training/206 (pid 3123416)] ValueError: The filepath provided must end in `.keras` (Keras model format). Received: filepath=test_keras_model.hdf5

Could you please let me know how to fix/debug this?

And is this the right pipeline to use when wanting to train AiZynthFinder without running any preparation step?

Many thanks!
Carmen

I fixed it by installing specific keras and tensorflow versions:
pip install keras==2.8.0
pip install tensorflow==2.8.0
pip install tensorboard==2.8.0
pip install tensorflow-serving-api==2.8.0

To avoid this issue to occur in the future, the dependencies could be added to the pyproject.toml.
However, I got now another error during validation:
FileNotFoundError: [Errno 2] No such file or directory: 'testing_template_library.csv'
Even if I did not configure the validation pipeline, it seems it's still running it.
Best,
Carmen

Thanks for this. We are aware of these issues with some version of tensorflow and/or keras. We are working on a re-factored codebase that will address this.


onLeave rc-1

I'm calling a function on the client to render data before the next route. This worked in previous versions, however doesn't seem to using 'onLeave' in the latest rc-1.
Router:
export default (renderRoute) =>
	<Route component={App}>
		<Route path="/" component={Home} onLeave={renderRoute}/>
		<Route path="page1" component={Page1} onLeave={renderRoute}/>
		<Route path="admin" component={Admin}  onLeave={renderRoute}/>
	</Route>

Client:
const initState = immutifyState(window.__INITIAL_STATE__)
const history   = createHistory();
const logger    = createLogger({level: 'info', collapsed: true });
const store     = applyMiddleware(thunkMiddleware, logger )(createStore)(reducer, initState);

const route = renderRoute((nextState, transition, done) => {
 // nextState returns 'undefined' using 'onLeave'
  fetchComponentData(store.dispatch, nextState.routes.map(b => b.component), nextState.params)
  .then(() => done())
  .catch(err => console.log(err))
})

ReactDOM.render(
  <Provider store={store}>
    <Router children={route} history={history}/>
  </Provider>,
  document.getElementById('app')
)

It works correctly if I use 'onEnter', however 'onLeave' returns undefined for the renderRoute arguments. Has the onLeave prop changed?

why would you do this in onLeave instead of onEnter?

But yes, onLeave is no longer asynchronous since the only valid use-case is to prevent navigation, which is a synchronous operation (its identical to onBeforeUnload)
You should be able to just do an onEnter of your root route, seems like the right place anyway.

One reason to do this in an isomorphic app is to avoid making duplicate data fetching calls on the server.   I found this approach originally recommended in this tutorial and found it fairly reasonable.  I'm fairly sure that @boyswan's example was also inspired by the linked tutorial.
As far as I understand, if you make a request for data in an onEnter hook on the server (or the component's componentWillMount method) it won't work because the response gets sent back to the client before the asynchronous request can finish.
To solve this problem on the server we do as recommended in react-router's advanced server rendering docs and use the renderProps.components to find each component's data requirements, load the data, and only reply with the server rendered route once the data comes back. Here's a pseudo code example (see the comments):
import { renderToString } from 'react-dom/server'
import { match, RoutingContext } from 'react-router'
import routes from './routes'
import store from './store'
import fetchComponentData from './store/utils'

serve((req, res) => {

  match({ routes, location: req.url }, (error, redirectLocation, renderProps) => {
    if (error) {
      res.send(500, error.message)
    } else if (redirectLocation) {
      res.redirect(302, redirectLocation.pathname + redirectLocation.search)
    } else if (renderProps) {
      // fetchComponentData will loop over the components looking for static load methods
      fetchComponentData(store.dispatch, renderProps.components, renderProps.params)
        .then(() => {
        // Now our data is loaded into our store and our components can render properly, plus we can hydrate the state
         const initialComponent = renderToString(<RoutingContext {...renderProps} />)
         const initialState = store.getState()
         const HTML = `

${initialComponent}

         `
         res.send(200, HTML)
        })
        .catch(err => res.send(500, err))
    } else {
      res.send(404, 'Not found')
    }
  })
})

Now the problem is that we don't have anything in place to call our component's static load methods on the client.  Even if I isomorphically render route A with all the required data, when I navigate client side to route B it will be empty of its data.  It would be great to put our fetchComponentData method into our route's onEnter hooks to call our component's static load method, except that it would cause duplicate data fetching calls on the server, which doesn't seem clean.
We know for sure however that a route's onLeave hook will never get called on the server and only on the client, so it is actually a perfect place to fetch data for the next component client side as shown in @boyswan's example.
Another solution is to put data requests into the componentDidMount method which we also know will not get called on the server.  However, this means that I will have a static load method and a componentDidMount method with duplicate code, which doesn't seem ideal.
Relay will of course help solve the problem of isomorphic rendering and declarative data requirements on the component, but not everyone is ready to implement a GraphQL server.
I hope my explanation makes sense, I'm still struggling a bit figuring out how to write universal apps.  Do you have any recommendations?  It seems we need to fetch data on the server one way, and then implement data fetching on the client another way.  In both client and server cases, the router seems like a perfect place to do it all if the API will let us.

It really depends on e.g. which Flux framework you're using - they'll generally have different recommendations for how to structure data fetching. In general you have to do something like the example or use a render loop to make things work on the server, but making this work without repeating code on the client is really more a function of the patterns available to you for structuring your code to call the same logic from multiple places.

In my case I happen to be using Redux.  Regardless of which flux implementation you use, there is still the issue of duplicate data fetching requests.
When I server render a page the route's onEnter hook will get called on the server and then again on the client.  Both calls are unnecessary as I've already fetched the data server side as shown in my example.
The componentDidMount hook will also get called as soon as react code bootstraps itself on the client, which is also an unnecessary request.
But I still need some kind of lifecycle hook to load data while I am navigating to new routes on the client that won't be server rendered.
The only hook that won't result in duplicated data requests is the onLeave hook.  Nearly any route could be either server rendered (if it's the user's first request or they refresh the page), or client rendered if the user has already loaded my react app.  When the onLeave hook is called I know that the user is navigating to a new route on the client.  With onEnter and componentDidMount that isn't the case, and for a server rendered page data fetching will be duplicated.
I feel like this is a valid use case for asynchronous data fetching in the onLeave hook.
As I think about it though there are alternatives, like tracking some isLoaded state in my store and then checking in componentDidMount to make sure I'm not duplicating a fetch for data that's already loaded.

I used this as my onEnter function, which prevents it being called twice on page load:
let route = renderRoutes((nextState, transition, done) =>
  nextState.location.action !== 'POP' ?
    fetchComponentData(store.dispatch, nextState.routes.map(b => b.component), nextState.location.pathname)
    .then(()=>done())
    .catch(err => console.log(err))
  : done()
)

That's not quite right - we don't want to override onLeave at this point.
In general async props are a bit tricky - you can take a look at what's been done previously, but blocking the transition completely is not how we want to encourage people to do it, I don't think.

Yes I agree that we don't want to block the transition on the client because that would result in a non-responsive UI. My component is subscribed to the redux store so we can go ahead and transition before the data fetch finishes, probably show a loading indicator, and when the data gets to the store it will be pushed to the component to re-render.
A slightly adjusted version of @boyswan's function seems to do the trick for me, I hadn't thought to test for the action type 'POP'
function onEnter(nextState, transition, done) {
    if (nextState.location.action !== 'POP') {
      const params = nextState.params;
      const components = nextState.routes.map(route => route.component);
      fetchComponentData(store.dispatch, components, params);
    }
    done();
  }

But again, why not allow this to be done in an onLeave hook by giving it the nextState argument?  It doesn't even need to be asynchronous to do the trick, it just needs to know the nextState

Your data fetching in this case is logically tied to entering the new route, not to leaving the old one. You shouldn't model it with a hook that runs on leaving the old route.

That makes sense, checking for the pop action before loading data seems to be a good solution.  Thanks @taion and @boyswan for your help!

Yes, onBeforeUnload is synchronous, but why do we have to enforce parity with that API? In my opinion, using a native confirm dialog is the last resort option. In our case, it is desirable to use an entirely custom dialog.


PG and default (0)::numeric

(0)::numeric set '(0)' not 0.
"user_timezone" numeric(5,2) DEFAULT (0)::numeric NOT NULL load 'user_timezone' => '(0)'

Q
A

Yii version
2.0.9

regex working well, default not...
http://www.phpliveregex.com/p/h1h

Please, give us a little bit more information.
What are you trying to do? How can be reproduce the problem? What is the expected result?

do loadDefaultValues() after create, db field and result in first post

please show the code you use for creation of the column and also the code to read it.

unable to reproduce, created a unit test to verify that it works: 06b045c

class ActiveRecord extends \yii\db\ActiveRecord {

	public function init() {
		parent::init();
	    $this->loadDefaultValues();
	}
}

CREATE TABLE "public"."phpbb3_users" (
...
"user_timezone" numeric(5,2) DEFAULT (0)::numeric NOT NULL,
...
)

var_dump((new User)->attributes); die;

[
...
    'user_timezone' => '(0)'
...
]

looks like it has been fixed in 2.0.10-dev by ad796dbf225ac52b41ee6652190c1361445e3b0b

oh... i think it's 2.0.9 code :D ok.. wait


Accessing a resource scoped configuration without providing a resource is not expected

[2021-06-15 11:48:30.089] [exthost] [warning] [vscode.markdown-language-features] Accessing a resource scoped configuration without providing a resource is not expected. To get the effective value for 'markdown.preview.breaks', provide the URI of a resource or 'null' for any resource.
[2021-06-15 11:48:30.089] [exthost] [warning] [vscode.markdown-language-features] Accessing a resource scoped configuration without providing a resource is not expected. To get the effective value for 'markdown.preview.linkify', provide the URI of a resource or 'null' for any resource.
[2021-06-15 11:48:30.089] [exthost] [warning] [vscode.markdown-language-features] Accessing a resource scoped configuration without providing a resource is not expected. To get the effective value for 'markdown.preview.typographer', provide the URI of a resource or 'null' for any resource.
[2021-06-15 11:48:30.096] [exthost] [warning] [vscode.markdown-language-features] Accessing a resource scoped configuration without providing a resource is not expected. To get the effective value for 'markdown.preview.breaks', provide the URI of a resource or 'null' for any resource.
[2021-06-15 11:48:30.096] [exthost] [warning] [vscode.markdown-language-features] Accessing a resource scoped configuration without providing a resource is not expected. To get the effective value for 'markdown.preview.linkify', provide the URI of a resource or 'null' for any resource.
[2021-06-15 11:48:30.096] [exthost] [warning] [vscode.markdown-language-features] Accessing a resource scoped configuration without providing a resource is not expected. To get the effective value for 'markdown.preview.typographer', provide the URI of a resource or 'null' for any resource.

What triggers this for you? I don't see it when opening a normal preview

Integration tests.

Should be fixed by a71ebc98ff3ce75cdde0d3222653bd9daa42f6f8

@mjbvz can you add verification steps? Sounds like checking the integration tests is enough?

Marking as verified based on test run logs


Use device descriptors to suggest auto-routes for devices in the path tree

Done in branch "next".


Emit declaration files for composite projects

Issue
Seems to me that the easiest thing one can do is run one tool at a time. So ideally I'd like to run only jest and have my monorepo just reference itself (its modules) when needed. I fail though when it comes to type checking (done within ts-jest), because composite packages needs packages on which they depend built to disk - otherwise they can't load them.
IMHO it would be nice to support this as it could work kinda out of the box then for the user. Having to run both tsc and jest at the same time feels kinda counter-productive.

Hi, would you please explain clearly what is the issue/error you encountered and a repo to reproduce your issue ?

This is really more a feature request than a bug report.
Problem is that it's often useful for another tooling (like an editor) to have types written to disk (especially when working in a monorepo), so it would be good (in my opinion) to have declaration files being generated while type-checking with ts-jest to avoid separate processes just for this.

does emit declaration file depend on tsconfig.json for composite project ?

Hm, not completely sure. Type-checking relies on it for sure, declaration files alone are not options-specific anyhow (I think?) so it actually could not rely on it. If my reasoning is correct then only options related to disk location are relevant - so to answer the question: yes, a little bit.

how do you want ts-jest to do with this ? declaration files are only known to typescript. ts-jest cannot itself emit declaration files. Do you have any suggestions ?
From typescript doc, when using composite option, declaration must be turned on. It means typescript will always emit declaration. Since ts-jest just calls typescript compiler and use your tsconfig so emit declaration should work.

They are not, as you are using JS API for TypeScript this kind of thing would have to be implemented in ts-jest (using TypeScript APIs, not from scratch), it doesn't behave as if tsc would get called.

Can you point out in ts-jest compiler which part is wrong ? ts-jest compiler is using typescript api so I don’t see why this doesn’t work.

I think your issue might be related to #1336 as now ts-jest doesn’t support well for monorepo as well as composite project.

Look like your request was solved in #747 and related to #745 . Please check them.

I will close this in favor of #1336

hi @Andarist, I was looking into this and #1336 seems to be related. I don't know how to make ts compiler API to do that. Do you have any hints ?

Issue
Seems to me that the easiest thing one can do is run one tool at a time. So ideally I'd like to run only jest and have my monorepo just reference itself (its modules) when needed. I fail though when it comes to type checking (done within ts-jest), because composite packages needs packages on which they depend built to disk - otherwise they can't load them.
IMHO it would be nice to support this as it could work kinda out of the box then for the user. Having to run both tsc and jest at the same time feels kinda counter-productive.

@Andarist I think I might have an idea how to tackle this issue. Just to understand you correctly: your request is while running tests with ts-jest, you want to have at the same time type declaration files are written on disk, similar to running tsc before running tests

@ahnpnl yes, exactly that. This would allow for not having to generate type declarations in a separate watch process and they are useful for other tools (such as VScode when dealing with project references)

thanks for confirmation 👍 , I would say this feature request should be applicable for both monorepo and non-monorepo.


Change CLI processing so that --flag value is equivalent to --flag=value

This has been tripping up people (and ocassionally me) for a while. Ideally our flag processing would work identically to how git handles it.
https://github.com/ogier/pflag
https://github.com/github/hub/issues/1819#issuecomment-444577641

@da-edra Please do! 🙏


Particle documentation is missing from the Vapor website

Particle documentation is missing from the Vapor website

FYI - It's incoming, with PR #2849.


Document Platform SSL

Complete: https://github.com/deis/workflow/blob/master/docs/src/managing-deis/platform-ssl.md

Closing in favour of deis/docs-v2#25