Create long form podcasts by building the transcript iteratively

Asking a LLM to generate long outputs is unreliable and can give to bad results anyway.
So what about enabling something like that (in pseudocode).
previous_transcript = ''"
content = [("c1", "c2"), ("c3", "c4")]
for content_chunk in content:
     transcript = build_transcript(previous_transcript, content_chunk)
     previous_transcript = transcript

I think this is a good idea, many use cases beg long-form podcasting with greater detail of the content. Right now both podcastfy and NotebookLM generate quite short conversations.
We would need to update the prompt too, so it has a header (with intros etc), and a footer (with bye bye's). And then your pseudocode in-between.

Yes, the problem I found with notebook-lm is the podcasts are only 10-15 minutes long. I'm used to 1-2 hour long podcasts. if I can create those I can create my own custom show on topics i like.

Any update?

Hi thanks for the follow up. I was out this week. Once I'm back I will
review and prioritize the backlog.
On Thu, Oct 24, 2024, 7:38 AM ralyodio @.***> wrote:

Any update?
—
Reply to this email directly, view it on GitHub
https://github.com/souzatharsis/podcastfy/issues/48#issuecomment-2434916113,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADTMY3K332YVWA3KXMYJD7LZ5DE2TAVCNFSM6AAAAABP3FJ6JWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMZUHEYTMMJRGM
.
You are receiving this because you commented.Message ID:
@.***>

Any update?

I'm curious about your use case. Why longer podcasts would help solve your
problem?
On Mon, Oct 28, 2024, 8:02 AM ralyodio @.***> wrote:

Any update?
—
Reply to this email directly, view it on GitHub
https://github.com/souzatharsis/podcastfy/issues/48#issuecomment-2441266583,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADTMY3OGLTEHDO5AHIIZ6YDZ5YKTPAVCNFSM6AAAAABP3FJ6JWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINBRGI3DMNJYGM
.
You are receiving this because you commented.Message ID:
@.***>

I am using this to generate a podcast but when I did it live it was like 1-2 hours long, I'm wanting to use AI to do the podcast now since I don't have the time to keep it going. The issue is these are only like 4-5 minutes long. I want to create a 1-2 hour long podcast.

This is interesting.
How do you structure your podcast for it to last so long while remaining
interesting?
Your experience would be helpful to better design a solution for it.
Thanks for sharing.
On Mon, Oct 28, 2024, 8:58 AM ralyodio @.***> wrote:

I am using this to generate a podcast but when I did it live it was like
1-2 hours long, I'm wanting to use AI to do the podcast now since I don't
have the time to keep it going. The issue is these are only like 4-5
minutes long. I want to create a 1-2 hour long podcast.
—
Reply to this email directly, view it on GitHub
https://github.com/souzatharsis/podcastfy/issues/48#issuecomment-2441382816,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADTMY3LH474XGBN2OYKRTFDZ5YRHDAVCNFSM6AAAAABP3FJ6JWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINBRGM4DEOBRGY
.
You are receiving this because you commented.Message ID:
@.***>

I can't think of a podcast that is 5-7 minutes long, so the use case is pretty clear I would think.  If for example I want to learn about trains using the wiki article. I would want the following segments:
Introduction
The Origins and Early History of Trains
The Technological Evolution of Trains
The Cultural and Economic Impact of Trains
Specialized and Unique Train Systems
Train Safety, Operations, and Components
The Future of Train Transportation
Outro
Using this as "dialogue_structure" in the config and asking for 9000 words for about an hour or so, you still just get 5-7 minutes. It's so surface level you don't really learn anything.
Issue #126 mentioned Storm which seems similar to what is needed. I haven't tried it yet though.

It's Gemini.
Using other base LLMs can give much more detailed and longer outcomes.
You can try it with the local llm option and I'll add openai too.
My understanding is that Storm will help with podcast generation from a
given input topic by doing we search plus nicely structuring processing and
developing the information .
Thanks for your thoughts.
On Wed, Oct 30, 2024, 11:28 AM jonjonsson @.***> wrote:

I can't think of a podcast that is 5-7 minutes long, so the use case is
pretty clear I would think. If for example I want to learn about trains
using the wiki article. I would want the following segments:
Introduction
The Origins and Early History of Trains
The Technological Evolution of Trains
The Cultural and Economic Impact of Trains
Specialized and Unique Train Systems
Train Safety, Operations, and Components
The Future of Train Transportation
Outro
Using this as "dialogue_structure" in the config and asking for 9000 words
for about an hour or so, you still just get 5-7 minutes. It's so surface
level you don't really learn anything.
Issue #126 https://github.com/souzatharsis/podcastfy/issues/126
mentioned Storm https://github.com/stanford-oval/storm which seems
similar to what is needed. I haven't tried it yet though.
—
Reply to this email directly, view it on GitHub
https://github.com/souzatharsis/podcastfy/issues/48#issuecomment-2447342970,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADTMY3KTLAF7THM2G3R5F7TZ6DUIFAVCNFSM6AAAAABP3FJ6JWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINBXGM2DEOJXGA
.
You are receiving this because you commented.Message ID:
@.***>

Thanks @souzatharsis, I'm looking forward to OpenAI integration. Really appreciate this project!
Any LLM will struggle with generating longform content though. Even GPT-4o-mini that can theoretically can output over 16k tokens doesn't really ever go over 1500 tokens no matter what you ask it.
Storm might be overkill. Generating an output for each dialogue_structure topic individually would go a long way, including anything generated before in the input for context. Eventually would need to tackle too many input tokens problem. Comes with a some challenges for sure...
Anyway just my 2 cents!

How would you go about "Generating an output for each dialogue_structure
topic individually"
That's what I was going to use storm for since it has built-in web search +
text cleaning for proper LLM downstream processing.
Thanks for sharing your thoughts.
http://linkedin.com/in/tharsissouza
On Wed, Oct 30, 2024 at 1:52 PM jonjonsson @.***> wrote:

Thanks @souzatharsis https://github.com/souzatharsis, I'm looking
forward to OpenAI integration. Really appreciate this project!
Any LLM will struggle with generating longform content though. Even
GPT-4o-mini that can theoretically can output over 16k tokens doesn't
really ever go over 1500 tokens no matter what you ask it.
Storm might be overkill. Generating an output for each dialogue_structure
topic individually would go a long way, including anything generated before
in the input for context. Eventually would need to tackle too many input
tokens problem. Comes with a some challenges for sure...
Anyway just my 2 cents!
—
Reply to this email directly, view it on GitHub
https://github.com/souzatharsis/podcastfy/issues/48#issuecomment-2447794111,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADTMY3KVV52YVLFTZSETUQTZ6EFEHAVCNFSM6AAAAABP3FJ6JWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINBXG44TIMJRGE
.
You are receiving this because you were mentioned.Message ID:
@.***>

The First user prompt would outline the dialogue_structure, and then ask to only generate the first item in the dialogue_structure.
Assistant outputs generation for first item only.
In the same conversation, second user prompt is to generate the second item in the dialogue_structure
Assistant outputs generation for second item only.
etc...
Something like that will make a deeper conversation. However if you are looking into Storm, it will handle all that anyway.

Here's a tutorial from Google for creating long-form audio
https://cloud.google.com/text-to-speech/docs/create-audio-text-long-audio-synthesis

Here's a solution to overcome the maximum output tokens limit:
https://github.com/mo-haggag/ROG
Rolling Output Generation (ROG). This technique allows for fully coherent, context-aware output generation up to the size of the context window, in a single initial call. Applied to OpenAI models, this enables generating 30X the normal output. Generating entire books or extensive codebases is suddenly within reach.

@jonjonsson @brumar @ralyodio
I've managed to implement long-form podcast generation.
Please see sample audio in this discussion - Would love your feedback before I prepare a PR:
https://github.com/souzatharsis/podcastfy/discussions/168

This has been implemented - I'd love feedback!
See https://github.com/souzatharsis/podcastfy/blob/main/usage/how-to.md#how-to-generate-longform-podcasts
[0.3.6] - 2024-11-13
Added

Add longform podcast generation support

Users can now generate longer podcasts (20-30+ minutes) using the --longform flag in CLI or longform=True in Python API
Implements "Content Chunking with Contextual Linking" technique for coherent long-form content
Configurable via max_num_chunks and min_chunk_size parameters in conversation config
word_count parameter removed from conversation config as it's no longer used

The trouble I am having with longer form content is that the TTS APIs have token limits. The Gemini TTS max is 5000 tokens and OpenAI 4096. With Elevenlabs and Edge you can push up to 10K but the quality of speech is significantly less natural. Elevenlabs is not bad but still nothing close to Gemini in my opinion. Does anybody have a fix or suggestion to circumvent this so that I can still use the gemini TTS API?
"Failed to generate audio: 400 Either input.text or input.ssml is longer than the limit of 5000 bytes. This limit is different from quotas. To fix, reduce the byte length of the characters in this request, or consider using the Long Audio API: https://cloud.google.com/text-to-speech/docs/create-audio-text-long-audio-synthesis."
I have not tried the long audio synthesis API but my intuition is that it wouldn't work out of the box and may deliver a lower quality of audio too. I may test and report back, but in the meantime any suggestions would be greatly appreciated. I am looking at how to adjust the code so that multiple API calls are made to gemini with the different parts that are generated with longform=True, and then stitching them together.
Thank you for this package, it's brilliant!


No UI pointer to show initial sync is in progress on the map view

Describe the feature request.
When conducting a sync on EUSM app, after selecting locations, on the map view the sync bar shows for a few seconds but then disappears. The usr is thus not able to know whether the sync is still in progress. One can simply think that the app is frozen and not doing anything.
The ask: Have a UI that let's the user know that the initial sync still progressing and they should give it some time.
https://github.com/user-attachments/assets/a9f9e881-d391-4812-8560-066535861ad2
Additional context

The map view is the default view on the EUSM app.
On the register view, there is a progress bar when the sync is in progress.

Acceptance criteria

[ ] There is a UI on the map view that lets the user know that sync is in progress. This should be intuitive.
[ ] User on the map view is notified when all the service points have been downloaded. i.e. sync is completed.

Area path
A list of ordered steps in the app on usage of the feature to support anyone testing it e.g. Code reviewer, QA e.g.

Login to the app with a username and password.
Select the locations as directed by the app.
Tap on Sync at the bottom of the selection panel.
Observe.

Implementation plan (For Engineers)
The plan for implementing the solution e.g. via a description or a check list for the various ordered tasks that will need to be completed.
i.e. Describe how you intend to solve the problem

@AnnieMungai  we had a chat with part of the Product team today. We have a go-ahead to add the spinner.  @HenryRae  @ellykits  should we use the same spinner we have on the registers?  i.e. the white background with a spinner in the centre

This happens because the custom resource (LocationHierarchy) sync does not use the SDK's Sync worker that transmits SyncJobStatuses. See the discussion I started in the SDK repo @dubdabasoduba https://github.com/google/android-fhir/discussions/2625.


Adding a job

We had some trouble with sync-ing this manual approach with our sheet, so we are going to ask you to please use the form instead. https://hpc.social/jobs/about/ Thanks!

@vsoch Sure, I've done that. I submitted this job via the form last Thursday, but nothing's appeared, so I'd figured the form wasn't working properly...

I've just done it again (again). All three times I've had the success message:
Form Submitted. Thanks.```

hmm I don't see it - let me test it out.

I have a feeling you are blocking something in your browser (or some other issue) because it worked for me:

Are you absolutely sure there isn't an error message in your browser console? Right click -> Inspect -> Console? In the future, perhaps try the form directly:
https://forms.gle/JQVsr3sVzsWhmtvz5
Sorry for the trouble! It should show up for the next round now that I added it.

Ah the link there gave me a clue ... our firewall swallows Google drive, Box, Dropbox and everything else, so it's likely that. Thanks for doing it for me - sorry about that!


hello this is not a bug but how do i change back to human?

yea i just dont know how to change from mob to human

You just scroll to the top with arrow keys and press enter on the X.


[Bug]: serviceAccount option was dropped from PipelineOptions DisplayData starting Beam 2.32.0

What happened?
Seems like https://github.com/apache/beam/pull/14970 resulted in 'serviceAccount' option being dropped from PipelineOptions DisplayData due to that option having a 'Hidden' attribute.
This results in the corresponding option not being visible for users of some runners (for example, Dataflow Cloud Console UI).
Issue Priority
Priority: 1 (data loss / total loss of function)
Issue Components

[ ] Component: Python SDK
[X] Component: Java SDK
[ ] Component: Go SDK
[ ] Component: Typescript SDK
[ ] Component: IO connector
[ ] Component: Beam examples
[ ] Component: Beam playground
[ ] Component: Beam katas
[ ] Component: Website
[ ] Component: Spark Runner
[ ] Component: Flink Runner
[ ] Component: Samza Runner
[ ] Component: Twister2 Runner
[ ] Component: Hazelcast Jet Runner
[ ] Component: Google Cloud Dataflow Runner

Fixed by https://github.com/apache/beam/pull/24642


Figure out what to do with more than one database

The /graphql URL will always query the first connected database.
If you have more than one, you can use /graphql/name_of_database to specifically query any named database.


Application error opening TWXP

Fixed an issue that was causing create database to create an empty database
twxp will no longer crash tring o load an empty database
twxp will no loner crash if there are no databases when opening


Add a button to the standard view

How can I add a button to the standard view of the model?
Use rails_admin 2.1.1/ruby 2.6.6

This depends on what you'd like the button to do (a custom action, simple link, something else). Some of this is covered in the Wiki, and there's the mailing list for support requests if you're unable to find anything in the Wiki for what you're looking to do.


[Issue]: Unless HIP_VISIBLE_DEVICES specified, initializing HIP runtime takes 10 seconds (strace attached).

Problem Description
This simple test program doing just a 1-byte hipMalloc takes 10 seconds to run:
#include <hip/hip_runtime.h>

int main() {
  char* p;
  (void)hipMalloc(&p, 1);
}

But if I specify HIP_VISIBLE_DEVICES=63 to pin to one specific GPU, then it runs in under 1 second.
I strace'd it and a difference that stood out was the AMDKFD_IOC_MAP_MEMORY_TO_GPU ioctl's. These often have latency over 60ms.
By default (when the program takes 10 seconds), there are 260 such ioctl's.
With HIP_VISIBLE_DEVICES=63, there are 135 such ioctl's.
Why is the number of AMDKFD_IOC_MAP_MEMORY_TO_GPU ioctl's 2x higher when not pinned to a GPU? I thought that once a device has been picked, HIP_VISIBLE_DEVICES shouldn't make a difference anymore.
Operating System
Ubuntu 22.04
CPU
2x AMD EPYC 9454 48-Core Processor
GPU
8x  MI300X in CPX mode (total 64 logical devices).
ROCm Version
ROCm 6.2.0
ROCm Component
HIP
Steps to Reproduce
No response
(Optional for Linux users) Output of /opt/rocm/bin/rocminfo --support
No response
Additional Information
No response

strace.zip

Another thing that stands out is that when not defining HIP_VISIBLE_DEVICES, strace attached above 👆 does 130 times this:
14:09:38.165070 openat(AT_FDCWD, "/dev/random", O_WRONLY) = 5
14:09:38.165108 write(5, "\230", 1)     = 1
14:09:38.165137 close(5)                = 0

(With the same value \230 each time).
When HIP_VISIBLE_DEVICES is defined, it does that only 4 times instead of 130.

Hi @bjacob. Internal ticket has been created to investigate your issue. Thanks!

Closing this issue, as it only reproduced on one machine, and doesn't reproduce anymore today, so that was just a machine in a bad state.


[BUG] Connect4

Describe the bug
When the first player clicks any button, the buttons turn into invite buttons.
To Reproduce
Follow the steps in the example, but select true in the slash command and change the message to interaction.
Expected behavior
The buttons should remain the same, not turning into invitation buttons.
Screenshots

Please complete the following information:

Node Version: v16.15.0
Gamecord Version: 4.0.0
Discord.js Version: 14.7.1

Additional context
X

Fixed in version 4.1.0 and 3.1.0  :D


backbox support

add support for backbox in the install script.

I like this idea.

thanks when do you think you can add it in?

I can probably get support for the latest (BB 4.2) added within the next week.

Just added this in commit fb025ee. It's currently in the dev branch but will be merged over into the master branch later this week.
Until then, if you'd like you can use it by cloning the project, then following the Beta Testing steps before running sudo tools/install.sh. The repo will have to be cloned locally and switched to the beta testing branch before running the script until dev is merged into master.

k thanks


Move around par files automatically; exclude date in write_par

This changes the logic in the write_par() function so that it doesn't include the date in the par file name by default (to avoid cluttering the "archive" directory; we can always look at older versions in the git history).
Additionally, it adds a function pp.checkin.shuffle_pars() to the notebook which automatically copies the new par file to the par directory and archives the old one, then stages the changes in Git. This should make things easier on timers who want to check in par files, compared to the NANOGrav 15-year dataset workflow.

CI is currently failing because staging the changes relies on the gitpython package, which is not in the environment yet.

OK, I've taken out the bit that warns you if there are uncommitted changes to the old timing_model. It was a bit misguided as a mechanism for protecting against losing work anyway. The new behavior (copy if the file isn't tracked, and move if it is) is easier to understand, and the only way it can make you can lose work is if you have uncommitted changes to the file in the archive directory that gets overwritten, which seems pretty unlikely.


[Issue#3976] [component/client] Adding config for ReaderImpl

Fixes #3976
Motivation
We wish to add a method which  would allow ReaderBuilderImpl to create a reader which starts reading from a specified timestamp by the user.
Modifications
Added a configuration and method to allow for a Reader created by ReaderBuilderImpl to start processing from a selected timestamp.

@sijie  I noticed that usually the user would either use startMessageId(long timestamp) or startMessageId(MessageId id) but not both. So should we make it so that we need to call only one of the methods to set the message id at which the user would need to start? Currently, it is required that MessageId id be a non-null parameter, so if the user wants to use startMessageId(long timestamp), they would also have to call startMessageId(MessageId id) with some dummy variable (id) as well.

The test added is broken, although I couldn't pinpoint the exact cause. The location where the consumer seeks to the timestamp the user selects might be incorrect, or is ineffective, seeing that when the reader polls messages, we still get my-message-0 as the first message instead of my-message-3. I will look into it.

@sijie I want to confirm that seek() should be used for this configuration right?

@ConcurrencyPractitioner yes.

So while investigating, a couple of things was revealed. There is actually two approaches to getting to the right initial startPublishTimestamp:

Add a new parameter or field  so the initial subscription command (i.e. Commands.newSubscribe) would entail broker to seek to startPublishTimestamp directly.
We call seek() from the client side instead to get to startPublishTimestamp.

@sijie  which one do you think is preferable?

Hi! In my humble opinion I think I would vote for option number 2 with proper documentation to avoid misuse from the user. Mainly for simplicity reasons.
Also, I'm testing 2nd option and facing a non deterministic issue about unlucky timing being disconnected and not connected after seek() operation.
I'm not sure if this is from the fact of having a non-durable consumer or what, I will keep looking around this issue.

I would also prefer option 2 as well for simplicity reasons.

Alright, so I ran the test on local and here was a segment of the log for the test I added:
Seeking to time:<PHONE_NUMBER>386
19:21:13.441 [TestNG-method=testReaderAtSelectedTimestamp-1:org.apache.pulsar.client.impl.ReaderBuilderImpl@94] INFO  org.apache.pulsar.client.impl.ReaderBuilderImpl - Setting messageId value since timestamp value is<PHONE_NUMBER>386
19:21:13.450 [pulsar-web-149-6:org.eclipse.jetty.server.Slf4jRequestLog@60] INFO  org.eclipse.jetty.server.RequestLog - <IP_ADDRESS> - - [09/Apr/2019:19:21:13 -0700] "GET /admin/v2/persistent/my-property/my-ns/testReaderAtSelectedTimestamp/partitions HTTP/1.1" 200 16 "-" "Pulsar-Java-v2.4.0-SNAPSHOT" 6
19:21:13.458 [pulsar-web-149-7:org.eclipse.jetty.server.Slf4jRequestLog@60] INFO  org.eclipse.jetty.server.RequestLog - <IP_ADDRESS> - - [09/Apr/2019:19:21:13 -0700] "GET /lookup/v2/topic/persistent/my-property/my-ns/testReaderAtSelectedTimestamp HTTP/1.1" 200 132 "-" "Pulsar-Java-v2.4.0-SNAPSHOT" 6
19:21:13.459 [pulsar-client-io-155-1:org.apache.pulsar.client.impl.ConsumerImpl@493] INFO  org.apache.pulsar.client.impl.ConsumerImpl - [persistent://my-property/my-ns/testReaderAtSelectedTimestamp][reader-e722c2aff4] Subscribing to topic on cnx [id: 0xf5d3a09a, L:/<IP_ADDRESS>:59183 - R:localhost/<IP_ADDRESS>:15075]
19:21:13.461 [pulsar-io-142-1:org.apache.pulsar.broker.service.ServerCnx@641] INFO  org.apache.pulsar.broker.service.ServerCnx - [/<IP_ADDRESS>:59183] Subscribing on topic persistent://my-property/my-ns/testReaderAtSelectedTimestamp / reader-e722c2aff4
19:21:13.461 [pulsar-io-142-1:org.apache.pulsar.broker.service.persistent.PersistentTopic@624] INFO  org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://my-property/my-ns/testReaderAtSelectedTimestamp][reader-e722c2aff4] Creating non-durable subscription at msg id -1:-1:-1:-1
19:21:13.462 [pulsar-io-142-1:org.apache.bookkeeper.mledger.impl.NonDurableCursorImpl@54] INFO  org.apache.bookkeeper.mledger.impl.NonDurableCursorImpl - [my-property/my-ns/persistent/testReaderAtSelectedTimestamp] Created non-durable cursor read-position=3:0 mark-delete-position=3:-1
19:21:13.462 [pulsar-io-142-1:org.apache.pulsar.broker.service.persistent.PersistentTopic@648] INFO  org.apache.pulsar.broker.service.persistent.PersistentTopic - Cursor's name is: null
19:21:13.463 [pulsar-io-142-1:org.apache.bookkeeper.mledger.impl.ManagedCursorImpl@1789] INFO  org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [my-property/my-ns/persistent/testReaderAtSelectedTimestamp-null] Rewind from 3:0 to 3:0
19:21:13.463 [pulsar-io-142-1:org.apache.pulsar.broker.service.persistent.PersistentTopic@571] INFO  org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://my-property/my-ns/testReaderAtSelectedTimestamp][reader-e722c2aff4] Created new subscription for 0
19:21:13.463 [pulsar-io-142-1:org.apache.pulsar.broker.service.ServerCnx@701] INFO  org.apache.pulsar.broker.service.ServerCnx - [/<IP_ADDRESS>:59183] Created subscription on topic persistent://my-property/my-ns/testReaderAtSelectedTimestamp / reader-e722c2aff4
19:21:13.464 [pulsar-client-io-155-1:org.apache.pulsar.client.impl.ConsumerImpl@588] INFO  org.apache.pulsar.client.impl.ConsumerImpl - [persistent://my-property/my-ns/testReaderAtSelectedTimestamp][reader-e722c2aff4] Subscribed to topic on localhost/<IP_ADDRESS>:15075 -- consumer: 0
19:21:13.468 [TestNG-method=testReaderAtSelectedTimestamp-1:org.apache.pulsar.client.impl.ConsumerImpl@1376] INFO  org.apache.pulsar.client.impl.ConsumerImpl - [persistent://my-property/my-ns/testReaderAtSelectedTimestamp][reader-e722c2aff4] Seek subscription to publish time<PHONE_NUMBER>386
19:21:13.473 [pulsar-io-142-1:org.apache.pulsar.broker.service.ServerCnx@210] WARN  org.apache.pulsar.broker.service.ServerCnx - [/<IP_ADDRESS>:59183] Got exception NullPointerException : null
java.lang.NullPointerException: null
        at java.net.URLDecoder.decode(URLDecoder.java:136) ~[?:1.8.0_191]
        at org.apache.pulsar.common.util.Codec.decode(Codec.java:45) ~[pulsar-common-2.4.0-SNAPSHOT.jar:2.4.0-SNAPSHOT]
        at org.apache.pulsar.broker.service.persistent.PersistentMessageFinder.<init>(PersistentMessageFinder.java:55) ~[classes/:?]
        at org.apache.pulsar.broker.service.persistent.PersistentSubscription.resetCursor(PersistentSubscription.java:335) ~[classes/:?]
        at org.apache.pulsar.broker.service.ServerCnx.handleSeek(ServerCnx.java:1153) ~[classes/:?]
        at org.apache.pulsar.common.api.PulsarDecoder.channelRead(PulsarDecoder.java:228) ~[pulsar-common-2.4.0-SNAPSHOT.jar:2.4.0-SNAPSHOT]

What I found curious in the log output was that a NonDurableCursor was created, but its name was null. But later, a decoder used by PersistentMessageFinder attempts to decode the name fails because the name has a null vale So its been bugging me since this shouldn't happen. @sijie Do you have any pointers on why this might be the case?

One more thing. I have tried to turn on DEBUG mode for log4j when running the tests. What I did was I copied the log4j.properties of pulsar-proxy to pulsar-broker/src/main/resources/log4j.properties and changed the first line of the file to DEBUG, CONSOLE. Unfortunately, that did not work. Is there a way where we could turn on DEBUG mode while running TopicReaderTest?

Alright, I'm not sure if option 2 still works. From log after test rerun:
21:36:55.342 [bookkeeper-ml-workers-OrderedExecutor-1-0:org.apache.pulsar.broker.service.persistent.PersistentMessageFinder@99] INFO  org.apache.pulsar.broker.service.persistent.PersistentMessageFinder - [persistent://my-property/my-ns/testReaderAtSelectedTimestamp][reader-3f11d6ab7e] Found position 3:2 closest to provided timestamp<PHONE_NUMBER>242
21:36:55.342 [bookkeeper-ml-workers-OrderedExecutor-1-0:org.apache.pulsar.broker.service.Consumer@369] INFO  org.apache.pulsar.broker.service.Consumer - Disconnecting consumer: Consumer{subscription=PersistentSubscription{topic=persistent://my-property/my-ns/testReaderAtSelectedTimestamp, name=reader-3f11d6ab7e}, consumerId=0, consumerName=2a5c6, address=/<IP_ADDRESS>:61416}
21:36:55.344 [pulsar-client-io-187-1:org.apache.pulsar.client.impl.ClientCnx@600] INFO  org.apache.pulsar.client.impl.ClientCnx - [localhost/<IP_ADDRESS>:15087] Broker notification of Closed consumer: 0
21:36:55.344 [pulsar-client-io-187-1:org.apache.pulsar.client.impl.ConnectionHandler@109] INFO  org.apache.pulsar.client.impl.ConnectionHandler - [persistent://my-property/my-ns/testReaderAtSelectedTimestamp] [reader-3f11d6ab7e] Closed connection [id: 0x2ac3250a, L:/<IP_ADDRESS>:61416 - R:localhost/<IP_ADDRESS>:15087] -- Will try again in 0.1 s
21:36:55.345 [bookkeeper-ml-workers-OrderedExecutor-1-0:org.apache.pulsar.broker.service.AbstractDispatcherSingleActiveConsumer@159] INFO  org.apache.pulsar.broker.service.AbstractDispatcherSingleActiveConsumer - Removing consumer Consumer{subscription=PersistentSubscription{topic=persistent://my-property/my-ns/testReaderAtSelectedTimestamp, name=reader-3f11d6ab7e}, consumerId=0, consumerName=2a5c6, address=/<IP_ADDRESS>:61416}
21:36:55.345 [bookkeeper-ml-workers-OrderedExecutor-1-0:org.apache.pulsar.broker.service.persistent.PersistentSubscription@499] INFO  org.apache.pulsar.broker.service.persistent.PersistentSubscription - [persistent://my-property/my-ns/testReaderAtSelectedTimestamp][reader-3f11d6ab7e] Successfully closed subscription [NonDurableCursorImpl{ledger=my-property/my-ns/persistent/testReaderAtSelectedTimestamp, ackPos=3:-1, readPos=3:10}]
21:36:55.346 [bookkeeper-ml-workers-OrderedExecutor-1-0:org.apache.pulsar.broker.service.persistent.PersistentSubscription@406] INFO  org.apache.pulsar.broker.service.persistent.PersistentSubscription - [persistent://my-property/my-ns/testReaderAtSelectedTimestamp][reader-3f11d6ab7e] Successfully disconnected consumers from subscription, proceeding with cursor reset
21:36:55.346 [bookkeeper-ml-workers-OrderedExecutor-1-0:org.apache.bookkeeper.mledger.impl.ManagedCursorImpl@822] INFO  org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [my-property/my-ns/persistent/testReaderAtSelectedTimestamp] Initiate reset position to 3:2 on cursor reader-3f11d6ab7e
21:36:55.346 [bookkeeper-ml-workers-OrderedExecutor-1-0:org.apache.bookkeeper.mledger.impl.ManagedCursorImpl$7@861] INFO  org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [my-property/my-ns/persistent/testReaderAtSelectedTimestamp] reset position to 3:2 before current read position 3:10 on cursor reader-3f11d6ab7e
21:36:55.346 [bookkeeper-ml-workers-OrderedExecutor-1-0:org.apache.pulsar.broker.service.ServerCnx@1154] INFO  org.apache.pulsar.broker.service.ServerCnx - [/<IP_ADDRESS>:61416] [persistent://my-property/my-ns/testReaderAtSelectedTimestamp][reader-3f11d6ab7e] Reset subscription to publish time<PHONE_NUMBER>242
21:36:55.347 [pulsar-client-io-187-1:org.apache.pulsar.client.impl.ConsumerImpl@1379] INFO  org.apache.pulsar.client.impl.ConsumerImpl - [persistent://my-property/my-ns/testReaderAtSelectedTimestamp][reader-3f11d6ab7e] Successfully reset subscription to publish time<PHONE_NUMBER>242
!!!!!!!!! FAILURE-- [TestClass name=class org.apache.pulsar.client.api.TopicReaderTest].null([])-------

So the seek was supposed to have worked judging from the log ouput. And the cursor was reset from 3:0 to 3:2. Although obviously, the test still failed afterwards.

@sijie @ConcurrencyPractitioner please take a look at pull #4031 , thanks!

Alright, so I've been working on approach 1 since approach 2 didn't seem to be working. So I have added a new field to PulsarApi.proto and I wanted to generate PulsarApi.java accordingly. However, that was where I ran into some problems since I was unable to perform the second step. I have added a protobuf dependency and protobuf:compile plugin goal to pulsar-common/pom.xml but no changes were made to PulsarApi.java. So right now, I probably need some advice on how to generate the PulsarApi.java file.

In order to generate PulsarApi.java try with generate_protobuf_docker.sh script, it always worked well for me.
I'm still think that path 1 is not quite a good idea.
I wouldn't say that pull #4031 isn't working, the only case we didn't cover so far is a reset() by time in the middle of a queue, in fact, the reset() is happening but not at the expected position, is happening on a previous position, which would result in a duplicates messages which I don't think is a critical situation, and also I think that normal Consumers are facing this issue too.
@sijie Any thought on this?

Oh, you might've a point. (BTW, just wanted to see if the generate_protobuf_docker.sh works and it did. So thanks for the help on that. :))

Alright, so here is my current approach on the issue.
resetCursor is known to cause the consumer to disconnect. That might've been the cause of why we are starting to process from my-message-0 despite the seek call. So what I have done is that during the subscribe command, I match the start publish time with the message id startId that is the closest to it, and then I will call getNonDurableSubscription(MessageId startMessageId) to create the subscription where startMessageId equals startId.
That is the basic approach that I'm currently doing.

@sijie Mind taking a look?

@sijie Mind taking a look?

Actually, @sijie Do you mind if you take a look at lovelle's PR first? Once his is in, I will build on top of his approach and finish up this feature. (My current approach is flawed, so there's no need to look at my PR at the moment).

@ConcurrencyPractitioner sure. doing it.

Retest this please.


MGDOBR-967: Add action definition test

https://issues.redhat.com/browse/MGDOBR-967
Create new test suite for the configuration of Processor's actions.

@jomarko I have checked that the component type is using by developers too, so I have decited to follow the current format.


Default value for function parameters

Hi.
I've got a minimal class which looks as follows:
class DB {
	constructor () {

	}

	save(key, val) {
		localStorage.setItem(key, JSON.stringify(val));
		window[key] = val;
	}

	load(key, def='{}') {
		var x = localStorage.getItem(key) || def;
		window[key] = JSON.parse(x);
		return window[key];
	}
}

Apparently, Edge is complaining about the use of def='{}' in the parameter list. The error is Expected ')' on Line 11 which tells me that Edge doesn't like the default parameter
The code works in almost every other browser, which got me thinking. Maybe this isn't actual native JavaScript syntax? Or maybe Edge is just being Edge. But as far as I can tell, it is standard syntax.
Edge is however having a ? in the Browser Compatibility list. Either way, I would strongly recommend that this be implemented in Chakra - seeing as it won't break anything, just add functionality.
Sources:

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions/Default_parameters

Just like back in the days where you had to develop specifically for IE, the workaround here would be:
load(key, def) {
    if(def === undefined)
        def = '{}';
    ...
}

Again, would be neat if a modern browser was following modern syntax.

Hi, @torxed. Can you tell me which version of Edge you're using? Newer releases, beginning with the 1607 release of Windows 10, do support this syntax.

Confirming: ChakraCore at least as far back as 1.2.* (which was approximately 1607 or Anniversary Update) supports this syntax.
## Source
function foo(val, a='{}'){print(val, a)}; foo(42);

┌──────────────────┬─────────┐
│ ch-1.2.3         │ 42 {}   │
│ ch-1.3.2         │         │
│ ch-1.4.5         │         │
│ ch-1.5.3         │         │
│ ch-1.6.0         │         │
│ ch-1.6-latest    │         │
│ ch-1.7.1         │         │
│ ch-1.7-latest    │         │
│ ch-master-latest │         │
└──────────────────┴─────────┘

Now running Microsoft Edge 25.10586.672.0 / EdgeHTML 13.10586, not sure if it auto-update during the testing but it's working now.
Weird to me that the later version wasn't on the ISO.


transactions with regard to connectors

In the connector_api/connection.h it has start_transaction, commit_transaction, and rollback_transaction.
In ODBC you start a transaction by turning off autocommit mode. After calling commit or rollback the autocommit mode doesn't change. I could turn on autocommit mode after running commit or rollback. This isn't optimal if the user wants to keep doing new transactions, though I'd imagine most people would only commit or rollback once.
sqlpp11-connector-odbc is currently doing it wrong by running SQL statements instead of using the ODBC functions.

I think your approach (turning on autocommit after commit/rollback) is quite reasonable.
But then again, I might be utterly wrong, since I have no ODBC experience at all.

Sounds good. I know I personally have never needed to start a transaction right after finishing one. I imagine that changing autocommit mode doesn't take a long time if you do want to keep it in manual commit, but it shouldn't be much worse than someone using a different connector.


Allow to use plan from outside Opear file

https://mvnrepository.com/artifact/com.adobe.acs/acs-aem-commons-oakpal-checks/4.3.4
For example, this artifact has no plans defined.
I'd like to use checklist for AEM  6.5 defined in this Opear file and set skipping Vault install hooks checking.
As of right now, when using -f parameter in CLI there is no way to provide plan from outside Opear file.
How such situation should be addressed? Is rebuilding Opear file really needed? I am afraid requiring setting up project for rebuilding this opear file could be too heavy and sometimes could discourage from using OakPAL.

Thanks for the request, @pun-ky . I have actually been waiting to define a feature for generic plan overrides in the CLI until I collected enough use cases to flesh out the requirements for it. I'll add this one to the list.
In the meantime, I think I can safely add a --no-hooks switch to the CLI to override the plan's installHook policies with SKIP.

sound great, cool, thanks ;)


error: ‘shared_ptr’ in namespace ‘pcl’ does not name a template type

When trying to build, the following error is occurring:
error: ‘shared_ptr’ in namespace ‘pcl’ does not name a template type
typedef pcl::shared_ptr< pcl::VoxelGrid > Ptr;
error: ‘shared_ptr’ in namespace ‘pcl’ does not name a template type
typedef pcl::shared_ptr< const pcl::VoxelGrid > ConstPtr;
My pcl version is the 1.8.1. I temporarily fixed it by replacing pcl::shared_ptr to boost::shared_ptr. I saw there was a recent pull request on this regard.

When trying to build, the following error is occurring:
error: ‘shared_ptr’ in namespace ‘pcl’ does not name a template type
typedef pcl::shared_ptr< pcl::VoxelGrid > Ptr;
error: ‘shared_ptr’ in namespace ‘pcl’ does not name a template type
typedef pcl::shared_ptr< const pcl::VoxelGrid > ConstPtr;
My pcl version is the 1.8.1. I temporarily fixed it by replacing pcl::shared_ptr to boost::shared_ptr. I saw there was a recent pull request on this regard.

I also met the same question and solved it by checkout to old version.

just reverse the changes to all .h files.
From pcl::shared_ptr to boost::shared_ptr

could you please tell me, which old version you are talking about?

When trying to build, the following error is occurring:
error: ‘shared_ptr’ in namespace ‘pcl’ does not name a template type
typedef pcl::shared_ptr< pcl::VoxelGrid > Ptr;
error: ‘shared_ptr’ in namespace ‘pcl’ does not name a template type
typedef pcl::shared_ptr< const pcl::VoxelGrid > ConstPtr;
My pcl version is the 1.8.1. I temporarily fixed it by replacing pcl::shared_ptr to boost::shared_ptr. I saw there was a recent pull request on this regard.

I also met the same question and solved it by checkout to old version.

I created melodic branch for backwards compatibility. Try to checkout it.

I created melodic branch for backwards compatibility. Try to checkout it.

Thank you, it was helpful :-)

I updated the code, and now master branch can be built on older environments as well


DNS_PROBE_FINISHED_NXDOMAIN error

Description
I've started getting random sites not responding today. I initially had an issue with a site that wouldn't update from it's repo, so I trashed the local repo and pulled again. After this I wasn't able to connect to the site, so trashed the Nitro host and re-created. No joy. Went to look at another site and that had stopped working too.
Currently, 6 out of the 10 Nitro hosts I have are not working. They all produce the same error when trying to browse. I've tried other browsers and they have the same message. Also tried deleting some of those sites and re-creating, and some I've created with different host names.
As far as I know, all of these sites have worked in the last week. The laptop hasn't installed any OS updates that I'm aware of.
Steps to reproduce

Visit an existing site OR create a new one

Additional info

Nitro version: 2.0.8
Docker version: 20.10.7 using WSL

Just removed all but the MySQL and the main host I'm trying to work on from the hosts file and the site loaded.
It's looking like I can only have 8 URLs after <IP_ADDRESS> and the MySQL host. Any further host URLs after the 8th give the DNS_PROBE error in Chrome.

@jamiematrix that is rather strange... Can you try another browser with more than 8 hosts?

I've tried Edge and Firefox, both do the same.

I've just split out the alias' onto 2 lines, both starting with <IP_ADDRESS> and I now have 13 hosts working.

I'm wondering if this is a linux WSL issue... I'm on a macOS system and can't replicate with more than 8 hosts... all of the domains resolve.
Can you try curl -v hostname.nitro and provide the output?

The superuser.com post suggests its a Windows issue regardless of WSL. I've had a quick look around and can't see anything to suggest it is just Windows, but there doesn't seem to be anything about an alias limit anywhere (so far).
Running that command I get the source (HTML) of the index page that would load. Does that sound right? There's a lot in there hehe.

Yup, not a WSL thing, but a Windows one. Just verified that I can only add 9 aliases to a single IP address on my Windows 11 box. We should probably start breaking these up one per domain.

Would be nice if it were documented slightly better than stumbling upon it, thanks Microsoft. Glad there is a workaround, and sounds like should be a relative easy fix to output the hosts text?


add link for crowdsourced list

Description of PR
Type of PR

[ ] Bugfix
[x] New feature

Relevant Issues
Fixes #...
Checklist

[x] Compiles and passes lint tests
[x] Properly formatted
[x] Tested on desktop
[x] Tested on phone

Screenshots

Closes #595


lo_reader->load( i_excel2007 =  lv_xstring )  from smw0    dump

  SELECT SINGLE a~relid,a~objid,b~value
    FROM wwwdata AS a
    LEFT JOIN wwwparams AS b ON a~relid = b~relid
                             AND a~objid = b~objid
                             AND b~name EQ 'filesize'
    WHERE a~srtf2 = 0
    AND   a~relid = 'MI'
    AND   a~objid = 'ZSDR003'
    INTO @DATA(ls_tempt).
  IF sy-subrc <> 0 .
    MESSAGE '模板不存在，请使用SMW0上传'(001) TYPE 'E'.
    EXIT.
  ENDIF.
  ls_key = VALUE #( relid = ls_tempt-relid objid = ls_tempt-objid ).
  CALL FUNCTION 'WWWDATA_IMPORT'
    EXPORTING
      key               = ls_key
    TABLES
      mime              = lt_mime
    EXCEPTIONS
      wrong_object_type = 1
      import_error      = 2
      OTHERS            = 3.
  IF sy-subrc <> 0.
* Implement suitable error handling here
  ENDIF.

  CALL FUNCTION 'SCMS_BINARY_TO_XSTRING'
    EXPORTING
      input_length = CONV i( ls_tempt-value )
    IMPORTING
      buffer       = lv_xstring
    TABLES
      binary_tab   = lt_mime
    EXCEPTIONS
      failed       = 1
      OTHERS       = 2.
  IF sy-subrc <> 0.
* Implement suitable error handling here
  ENDIF.

  CHECK lv_xstring IS NOT INITIAL.
  CREATE OBJECT lo_reader TYPE zcl_excel_reader_2007.
  DATA(lo_excel) = lo_reader->load( i_excel2007 =  lv_xstring ).
  lo_worksheet = lo_excel->get_active_worksheet( ).

Please format the issue better and provide a sample report that allows to replicate the issue.

I have formatted the code for @LHSAVOKI. Unfortunately, the screenshot is missing (dump header?), the issue cannot be reproduced because the MIME repository file is required too. Finally, I guess that the dump happens because the exceptions from the class ZCX_EXCEL are not caught. If there is no clarification soon, I propose to close this issue.

SMW0 upload XLXM files will not DUMP, but XLSX files will DUMP

SMW0 upload xlsm files will not DUMP, but XLSX files will DUMP

Still a lot of information is missing, see my message above. The word "DUMP" is not sufficient to investigate.

The XLSX file uploaded by SMW0 can be reproduced using the example program ZDEMO-EXCEL29 P_OBJID

Thanks, I'll check in 10 hours approximately.

I have used the program below (with the file ZDEMO_EXCEL29_INPUT from SMW0) and couldn't reproduce the dump. It means that the problem is not related to SMW0, but it's related to the Excel file you are opening.
Could you attach it please? (you may cleanup your Excel file, but make sure the error still occurs).
Thank you.
Also make sure that you are using the latest abap2xlsx version.
Minimal reproducible program:
REPORT z_issue_1256.
DATA ls_key TYPE wwwdatatab.
DATA lt_mime TYPE TABLE OF w3mime.
DATA lv_xstring TYPE xstring.

SELECT SINGLE a~relid,
              a~objid,
              b~value
  FROM wwwdata AS a
         LEFT JOIN
           wwwparams AS b ON  a~relid = b~relid
                          AND a~objid = b~objid
                          AND b~name  = 'filesize'
  WHERE a~srtf2 = 0
    AND a~relid = 'MI'
    AND a~objid = 'ZDEMO_EXCEL29_INPUT'
  INTO @DATA(ls_tempt).
IF sy-subrc <> 0.
  MESSAGE '模板不存在，请使用SMW0上传'(001) TYPE 'E'.
  EXIT.
ENDIF.
ls_key = VALUE #( relid = ls_tempt-relid
                  objid = ls_tempt-objid ).
CALL FUNCTION 'WWWDATA_IMPORT'
  EXPORTING  key               = ls_key
  TABLES     mime              = lt_mime
  EXCEPTIONS wrong_object_type = 1
             import_error      = 2
             OTHERS            = 3.
IF sy-subrc <> 0.
  " Implement suitable error handling here
ENDIF.

CALL FUNCTION 'SCMS_BINARY_TO_XSTRING'
  EXPORTING  input_length = CONV i( ls_tempt-value )
  IMPORTING  buffer       = lv_xstring
  TABLES     binary_tab   = lt_mime
  EXCEPTIONS failed       = 1
             OTHERS       = 2.
IF sy-subrc <> 0.
  " Implement suitable error handling here
ENDIF.

CHECK lv_xstring IS NOT INITIAL.
DATA lo_reader TYPE REF TO zif_excel_reader.
lo_reader = NEW zcl_excel_reader_2007( ).
TRY.
    DATA(lo_excel) = lo_reader->load( i_excel2007 = lv_xstring ).
    DATA(lo_worksheet) = lo_excel->get_active_worksheet( ).
  CATCH cx_root INTO DATA(error).
ENDTRY.
ASSERT 1 = 1. " Debug helper to set a break-point

出货期评估表.XLS
Using this template, you will DUMP

If the XLSX file is uploaded, it will not be DUMP, but another issue is found, which is that the font style of some cells in the template downloaded after program conversion has changed, such as

Unfortunately, all your attachments failed to upload. Could you try again, and make sure they are effectively uploaded?

AAA.XLS

Thanks. It's not an XLSX file. It's a ZIP which contains these files:

ETExtData
Workbook
CompObj
DocumentSummaryInformation
SummaryInformation

It looks like an XLT file (Excel 97-2003 Template).
abap2xlsx supports only XLSX, XLSM (known as the formats Excel 2007 / Office Open XML).


feat(storybook): updated html title

iframe.html and index.html files in storybook-static folder (generated with build-storybook) contain <title>Webpack App</title> as default.
Added webpackFinal (for iframe.html) and managerWebpack (for index.html) into .storybook/main.js to override HtmlWebpackPlugin's title option.

Thank you for your contribution @ayhandoslu! 🎉
I'm having a look why Sonar job failed here...

Chromatic does not customize your webpack config. It invokes build-storybook to generate a static Storybook build, and uploads those files to S3. What you end up seeing published on Chromatic is 99% your static build, with some very small additions to handle embeds and such. The page title is not rewritten in any way. Can you run a static build yourself (npm run build-storybook) and verify the contents of storybook-static/index.html?

Thank you for the clarification @ghengeveld! Now, when I check same url on chromatic I can see desired title there too. Weird but maybe It was a temporary cache issue.
Now I don't see any issues. Thank you all!


Fix RPath

Is this pull request associated with an issue(s)?
Please list issues associated with this pull request, including closing words
as appropriate.
N/A
Description
Describe what this pull request will accomplish.
My goal is to fix the rpath issues encountered when a shared library needs to find the libraries installed by TiledArray. In that process I am learning how the package manager component works and writing up notes. At present this PR is just notes.
TODOs
For draft pull requests please include a list of what needs to be done and check
off items as you complete them.

[ ] finish notes.
[ ] fix rpath.

@zachcran can you confirm that CMakePackageManager::get_package is supposed to be overwriting PackageManager::fetch_package?

@zachcran can you confirm that CMakePackageManager::get_package is supposed to be overwriting PackageManager::fetch_package?

@ryanmrichard Yes, it looks like CMakePackageManager::get_package is supposed to be overwriting PackageManager::fetch_package. One or the other should probably be changed so they are the same name. It looks like that  function documentation is completely wrong at this point after various refactorings. Sorry about that!


can you teach me how to use it

Install live streaming software, such as Tivimate, Televizo, Perfect Player, DIYP, or TVbox. Tivimate is recommended.
Import live streaming sources by copying and pasting my live streaming subscription link into the software.
Import Electronic Program Guide (EPG).
Note: Please make sure you are in mainland China and have an IPv6 environment.


deps: update to<EMAIL_ADDRESS>

Let's incorporate a changelog update (what got fixed) + preparation for patch release right away.


Bug: No typescript support

Prerequisites

[X] I have searched the repository’s issues and Kinde community to ensure my issue isn’t a duplicate
[X] I have checked the latest version of the library to replicate my issue
[X] I have read the contributing guidelines
[X] I agree to the terms within the code of conduct

Describe the issue
There is no typescript inference and intellisense, upon furthering investigating, I have seen the types are probably not "connected" to the library please fix it soon as possible, as I cannot correctly use the library as there is no intellisense
Library URL
https://github.com/kinde-oss/kinde-auth-nextjs
Library version
^2.3.8
Operating system(s)
Other Linux
Operating system version(s)
Fedora 39
Further environment details
No response
Reproducible test case URL
No response
Additional information
No response

Especially in the withAuth middleware function

thanks @abdulahd2008 - we'll add some TS support to the middleware

Also the login and register function is available only in react sdk

Hey @abdulahd2008 would you be able to expand on that comment about login + register? Just want to know exactly what you mean so we can help out :)


How can I download the block backup data?

Now, https://eosnode.tools/blocks does not provide the block backup data download anymore. And I can download the block backup data from AWS S3. So how can download the block backup data now?

This issue has nothing to do with the EOSIO software.  I recommend posting the question to relevant telegram or stack exchange groups.
You also do don't indicate which network you want to download blocks for.  There are many blockchains based on EOSIO.


Not working with latest HidHide release

Anytime I'm trying to connect the Stadia controller, I am greeted with this constant, repeated message:

This has apparently started happening after updating HidHide.
Here's my current HidHide configuration:

If I disable the hiding, the program recognizes the controller as usual.
What could be done to fix this?

Sorry about the second screenshot; the first option is actually disabled (I ticked it when doing the screenshot accidentally)

I solved it by re-adding the stadia-vigem-x64.exe Application again. It seems like the last update messed up some paths.


Improve documentation, partly resolving issue #147.

Thanks for making this PR, I modified it and merged


[cli] Clean up version flag handling

What does this PR do?
Improves handling of the version info from the CLI. Version handling inlined.
Motivation
Make the version print out be minimal
Additional Notes
N/A
Possible Drawbacks / Trade-offs
N/A
Describe how to test/QA your changes

Build
Run datadog-secret-backend --version

/merge


Added  Whole website JavaScript Projects Showcase with Search and Admin Update Functionality

Developer Checklist

[ x] Followed guidelines mentioned in the readme file.
[x] Followed directory structure. (e.g. ProjectName/{USERNAME}/...yourfiles)
[x] Starred ⭐ the Repo (Optional)

Pull Request Title: Added JavaScript Projects Showcase with Search and Admin Update Functionality
Issue:#942
Description:
This pull request introduces a new feature to our website that showcases various JavaScript projects. It includes a search feature to easily find specific projects. Additionally, an admin update functionality has been implemented to allow administrators to add new projects via the /Update route.
Changes Made:

Added a new page to showcase JavaScript projects.
Implemented a search functionality to quickly find projects by name or keyword.
Introduced an admin update route /Update for adding new projects.
Enhanced the user interface for a seamless experience.
Here is deployed url:
https://testing-website-three.vercel.app/
please see

Screenshots:
Video
https://github.com/thinkswell/javascript-mini-projects/assets/107855172/695a0b46-5539-4ef6-8dc6-1179944ee7de
Please see this video
Testing:

Performed extensive testing to ensure the search and admin update functionalities work as expected.
Tested the new feature on different devices and browsers to ensure responsiveness.
Verified that the admin update functionality only allows authorized users to add projects.

To-Do:

[ ] Add detailed documentation for the admin update route.
[ ] Conduct further testing for edge cases and performance optimizations.
[ ] Address any feedback and suggestions from the team.

Related Issues:

Closes #IssueNumber (if applicable)

Additional Notes:

This feature is expected to improve the user experience and provide better project management options for administrators.

@JayeshYadav99 Add a README.md to explain your stack, credits if any assets used and requires to be credited, also Explain everything to a Dev and a Non-Dev, so everyone would be able to understand.
Few pointers being,
Non- Dev (need):

How to install
How to use
etc..,
Dev (need):
How to clone and update
Requirements.txt if any
Add tech used.
Skills used
Attach a Image and Vid in the README.md section.
Also describe the license for this project.

As it is not a normal project, but a project to show collection of project.

@JayeshYadav99 Thanks, for such an wonderful, project.
Will be thrilled to know, what frameworks you used, etc...,
Because this repository is not just a collection of projects, but a collection of projects where we can learn from it, also.

@JayeshYadav99 Love this.

@JayeshYadav99 That is actually superb.

@NitkarshChourasia @PBJI ,I have made new pull request #956   beacuse of folder structure i have added website code in "Website " folder


Werkzeug, Flask app Working outside of request context error

Environment:

Python version: 3.7
Werkzeug version: 2.0.0

cli_1  |   File "/usr/local/lib/python3.7/site-packages/werkzeug/local.py", line 544, in _get_current_object
cli_1  |     return self.__local()  # type: ignore
cli_1  |   File "/usr/local/lib/python3.7/site-packages/flask/globals.py", line 37, in _lookup_req_object
cli_1  |     raise RuntimeError(_request_ctx_err_msg)
cli_1  | RuntimeError: Working outside of request context.

I was using the Werkzeug==1.0.1 in my Flask app, today it was upgraded to version 2.0.0 and the pod went down, the app couldn't start, the reason was the issue above,
I quickly did a downgrade to Werkzeug==1.0.1 all good but using the newer version cause this problem

There is insufficient information here for us to figure this out. Could you provide a code snippet that reproduces the problem?


updated codeowners

Description
Changes proposed in this pull request:

updated codeowners
...
...

Related issue(s)

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Sets TTL support for mongodb state store to enabled

Implemented here.

@hhunter-ms code PR is merged. @JoshVanL to make sure this docs PR matches the final implementation.

As I understand, this is all the docs I need. Unless I need to add some releases notes, or similar, somewhere?

@JoshVanL should the MongoDB component spec be updated to reflect this?

@hhunter-ms thanks, I agree. We should update that page with a mention of TTL like the others. Leave that to me

Done


What should we do with the original WinXP .sst files?

Should the new .sst files be placed next to the original Windows XP .sst files in a Package Installer or should we replace the "roots.sst" with the "WindowsRoot.sst" file and the "updroots.sst" with the "WindowsIntermediate.ss" file?
https://msfn.org/board/topic/175170-root-certificates-and-revoked-certificates-for-windows-xp

Once you've imported the certificates, Windows doesn't need the files anymore. So you can delete the old ones. And the new ones after importing if you'd like.

Do you know a command code to install this with a Batch file?


Cannot be compiled to json with ttystudio

Perhaps an issue with vendor code, but creep cannot be compiled to json format by the toolset in ttystudio https://github.com/chjj/ttystudio#choosing-a-new-font-for-your-terminal-recording
It would be very nice to use this font for terminal recording, and drawing from it's json pixel format. It seems as though this script cannot find the glyphs properly. I have used this script on other bdf formatted fonts and it's always worked.
vendor/pxxl.js:755
     var bitmap = glyph.bitmap;
                       ^
TypeError: Cannot read property 'bitmap' of undefined

Feel free to close if this issue is a 'can't care, other guys script problem' type nit.

Since the actual error is in pxxl.js, that repo is https://github.com/remcoder/Pxxl.js in fact.

I'm sorry I didn't answer yet! I'll look into it soon!

Any progress on this?


About waveletpacket reconstruct

Hello， my friends，
Why did I get the wrong data length from waveletpacket reconstruct?
For example:
x = [1, 2, 3, 4, 5, 6, 7, 8]
wp = pywt.WaveletPacket(data=x, wavelet='db2', mode='symmetric', maxlevel=3)
new_wp = pywt.WaveletPacket(data=None, wavelet='db2', mode='symmetric')
new_wp['aaa'] = wp['aaa'].data
print(new_wp.reconstruct(update=True))

the length of new_wp is not equal to x

This is a bit tricky, but let me try to explain what is going on. I think this is a subtlety that we have not documented well.
The first point is that for the DWT or wavelet packet transforms with all boundary modes other than 'periodization' some additional boundary coefficients must be retained to enable perfect reconstruction. During the inverse transform it is necessary to know what the original signal size was in order to properly trim any excess coefficients upon reconstruction. You might think we could guess, but it is actually not unique as the following short example shows:
import numpy as np
import pywt

c1 = pywt.dwt(np.random.randn(9), wavelet='sym4', mode='symmetric')
print(c1[0].size)
c2 = pywt.dwt(np.random.randn(10), wavelet='sym4', mode='symmetric')
print(c2[0].size)

Both length 9 or length 10 input to DWT gives 8 approximation coefficients. So, just from the size of the coefficients we cannot guess which was the original signal length.
In your wp wavelet packet object, when you created from an existing set of data of length 8, it actually stored that length in a private attribute ._data_shape. You can inspect it via:
print(wp._data_shape)`

and you will see (8,). This allows that transform to automatically trim any excess coefficients back to the original length on reconstruction. So if you call wp.reconstruct(update=True) you will get a length 8 signal.
In the second case, you created a WaveletPacket with data=None, so this shape information was unavailable and the transform kept some extra coefficients on reconstruction that it thought might be needed. This is due to the ambiguous original signal length issue discussed above. There are currently two ways to resolve this in your example:
1.) initialize new_wp using data=np.zeros(len(x)). Then your last call to reconstruct will give you a length 8 signal.
2.) Alternatively you can keep only the first 8 entries in the array returned by new_wp.reconstruct

Dear friends,
Thanks for your detailed and patient reply! I know how to solve my problem and why I get more coefficients than the original signal length.
Thank you very much!!!
Best wishes,
seventeen
seventeen<EMAIL_ADDRESS><EMAIL_ADDRESS>
------------------ 原始邮件 ------------------
发件人:                                                                                "PyWavelets/pywt"<EMAIL_ADDRESS>发送时间: 2020年8月14日(星期五) 凌晨2:56
收件人<EMAIL_ADDRESS>抄送: "陈曦"<675284835<EMAIL_ADDRESS>主题: Re: [PyWavelets/pywt] About waveletpacket reconstruct (#561)
This is a bit tricky, but let me try to explain what is going on. I think this is a subtlety that we have not documented well.
The first point is that for the DWT or wavelet packet transforms with all boundary modes other than 'periodization' some additional boundary coefficients must be retained to enable perfect reconstruction. During the inverse transform it is necessary to know what the original signal size was in order to properly trim any excess coefficients upon reconstruction. You might think we could guess, but it is actually not unique as the following short example shows:
import numpy as np import pywt c1 = pywt.dwt(np.random.randn(9), wavelet='sym4', mode='symmetric') print(c1[0].size) c2 = pywt.dwt(np.random.randn(10), wavelet='sym4', mode='symmetric') print(c2[0].size)
Both length 9 or length 10 input to DWT gives 8 approximation coefficients. So, just from the size of the coefficients we cannot guess which was the original signal length.
In your wp wavelet packet object, when you created from an existing set of data of length 8, it actually stored that length in a private attribute ._data_shape. You can inspect it via:
print(wp._data_shape)`
and you will see (8,). This allows that transform to automatically trim any excess coefficients back to the original length on reconstruction. So if you call wp.reconstruct(update=True) you will get a length 8 signal.
In the second case, you created a WaveletPacket with data=None, so this shape information was unavailable and the transform kept some extra coefficients on reconstruction that it thought might be needed. This is due to the ambiguous original signal length issue discussed above. There are currently two ways to resolve this in your example:
1.) initialize new_wp using data=np.zeros(len(x)). Then your last call to reconstruct will give you a length 8 signal.
2.) Alternatively you can keep only the first 8 entries in the array returned by new_wp.reconstruct
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Hello, and thank you for the great explanation. I just have another related question. I have an image of size 512512, and many of the implemented wavelet families give larger coefficients (e.g. 258258 or 280*280. I am using pywt.dwt2()).
If I wanted to represent the wavelet transform in a traditional way (stacking the coefficients to form an image of the same size of the original, with the LL coeff in the upper left corner and the HH in the bottom right), what would be consider best practice? To form my image with a larger size? Or to trim/reshape the coefficients? If trimming, which coefficients to discard?
Thank you very much,
Alejandro Noguerón

On Sat, Oct 10, 2020 at 9:52 AM Alejandro-1996<EMAIL_ADDRESS>wrote:

Hello, and thank you for the great explanation. I just have another
related question. I have an image of size 512512, and many of the
implemented wavelet families give larger coefficients (e.g. 258258 or
280*280. I am using pywt.dwt2()).
If I wanted to represent the wavelet transform in a traditional way
(stacking the coefficients to form an image of the same size of the
original, with the LL coeff in the upper left corner and the HH in the
bottom right), what would be consider best practice? To form my image with
a larger size? Or to trim/reshape the coefficients? If trimming, which
coefficients to discard?

Hi Alejandro,
If you run the transform with wavedec, wavedec2 or wavedecn then there is
already a utility called coeffs_to_array that will stack the coefficients.
For example:
image = pywt.data.camera()
coeffs = pywt.wavedec2(image, wavelet='sym4', mode='symmetric', level=4)
stacked, coeff_slices = pywt.coeffs_to_array(coeffs)
In this case, mode='symmetric' will result in some extra coefficients, so
in order to stack into a contiguous array, padding with zeros as needed is
performed. This array will be slightly larger than the original image,
though. For the example above, the original is (512, 512) but the stacked
version is (538, 538). To get identical size to the original you would have
to use mode='periodization' (with a signal that is a multiple of 2**level
in length along each axis).
There is not currently an option to discard any boundary coefficients when
stacking, but if you wanted to implement that, you should discard the
samples from from the end, not the start. The issue with discarding
coefficients is that you would no longer be able to do the reverse
operation (array_to_coeffs followed by waverecn) to get back the original
signal.

Thank you very much,
Alejandro Noguerón
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/PyWavelets/pywt/issues/561#issuecomment-706552383,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABRZ7PKJFXPBRNHLQK73FQDSKBRJVANCNFSM4P4I3W4Q
.

Hello,
Thank you for the for the explanation. I have been searching all day for a discussion like this. How would one resolve this issue when the WaveletPacket class is not being used?
I am facing this issue when I decompose a 1D signal using wavedec, then reconstruct with wavrev, and wacerec does not have a data argument that can be used to set the desired output size. Can I simply throw away the extra entry in my array generated by waverev as you suggest? Does one always pick the final entry in the reconstructed array as the one to throw out? I would really love to see a reference for that, please forgive my ignorance.

Yes, the coefficients to discard will all be at the end. You can verify by tacking the round trip transform with random data to see that it matches (to within some floating point precision)

To clarify for others who may read this, you mean to reconstruct the signal, then discard the final entry in the reconstructed signal, right? You referred to the items to drop as coefficients in your reply, so I want to be sure you are not saying to drop the final wavelet coefficient, then reconstruct the signal with (n-1) coefficients, where n is the level defined in the wavedev used to decompose the signal.

Yes, I meant drop the last samples of the reconstructed signal

This is a bit tricky, but let me try to explain what is going on. I think this is a subtlety that we have not documented well.

The question was answered, but let's leave this open as a documentation task.


Calendar pane has disappeared

I was able to use Periodic Notes on Sunday without a problem. Today when I went in to make a change, I could not get the calendar to appear. I have disabled and re-enabled, closed and reopened the vault, etc. but no luck. It still tells me that the plug-in that created the pane is no longer active.

Hi @pdworkman, the calendar is a separate plugin from Periodic Notes. If you have the Calendar plugin installed though, double check that the right sidebar isn't collapsed.
There is a command called "Toggle right sidebar," so first check if the pane is just hidden.

If the sidebar is showing and there's still no calendar, can you search in the command palette for Calendar: Open View

That command will only appear if you accidentally closed the calendar.

Let me know if any of that fixes your issue!

I have resolved.
Calendar: Open View did not show up in the command palette.
I went to the plugins and Calendar was disabled. I tried to enable it, and got the message that it could not be enabled. I removed the plugin, then reinstalled it. Calendar could then be toggled on. I still did not get the Calendar: Open View command in the command palette. I did get the little "go to today" icon on the left side, which would open my today note, but still didn't show the calendar in the right sidebar.
I closed the vault and reopened it, and then got the calendar tab in the right sidebar.
Now working!
I am not sure what might have triggered this problem.

Live Preview came out this weekend, and I did turn it on for part of Sunday, then found that Advanced Tables didn't work and disabled it.
I started using Sync to synchronize this vault on Sunday.

I don't think the Live Preview release would have affected it but maybe Sync caused it to get disabled. Glad to hear you have it working though!

I tried to enable it, and got the message that it could not be enabled. I removed the plugin, then reinstalled it

This is a little concerning though, I wonder if you ran into a bug that caused it to get disabled instead. It will be hard to debug unless you had a copy of the error logs though.


the main(config) defination is wrong

it should be like this:
def main(unused, config, ngpus_per_node):

Yes, you are right.


Support binary input fields

JSON does not natively support byte data, so I don't think it's possible to deploy a model with a binary input field.

Thanks for reporting @Andrew-Crosby. In an ideal world, how would you want this to be supported? The options that occur to me are:
POST the binary as a simple blob (i.e. with Content-Type: application/octet-stream instead of application/json)
For: simplest to implement in model and client(s)
Against: API does not directly reflect model input schema
Against: precludes support for models with multiple inputs
Assume binary data columns/fields will be sent as base64 encoded
For: retains JSON as the API and Content-Type - and so supports multiple inputs
Against: API does not directly reflect model input schema
Explicitly state that binary is not supported, suggest that MLflow model be modified to decode a base64 encoded column
For: retains JSON as the API and Content-Type - supports multiple inputs
Against: forces MLflow model to address a concern specific to real-time inference up-front


Add Brightway to Featured Example Gallery Grid

Hello again, dear pydata-sphinx-theme devs - again many thanks for your hard work on this fantastic theme!
In these two PRs:

https://github.com/pydata/pydata-sphinx-theme/pull/1119
https://github.com/pydata/pydata-sphinx-theme/pull/1232

I had a discussion with @12rambau on adding our documentation brightway-documentation to the featured example gallery.
At the time, @12rambau suggested that I add our project to the "Other projects using this theme" section, while remaining open for moving to the featured example gallery later.
We have since completed our second revision of our documentation pages.
From what I can see in the other examples, ours is still the only project using git submodules (+GitHub Actions for updating) extensively. It took quite a while to set up, but we now have a rather neat setup, which is explained in detail in the readme
I am certain this would be useful to projects looking to include docstrings from multiple repos in their API documentation. Despite the obvious advantages of a monorepo, some communities won't be able to give up distributed repos anytime soon.
With the extensive technical description in our readme, we feel that our page provides real benefit for potential (first-time) users of the theme that want to use submodules or GH actions in their documentation.
Let us know what you think. Thanks!

Sorry for the long review, I'm still in to move Brightway to the featured example of the gallery for all the reasons you mentioned in your comment. I'll approve this one and other maintainers decide if they agree or not.


iox-#2007 Adapt docs with new meetup cadence

Pre-Review Checklist for the PR Author

[x] Code follows the coding style of CONTRIBUTING.md
[x] Tests follow the best practice for testing
[x] Changelog updated in the unreleased section including API breaking changes
[x] Branch follows the naming format (iox-123-this-is-a-branch)
[x] Commits messages are according to this guideline

[x] Commit messages have the issue ID (iox-#123 commit text)
[x] Commit author matches Eclipse Contributor Agreement (and ECA is signed)

[x] Update the PR title

Follow the same conventions as for commit messages
Link to the relevant issue

[x] Relevant issues are linked
[x] Add sensible notes for the reviewer
[ ] All checks have passed (except task-list-completed)
[x] All touched (C/C++) source code files from iceoryx_hoofs are added to ./clang-tidy-diff-scans.txt
[x] Assign PR to reviewer

Notes for Reviewer

Update documentation about monthly meetup cadence

Checklist for the PR Reviewer

[ ] Commits are properly organized and messages are according to the guideline
[ ] Code according to our coding style and naming conventions
[ ] Unit tests have been written for new behavior
[ ] Public API changes are documented via doxygen
[ ] Copyright owner are updated in the changed files
[ ] All touched (C/C++) source code files from iceoryx_hoofs have been added to ./clang-tidy-diff-scans.txt
[ ] PR title describes the changes

Post-review Checklist for the PR Author

[ ] All open points are addressed and tracked via issues

References

Closes #2007

@ariexi Could you please change the Eclipse Zoom invite accordingly?  :pray: I would suggest the starting date for the new invite to be 2023/09/07.

@ariexi Could you please change the Eclipse Zoom invite accordingly?  :pray: I would suggest the starting date for the new invite to be 2023/09/07.

Just one problem, I am currently on vacation and I will back on September 12th...
After that, I can change the invite, till then, you can use the current zoom link I guess.

@ariexi once you are back from vacation, would it be possible to use a free service which is build on open source and respects privacy for the meetings? Something like jitsi? It works without user name and password and everybody can easily join, e.g. https://meet.jit.si/eclipse-iceoryx. I'm always hesitant to enable the camera on Zoom.
cc @mossmaurice

would it be possible to use a free service which is build on open source and respects privacy for the meetings? Something like jitsi?

We can discuss that in the first developer meetup after the holiday season on 2023/09/07.

would it be possible to use a free service which is build on open source and respects privacy for the meetings? Something like jitsi?

We can discuss that in the first developer meetup after the holiday season on 2023/09/07.

@mossmaurice, @elBoberido
I am not able to provide another service then Zoom as Eclipse uses Zoom, sorry.
Is there a decision for changing the tool or will you still use Zoom?
Should I re-schedule the meeting series, beginning on September 7th?

would it be possible to use a free service which is build on open source and respects privacy for the meetings? Something like jitsi?

We can discuss that in the first developer meetup after the holiday season on 2023/09/07.

@mossmaurice, @elBoberido I am not able to provide another service then Zoom as Eclipse uses Zoom, sorry. Is there a decision for changing the tool or will you still use Zoom? Should I re-schedule the meeting series, beginning on September 7th?

I created a help desk issue at Eclipse

@ariexi regarding Zoom meeting. There was no conclusion at the last meeting, just that we would like to move to something else if it is allowed by the Eclipse guidelines or if we could ask Eclipse for an exception.
A good alternative would be the video call feature of gitter (nowadays just a rebranded  Element Web client). One can access the meeting with a link without the requirement to register. This is the link to a dummy meeting I created on gitter https://app.element.io/jitsi.html#conferenceDomain=meet.element.io&conferenceId=JitsiVtfrqukadefbqiqfxryxabai&isAudioOnly=false&startWithAudioMuted=false&startWithVideoMuted=true&isVideoChannel=false&displayName=&avatarUrl=&userId=&roomId=!AooDAAwkyNWwkMElpt%3Agitter.im&theme=&roomName=eclipse%2Ficeoryx&supportsScreensharing=false&language=de

I had a chat w/ EF...
https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/3678

Projects don't need to use Zoom. They can use whatever, as long as it's a open field for all.
We regard a video conferencing solution as an external resource.

So you have to follow the external resource

@ariexi thanks very much for clarifying this.
@mossmaurice can we put this on the agenda for the next meetup?


Default behavior of inserting a hyperlink inserts it to the front of text

Repro Steps:

C&P'ing from https://github.com/microsoft/roosterjs-react/blob/master/sample/script/ReactEditor.tsx
Click the Ribbon "Insert Hyperlink"
Type or C&P the url
Click ok
The hyperlink is inserted to the front of the entire text block, not to where the previous cursor position was.
Note that I verified the component is remembering the cursor position in general, even after clicking insert hyperlink then cancelling, the prior cursor position is remembered.

Env:

"roosterjs-react": "0.0.71",
"roosterjs-react-ribbon": "0.0.71",
The ReactEditor.tsx is being opened from an Office Fabric React Panel.

This repo is archived now. For react wrapper of roosterjs, please refer to roosterjs-react package in roosterjs repo. Thanks.


projections database add-on

Hey, thanks for this awesome library. I've been thinking of starting a new project, something similar to this but for keeping track of experts' projections.
I just thought I'd ask for advice, as I'd want it to integrate with nfldb--at least key off of game ID and player ID. But how tight is the question. Are things going to screw up if I had more tables to the database? And how hard would it be to leverage the query system as well? Appreciate any thoughts you might have.
As for how to make it available, I think the best option is a separate package with a dependency of nfldb. But if you think it would make more sense as a pull request, or even a fork, let me know.

@BurntSushi to keep track of the add-on version, would you recommend adding a value to meta or adding a new table?

@BurntSushi to keep track of the add-on version, would you recommend adding a value to meta or adding a new table?

@hsharrison Unless you have a really good reason for doing so, I would very much try to stay away from modifying existing tables.

@hsharrison Unless you have a really good reason for doing so, I would very much try to stay away from modifying existing tables.

Makes sense, thanks.

Makes sense, thanks.

The ERD is a big spaghetti mess but I think I've got all the major issues sorted bar one: handling games that aren't in nfldb yet. I'm at a bit of a loss here. The best plan I can think of is to use some placeholder value in gsis_id and have a separate script that scans for new games and updates the ids accordingly. It could be incorporated into an nfldb-update wrapper for convenience.
What do you think? Is there better option that I'm missing?

Could you outline the use case here? Can it just not add info for games that don't exist in the database? Generally speaking, games can be in nfldb as soon as the schedule for them is known. For example, when the season starts, an entry for every game in the season gets added as soon as the schedule is released. Similarly for playoffs---you just have to update the nflgame schedule.

Well, that solves the problem then!
I just saw the comment in nfldb.update about only inserting games into the database 15 minutes before kickoff (if I'm interpreting that correctly) so I guess the solution is to grab the IDs from nflgame instead of nfldb.

Are you looking at the games_schedule function? Note the docstring. :-)

Returns a list of GSIS identifiers corresponding to games that
have schedule data in the database but don't have any drives or
plays in the database. In the typical case, this corresponds to
games that haven't started yet.
The list is sorted in the order in which the games will be played.

Unfortunately, the update script is rather complex. It has to be smart about what it updates, or else it will be really slow. e.g., It will bulk insert schedule data (which are just rows in the game table). But it has to avoid pinging NFL.com for drive/play data for every game, otherwise it would be pretty ridiculous. So it checks the start date/time of the game to see whether it should query for additional data.
And once a game is over, it never changes.
An interesting property of this approach is that it is idempotent. If you run nfldb-update on a completely updated database, it should make no changes. If you delete any row from the game table and re-run nfldb-update, that row and all of its drive/play data will get re-inserted automatically. (This is especially useful on the rare occasion that game data gets corrupted.)
nfldb-update is pretty smart about what it does, but it pays for that in complexity.

I was looking at this comment just a few lines below that docstring. In short, games more than 15 minutes in the future are skipped. But now I see that's just for purposes of checking for more info rather than not inserting the game at all.
So I'm not sure why I didn't have the wild-card games in my database until I updated again just now. I thought I had updated yesterday or the day before, but I guess not.
Well, it looks like this was a false alarm. I'll just make sure nfldb is updated before inserting projections, and it should be able to find all the game IDs it needs. Idempotency will come in handy, then. :-)

So I'm not sure why I didn't have the wild-card games in my database until I updated again just now. I thought I had updated yesterday or the day before, but I guess not.

That's because I just updated nflgame. You can see this comment here: https://github.com/BurntSushi/nflgame/issues/106#issuecomment-68490377 --- that command will let you update the nflgame schedules without having to wait for me to do it (after which, you'll need to do a pip install --upgrade nflgame). Once you update the schedules in nflgame, the next run of nfldb-update will pick them up.

Ooooh, everything makes sense now. Awesome.
By the way, I've spent the last couple days looking through your code extensively (in order to mimic your approach wherever possible), and I have to say it's been way easier than I expected to figure out what's going on. So thanks for all the hard work you put into this, the final product is a joy to work with.

No problem. :-) But you may not be saying that once you get to SQL generation. :-) This one is pretty crazy: https://github.com/BurntSushi/nfldb/blob/master/nfldb/sql.py#L177

Yeah, I almost added a line "but I haven't got the query system yet". :-)

Still making progress...
Any idea how I could handle DST "players"? The only option I have seems to be allowing the player_id column of the various projection tables to match with either player.player_id or team.team_id. Does Postgres allow this kind of dual foreign-key constraint? It sounds like something that will create headaches.
Alternatively, would you be willing to hardcode these rows into the player table? Or do you have any other ideas?

I definitely do not want to be adding defensive teams as players in the player table. That seems very wrong.
In all the code I've written, I've handled "defense players" as a special case unfortunately. I haven't yet discovered a good abstraction over regular players and defensive team "players."
The other option is to have two projection tables. One for regular players and one for defensive teams. I'd probably try that.

I definitely do not want to be adding defensive teams as players in the player table. That seems very wrong.

Fair enough.

The other option is to have two projection tables. One for regular players and one for defensive teams. I'd probably try that.

Just curious, what's the rationale? A pro seems to be that I could then split the offensive statistics from the defensive categories. The con is that queries become a lot more complicated if one is interested in fantasy points across all positions.
Right now I'm leaning toward the intermediary table with a trigger to keep it updated when players are added. I think it will keep things easier, but I'm curious what your reason is to prefer splitting them.

Hmm, I got a second opinion for the option of splitting them, so I'm willing to accept my intuition here is wrong.

I absolutely think you should go with your intuition. I'm not happy with any solution I've ever come up with---I'm just tossing out ideas. :-)

Just curious, what's the rationale? A pro seems to be that I could then split the offensive statistics from the defensive categories. The con is that queries become a lot more complicated if one is interested in fantasy points across all "positions".

That second point is precisely the issue. If you split them, then running one query to grab everything is going to be difficult (so I agree with you there). It may be much easier to say, "If you want projections for individual players, then write this kind of SQL. And if you want projections for defensive teams, then write this kind of SQL." If you end up developing a library or a UI around this stuff, that's where you might be able to merge them under one common interface.
Your idea of using one id column to store either a player id or a team id is interesting. You're right to point out a foreign key problem. Maybe you just don't use a foreign key for that column?
Another option is for each of your projection tables to have two columns: player_id and team_id with the invariant that exactly one of them must always be NULL.
I really don't see a clear answer here. I'd suggest trying both and see which one works out better. :-)

Alright, thanks for the advice. Sounds like our thoughts on this are similar. I'll give..something...a shot.

Still thinking, but putting this link here for posterity as it touches upon all the options we just mentioned: http://stackoverflow.com/questions/441001/possible-to-do-a-mysql-foreign-key-to-one-of-two-possible-tables


Out of memory error when trying to analyze large dump

[2024-11-06T13:29:13Z ERROR binwalk::common] Failed to read file /home/user/documents/boox/backup/flash.bin.lun0 into memory: out of memory
Analyzed 1 file for 85 file signatures (187 magic patterns) in 2.0 milliseconds

The file is very large to be fair.
115G	flash.bin.lun0
Is there a workaround other than swap, like splitting up large files into chunks and analyzing those? System memory is 64GB, with a 4GB swap.

Yes, this is a limitation of the current implementation; files are read into memory in their entirety so you need as much free memory as the size of the file(s) you're analyzing.
You can split the file into multiple smaller files and analyze them individually, though this may cause binwalk to think that some otherwise valid signatures are invalid, depending on the contents of the flash.bin.lun0 file. Example: if the entire file is a single FAT32 file system and you split the file in half, binwalk will think the FAT32 is invalid since the FAT32 image reports that it is 115GB in size, but the file you are scanning is only 57.5GB. If you set the RUST_LOG=debug environment variable, you would see the log messages report something along the lines of "Found fat magic bytes...fat extends beyond EOF, ignoring".

I

This is a bit disappointing, especially given how the 2.x series had no problems supporting it...


ref: Warn user when using capture/wrap without installing Raven

Ref: https://github.com/getsentry/raven-node/issues/396
I don't want to change breadcrumbs behaviour right now, therefore this is the least obtrusive solution.

flaky test?

yup

@jan-auer I wouldn't say flaky, as this guard was intentionally put in the code. I'd say it's just lack of any mentions about it in the docs.


[FEAT]: UX Design: Input resource tags for operator drill-down headers

When viewing an operator's drill-down, we need to display resource inputs as tags in the header (in line with the operator's title) so the user knows what resources they are working with in the operator.
Figma link.
Prime Vue tag component link.

Fill: #F7F7F7
Outline: #E1E1E1, 1px
Icon / text: #020203
16px spacing between tags, operator title and close X button

If an operator has only one input (in this case, a model) then only the one tag would be displayed in the header:

Additional resource inputs would display in horizontally aligned tags, filling the space and truncating if the names exceed the allotted space.

Any file names that exceed the allotted space should be truncated:

Let me know if you have any questions, thanks! -Neil

I'm worried we are trying to put too much information in those drill-down, to the point that they are loosing their simple appearance as an overlay screen.
They represent 80% of the application, and we do not give them the full application visual real estate.
A full page with the Terarium header and footer with a nice breadcrumb/link to represent the hierarchy of where we are in the workflow, with side panels to display inputs/outputs and their related action, would do a clearer job.
I will work on a small mock-up of what I’m thinking to see if it’s worth looking into it.

I'm not against that... sounds kinda interesting. We're using drilldown modals because they keep you in context a little better than a full screen takeover. I'm not sure how much difference 24px off the sides and top makes... but I'm curious to see what you have in mind for displaying the inputs and outputs.
Until we make a decision and build that, however, this mini-feature fixes a design fail in our current header. I often find myself working in an operator and forgetting what resource I piped into it. So I have to close it, look at the node, and then reopen it.
I don't agree that this is putting too much information in the drilldown. The name of the thing you're editing, transforming, stratifying, etc. is key information! :)


PowerShell class should have an InvokeAndClear() method

Summary of the new feature/enhancement
A classic pitfall of the PowerShell class is doing something like this:
using (var pwsh = PowerShell.Create())
{
    DateTime? firstDate = pwsh.AddCommand("Get-Date").Invoke<DateTime?>().FirstOrDefault();
    DateTime? secondDate = pwsh.AddCommand("Get-Date").Invoke<DateTime?>().FirstOrDefault();

    Console.WriteLine($"1: {firstDate}");
    // secondDate is null??
    Console.WriteLine($"2: {secondDate}");
}

In this scenario, the PowerShell object's state is affected by the first AddCommand() call and we need to clear the commands. This is easily seen here but when pwsh is a field being shared between calls it can be subtle.
In a lot of my code I end up writing this:
static class PowerShellExtensions
{
    public static Collection<PSObject> InvokeAndClear(this PowerShell pwsh) => pwsh.InvokeAndClear<PSObject>();

    public static Collection<T> InvokeAndClear<T>(this PowerShell pwsh)
    {
        try
        {
            return pwsh.Invoke<T>();
        }
        finally
        {
            pwsh.Commands.Clear();
        }
    }
}

We should consider adding that to the PowerShell SDK, since I suspect this is a very common use case.

Also the error, verbose, etc streams

Just to spell it out, because the fact that your sample code actually does work confused me:
The commands accumulate in the pwsh.Commands collection, and subsequent .Invoke() calls try to execute them all, in addition to accumulating stream output in pwsh.Streams across calls, as @SeeminglyScience  points out.
Your second .Invoke() call is therefore roughly equivalent to:
[powershell]::Create().AddCommand("Get-Date").AddCommand("Get-Date").AddArgument('1970-01-01').Invoke()

which in turn is equivalent to:
Get-Date | Get-Date

and the latter just happens to work, because Get-Date also accepts a [datetime] instance via the pipeline, so this pipeline is in effect the same as just calling Get-Date alone.
Adding a -Date argument to the 2nd GetDate call would surface the problem, however, because something like Get-Date | Get-Date 1970-01-01 indeed fails.

As for the API surface and your extension method:
If the intent is to also clear the streams, you cannot include .Streams.ClearStreams() in the .InvokeAndClear() method, as you would instantly wipe out the .Invoke() call's non-success stream output.
So I wonder if something along the following lines is called for instead:
A fluent .Reset() method that calls .Commands.Clear() and .Streams.ClearStreams() and returns the PowerShell instance itself, as the existing methods do.
This means that resetting can then either be performed stand-alone:
pwsh.Reset();  // same as: .Commands.Clear(); .Streams.ClearStreams(); return this;

Or, fluently, as part of recurring invocations on a reusable PowerShell instance:
pwsh.Reset().AddCommand("Get-Date").Invoke<DateTime?>().FirstOrDefault();

I would definitely love to see a PowerShell.Reset() method as proposed by @mklement0!
I've found myself doing pwsh.Commands.Clear(); pwsh.Streams.ClearStreams(); too (and once discovered the hard way that I had to).

Quick thought since I have the issue open:
[Flags]
public enum PowerShellClearOptions
{
    None = 0x0,
    Commands = 0x1,
    Streams = 0x2,
}

...

public PowerShell Reset() => Reset(PowerShellClearOptions.Commands);

public PowerShell Reset(PowerShellClearOptions clearOptions)
{
    if (clearOptions.HasFlag(PowerShellClearOptions.Commands))
    {
        Commands.Clear();
    }

    if (clearOptions.HasFlag(PowerShellClearOptions.Streams))
    {
        Streams.Clear();
    }

    return this;
}

I suggest that because in various scenarios, I don't want to clear the streams.


feat(#major): MUX Protocol subgraph; add MUX Protocol subgraph

@corerouter you can mark this as Ready for review once it is done and you have deployment links in the header comments ✅

@corerouter you can mark this as Ready for review once it is done and you have deployment links in the header comments ✅

Gotcha. I just finished subgraph  for MUX own Leveraged Trading Protocol, and am working on subgraph for MUX Aggregator, which include both GMX and GNS. Thanks!

Deployments:

[ ]

arbitrum: https://okgraph.xyz/?q=QmY8zEbstD3gzDyzYSHRMDFR1TLcqiJqnGNXA64dqdg4my
avalanche: https://okgraph.xyz/?q=QmVZokEXfJ4VVQMEzweWGXdQcok1dEb4rNcze4XDpzb1wb
bsc: https://okgraph.xyz/?q=QmYfiunoaB3rHogksbMHigLZo9NAKuz1XcawLdGoufgvSk
fantom: https://okgraph.xyz/?q=QmVzjqtmcQBxU5S4Rm587DsNUG8mKZrBKGnLAT7EVbDtfN
optimism: https://okgraph.xyz/?q=QmZeBzc15ySS4DudqSWn9gaU6g1SQyuwQ2f1oe7o3cBr42

Looking good! Left a few questions, and a really minor thing that I'm not sure why the linter didn't catch 💪

Thanks a lot for your help!


Feature #141: Mainnet compatibility

This PR:

Implements #141
Adds two new configurations: production-mainnet and development-mainnet
The network is passed via REACT_APP_NETWORK environment variable
Refactor getEtherscanLink to use the interface's network, not the network user is connected to

Travis automatic deployment:
https://pr184--safereact.review.gnosisdev.com
Storybook book automatic deployment:
https://pr184--safereactstorybook.review.gnosisdev.com

Travis automatic deployment:
https://pr184--safereact.review.gnosisdev.com
Storybook book automatic deployment:
https://pr184--safereactstorybook.review.gnosisdev.com

Travis automatic deployment:
https://pr184--safereact.review.gnosisdev.com
Storybook book automatic deployment:
https://pr184--safereactstorybook.review.gnosisdev.com

Travis automatic deployment:
https://pr184--safereact.review.gnosisdev.com
Storybook book automatic deployment:
https://pr184--safereactstorybook.review.gnosisdev.com


Test cases for extensions: VTK, HTML and PDF

Test cases implemented for VTK, HTML and PDF extensions.

Hi @dannyrb , PR is ready for review. The CI is failing on VTK extension tests because the 2D MPR button is not being displayed on the toolbar.
Also, the command cy.isNotInViewport started to act intermittent, so I commented it for now and added an explicit wait for 2s, just to be sure that the panel animation ended. If that's ok, we can merge this PR (that is starting to get too big) and I'll investigate the cy.isNotInViewport in another branch.
Once the VTK button is back, all tests should pass.