Fix goroutine leaks in plugin/sampling/strategystore/adaptive

Which problem is this PR solving?

Solves part of https://github.com/jaegertracing/jaeger/issues/5006

Description of the changes

This mainly involved ensuring that all goroutines started by the Processor are shut down in a Close method (which also blocks on them returning via a WaitGroup).
Adding this flagged an issue where the runUpdateProbabilitiesLoop had a long delay, so tests need to be able to override the default Processor.followerRefreshInterval, or Close would take up to ~20s to return. More context on this here https://github.com/jaegertracing/jaeger/issues/5006#issuecomment-2027292668 - specifically regarding how to override this in the Factory.
I also needed to fix a deadlock where a test was not unlocking a lock which would allow the Close method to return: https://github.com/jaegertracing/jaeger/pull/5310/files#diff-c9d6c33e36122502911585c65618d1af863079a70a68543adcc0ae2798faa348R468

How was this change tested?

make test lint

Checklist

[x] I have read https://github.com/jaegertracing/jaeger/blob/master/CONTRIBUTING_GUIDELINES.md
[x] I have signed all commits
[ ] I have added unit tests for the new functionality
[x] I have run lint and test steps successfully

for jaeger: make lint test
for jaeger-ui: yarn lint and yarn test

Haven't looked at the code yet, but from the description it sounds like the exit is simply not properly implemented if you have to wait for 20sec. If there is a loop with a timer, probably blocking in select, the good approach is to add another "stop" channel that Close function can close and it would cause an exit from select.

Ah yes, that would be better. I'll have a go at reworking it.


(BEAST) Redirect BEAST log file to output namespace

BEAST output ('log') files were being redirected to the 'logs' folder, instead of being made available downstream via the output_namespace.

@rhysinward FYI - change to BEAST module output location.


Add rationale appendix section why packets can't switch from FROM_NET to TO_NET inside main

and similarly not from FROM_HOST to TO_HOST.
Hardware reasons involving flow control / credits/etc. are the primary reason.

AI Andy: Compare two PNA-like architectures:
(a) one like the currently defined one, in which FROM_NET packets must leave as TO_HOST packets, and in which FROM_HOST packets must leave as TO_NET packets.  In such an architecture, a FROM_NET packet can leave as TO_HOST with the output port set as an Ethernet port, and then it will loop back on the host side, and come back as a FROM_HOST packet and be processed again, and unless the output port is changed to a host-side vport, it will leave as a TO_NET packet and go out an Ethernet port.  The second pass through the parser, main control, and main deparser is NOT optional for such packets in this architecture.
(b) another one that is the same, except FROM_NET packets may leave as either TO_HOST or TO_NET packets, and FROM_HOST packets may leave as either TO_NET or TO_HOST packets.  In such an architecture, a FROM_NET packet can lavea as TO_NET with the output port set as an Ethernet port, and it will NOT loop back on the host side, but go straight to an Ethernet port.
If a program is written by a developer assuming architecture (a), is it straightforward for a compiler to take that source code and compile it to a hardware target that works like (b)?  If so, how complex is the compiler's job, and does it introduce any extra processing passes in the target device?
Similar questions for a program written for architecture (b), but compiled to a hardware target that works like (a).
Note: Another issue with a hardware target that works like (b) is that there can be long bursts of time when packets are received "at line rate" from both network ports and the host, and they are all destined for the host.  Or they are all destined for network ports.  Thus the buffering/queueing parts of the NIC must be designed for this somehow.
Technically the same traffic pattern could occur for a long time for a hardware target that works like (a) -- similar issues can arise with buffering/queueing parts of the chip that could cause, e.g. if all traffic from both host and network ports were destined for the host, eventually some traffic either must be lost, or the NIC->host direction must somehow be designed with enough bandwidth to accept that total rate of traffic.  That last part seems unlikely to me, since most likely the host->NIC and NIC->host data paths would be equal or near equal bandwidths, just as the Ethernet port->NIC and NIC->Ethernet port would be equal or near equal bandwidths.

If a program is written by a developer assuming architecture (a), is it straightforward for a compiler to take that source code and compile it to a hardware target that works like (b)?

The compiler would have to make H2H and N2N packets recirculate through the (b) pipeline to achieve the (a) semantics. This would require some hardware support - not much if recirculation is already supported, but quite a bit if it isn't. Depending on the implementation, this may also halve the achievable packet rate/throughput since each packet needs to be processed twice.

Similar questions for a program written for architecture (b), but compiled to a hardware target that works like (a).

This would require inhibiting any side-effects of the second pass through the pipeline (making it into a no-op). I believe this can be achieved by making any logic in there conditional (i.e. wrap the entire P4 code in a big if statement).

Note: Another issue with a hardware target that works like (b) is that there can be long bursts of time when packets are received "at line rate" from both network ports and the host, and they are all destined for the host. Or they are all destined for network ports. Thus the buffering/queueing parts of the NIC must be designed for this somehow.

I don't see a fundamental difference between both architectures here. If the host path is oversubscribed, packets will need to be dropped, or the sources backpressured. If anything, architecture (a) will have more trouble coping since the N2H pipeline will need to process 2x the traffic.

In terms of hardware implementation, recirculation and the associated possibility of deadlock need to be carefully considered. It must not be possible to recirculate a packet infinitely through the datapath. One option is to not support recirculation. Another possibility is to include a "passes" counter which is increased at each pass and causes the packet to be dropped if it reaches a defined maximum.
In terms of software programmability, if the PNA is aiming to be a target for SmartNICs which offload vSwitch semantics, option (b) seems like a more natural fit: the switch abstraction processes packets from any source port and egresses them to any output port(s). The concept of RX/TX direction is more applicable to previous-generation NICs which did not expect to process significant amounts of traffic along the H2H and N2N paths.


Extending edges/connections with metadata?

Thanks for this implementation. Very helpful to have a TypeORM/TypeGraphQL reference implementation of the Relay spec. One question though is how I would add metadata to either Connection or Edge types using the decorators? I have been having issues trying to just add it to the decorated class. For example:
@ConnectionType(ReviewEdge)
export class ReviewConnection {

    @Field(()=> Float)
    totalReviews: number;

    @Field(()=>Int)
    averageScore: number;
}

ends up with just the pageInfo and edges showing up on ReviewConnection in the schema.
I'm not sure what I'm doing wrong because I see in the decorator class that they should be extending the target class being passed in, and so should inherit those properties right? Let me know if this is something you guys have figured out and thanks again for this repo!

@mattpetters That is a really good question. In theory it should work as you mentioned, the decorator class is extending the target but for some reason it's not.
Just to give you some context: the usage of decorators is kinda tricky because of things like that. The first implementation I did for the connection arguments was using decorators and I found the same problem you are having when adding a new argument.
Perhaps the way to solve that is removing the decorators (something that I thought about already) and using Generic Types.

That worked!
export function ConnectionType<V>(EdgeType: ClassType<V>) {

  // `isAbstract` decorator option is mandatory to prevent registering in schema
  @ObjectType({ isAbstract: true, description: 'A connection to a list of items.'})
  abstract class ConnectionClass implements Connection<V>{
    // here we use the runtime argument
    @Field(() => [EdgeType], {
      description: 'A list of edges.',
      nullable: 'itemsAndList'
    })
    readonly edges!: Array<Edge<V>>

    @Field({ description: 'Information to aid in pagination.' })
    pageInfo!: PageInfo;
  }
  return ConnectionClass;
}

Then to use it
@ObjectType()
export class ReviewConnection extends ConnectionType(ReviewEdge) {

    @Field(()=> Float)
    totalReviews: number;

    @Field(()=>Int)
    averageScore: number;
}

The same approach could be used to put metadata on the edges.

Exactly. I'm glad it worked. I'm planning on removing decorators in favor of Generic Types, it makes it more flexible.
I started creating the tests for the repository but unfortunately I'm not having much time nowadays, ASAP I will be adding them (of course if anyone is willing to help I'd be happy) and with that make a few changes that would improve even more the boilerplate.


Add filtering to references API

Signed-off-by: Michael Brown<EMAIL_ADDRESS>

we'll also need to add the endpoint API changes

@sudo-bmitch - sorry confused.. are you referring to endpoints table? separating into a 12a/12b ?

we'll also need to add the endpoint API changes

@sudo-bmitch - sorry confused.. are you referring to endpoints table? separating into a 12a/12b ?

@jdolitsky Yup. (Much to your likely disappointment.) One line for the API without a filter parameter, and one with.

I think I've made the updates requested. Anything else? @sudo-bmitch @jdolitsky


txtar: add fs.FS support

Fixes #44158
From implementation copied with permission from
https://github.com/josharian/txtarfs/blob/main/txtarfs_test.go
https://github.com/golang/go/issues/44158#issuecomment-797672335
Co-authored-by: Josh Bleecher Snyder<EMAIL_ADDRESS>

@josharian Can you try consenting one more time?

@googlebot I consent.

@googlebot I consent

@googlebot I consent.

Thanks for the effort, @carlmjohnson. It's not worth fighting the bots.


Get input shape in tf.keras.Model (Imperative API)

I use Tensorflow 2.0 and I have a model that was defined in Imperative API. In call method I use something like this:
b, h, w, c = images.shape
k_h, k_w = kernels.shape[2], kernels.shape[3]

images = tf.transpose(images, [1, 2, 0, 3])  # (h, w, b, c)
new_shape = tf.TensorShape([1, h, w, b * c])
images = tf.reshape(images, new_shape)

When I train my model with custom loop -- no problem. But I want to porting my model to SavedModel format. I use the following function:
tf.keras.experimental.export_saved_model(
        model, file_path,
        serving_only=True,
        input_signature=[
                tf.TensorSpec(shape=[None, None, None, 3], dtype=tf.float32),
            ]
        )

And i got error:
TypeError: unsupported operand type(s) for *: 'NoneType' and 'int'

Moreover, I can't do it even If I specify shape=[1, None, None, 3], because I got:
ValueError: Tried to convert 'shape' to a tensor and failed. Error: Cannot convert a partially known TensorShape to a Tensor: (1, None, None, 3)

It means that I can't do reshape at all. But I need it. How can I do it?

Here is the code with necessary changes made.
image_shape = tf.shape(images)
kernel_shape = tf.shape(kernels)
b, h, w, c = image_shape[0], image_shape[1], image_shape[2], image_shape[3],
k_h, k_w = kernel_shape[2], kernel_shape[3]

images = tf.transpose(images, [1, 2, 0, 3])  # (h, w, b, c)
new_shape = tf.TensorShape([1, h, w, b * c])
images = tf.reshape(images, new_shape)

@srihari-humbarwadi thank you!


Create STAVR.toml

Description
All previous genesis validators should name their PRs "Update {validator_alias}.toml for tesntet-15" and provide links to previous PRs merged.
If this is an UPDATE for a previous genesis validator
Checks are made against your net_address. If this has changed since the previous testnet, make sure you provide links of previous prs merged from your previous git username.
Thanks in advance!
Testnet-1
https://github.com/anoma/namada-testnets/pull/247
Testnet-6
https://github.com/anoma/namada-testnets/pull/657
https://github.com/anoma/namada-testnets/pull/823
Testnet-9
https://github.com/anoma/namada-testnets/pull/913
Testnet-14
https://github.com/anoma/namada-testnets/pull/2326
Checklist before merging

[ ] Only one toml is added in this PR
[ ] The file being added is indeed a .toml file
[ ] The toml being added is to the correct folder, and only to the correct folder
[ ] The eth_hot_key and eth_cold_key are present
[ ] The email, discord, elements telegram, and twitter fields are present and valid

Successfully validated transactions.toml file for commit hash 21326be


Allow overriding widget theme options

!default was missing in declaration.

Coverage remained the same at 99.031% when pulling 48af5b935a3cba735b089b5ae7ebc054a4d917e9 on tf:widget-theme-options-fix into f740e55573f7b54fefc5a78bcbac1eda9e59c93a on codevise:master.

Coverage remained the same at 99.031% when pulling 48af5b935a3cba735b089b5ae7ebc054a4d917e9 on tf:widget-theme-options-fix into f740e55573f7b54fefc5a78bcbac1eda9e59c93a on codevise:master.


Bug: Aerbunny fall speed

What happened?
Pretty sure aerbunnies are supposed to fall slower than normal, but they fall at normal speed.
Version
Dev
Fabric API Version
No response
Trinkets Version
No response
Relevant log output
No response
Other mods
No response
Additional Information
No response

Alright I'll use the blank format next time, this takes up a lot of space


Parameter Exploration Example

It may be worth adding a new example (or modifying an existing example) to showcase how parameter exploration can be performed as of FLAME GPU 1.5.
I.e. Load parameters from 0.xml as environment constants, using multiple initial states files in separate folders.
This could simply be a tweak to an existing model, with a new subfolder in an iterations directory showcasing this behaviour (with a readme?) and reference from the documentation.
https://github.com/FLAMEGPU/FLAMEGPU/issues/228#issuecomment-400688132

This will be described in the documentation (FLAMEGPU/docs#18) but an example will not be provided in the FLAME GPU 1 repository. Parameter exploration is a planned feature of FLAME GPU 2.x


Error during transfering data via http-data-plane

Bug Report
The provider connector fails to connect to the endpoint given by the consumer.
Describe the Bug
I want to transfer data via the http-data-plane. I can negotiate a contract and start the transfer. The transferprocess starts as aspected:
DEBUG 2023-10-10T07:16:00.689750214 TransferProcess 6831b5bd-e3d2-49c3-9867-31c20615b341 is now in state INITIAL
DEBUG 2023-10-10T07:16:01.244568756 TransferProcess 6831b5bd-e3d2-49c3-9867-31c20615b341 is now in state PROVISIONING
DEBUG 2023-10-10T07:16:01.247355506 TransferProcess: ID 6831b5bd-e3d2-49c3-9867-31c20615b341. Provisioning
DEBUG 2023-10-10T07:16:01.252211714 TransferProcess 6831b5bd-e3d2-49c3-9867-31c20615b341 is now in state PROVISIONED
DEBUG 2023-10-10T07:16:01.253795214 TransferProcess 6831b5bd-e3d2-49c3-9867-31c20615b341 is now in state STARTING
DEBUG 2023-10-10T07:16:01.256427589 TransferProcess: ID 6831b5bd-e3d2-49c3-9867-31c20615b341. Initiate data flow
DEBUG 2023-10-10T07:16:01.287385964 Transferring from HttpData to HttpData.
DEBUG 2023-10-10T07:16:01.289191089 Executing HTTP request: https://jsonplaceholder.typicode.com/users
DEBUG 2023-10-10T07:16:01.295907048 TransferProcess: ID 6831b5bd-e3d2-49c3-9867-31c20615b341. Send TransferStartMessage to http://company2:8282/api/dsp
DEBUG 2023-10-10T07:16:01.335402631 TransferProcess: ID 6831b5bd-e3d2-49c3-9867-31c20615b341. Send TransferStartMessage to http://company2:8282/api/dsp
DEBUG 2023-10-10T07:16:01.335895839 TransferProcess 6831b5bd-e3d2-49c3-9867-31c20615b341 is now in state STARTED

But when the actual data-transfer should happen, I get the following error message:
SEVERE 2023-10-10T07:16:17.025202638 Error writing HTTP data null to endpoint http://localhost:4000/api/consumer/store for request: Request{method=POST, url=http://localhost:4000/api/consumer/store}
java.net.ConnectException: Failed to connect to localhost/<IP_ADDRESS>:4000
        at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.kt:297)

I start the transfer with the following POST:
curl --request POST \
  --url http://localhost:9192/api/management/v2/transferprocesses \
  --header 'Content-Type: application/json' \
  --header 'User-Agent: insomnia/8.1.0' \
  --header 'x-api-key: ApiKeyDefaultValue' \
  --data '{
        "@context": {
          "edc": "https://w3id.org/edc/v0.0.1/ns/"
        },
        "@type": "TransferRequestDto",
        "connectorId": "consumer",
        "connectorAddress": "http://company1:8282/api/dsp",
        "contractId": "YWdncmVnYXRlZERhdGE=:YWdncmVnYXRlZERhdGE=:MDdmYmY5ZTItMWM4Yy00NTEzLTk4NzEtZWY0Zjc5MGJiODBk",
        "assetId": "aggregatedData2",
        "protocol": "dataspace-protocol-http",
        "dataDestination": {
          "type": "HttpData",
          "baseUrl": "http://localhost:4000/api/consumer/store"
        },
				"managedResources": true,
	 			"transferType": {
        "isFinite": true
    }
}'

The endpoint "http://localhost:4000/api/consumer/store" is reachable and should work for the transfer. What could be the problem here?
Expected Behavior
The data should be transfered
Steps to Reproduce
Steps to reproduce the behavior:

Run the MVD locally
Register Assets and Dataplane for Http-data-transfer
Get Contractagreement
Start Transfer

Also the HttpDataSource appears to be failing to fetch data from the sample endpoint https://jsonplaceholder.typicode.com/users, as the returned content is null. @ndr-brt  Do you have any insights regarding this issue?

your dataDestination value is telling the provider connector that the consumer is expecting the data to be pushed to an endpoint exposed at the url http://localhost:4000/api/consumer/store.
The error java.net.ConnectException: Failed to connect to localhost/<IP_ADDRESS>:4000 is telling us that there's no endpoint exposed on that port.

That's somehow not the problem. I started the server in the terminal with the following command:
HTTP_SERVER_PORT=4000 java -jar http_request_logger/build/libs/http_request_logger-0.2.0.jar
The logs confirmed that the HTTP request server logger successfully started at port 4000, which is the expected behaviour. To further verify this, I tested it with the HTTP-push sample and indeed it functioned correctly.
However, when attempting to use it in the MVD, I get this error.

did you started the endpoint into the docker network? I don't think so, because otherwise the hostname won't be localhost. the MVD is running into a docker compose network, so every other service will need to do that as well

Thank you for your help. I hadn't considered this. I'm going to close the issue now.


Builder: Action/tracking notifications "X" close button not functioning

@saleiva
Context
As I was working with multiple layers in a single map in Builder, I noticed that the action/tracking notifications were starting to stack on top of each other (see screenshot 1 below). I tried to close out of them (using the "X" as a close button) but the notification was not removed.
Screenshot 1:

Steps to Reproduce

In Builder while working on a map, complete multiple actions (adding layer, renaming layer, etc.) and notice the action/tracking notifications appear
Try to close out of the action/tracking notification by clicking the "X"
Try refreshing the page

Current Result
I was unable to close-out of the action/tracking notifications. I tried refreshing the page and this only removed the notifications sometimes. Eventually, this was the result from the action/tracking notifications building up (screenshot 2 below):
Screenshot 2:

Expected result
I expected for the action/tracking notifications to close
Browser and version
Firefox 50.0
.carto file
n/a, not an issue with the files themselves
Additional info

I was not able to replicate this behavior (I got a message saying "you cannot add more layers, please upgrade your account"), @lfgreenfield could you please share with us the .carto file? I am using also Firefox v50.


Linux: Second build with libasync yields a libevent linked application

I have a simple test project created with dub init vibe.d. When vibe.d is not available (clean build environment e.g.) using libasync in dub.sdl doesn't have a dependency to libevent
dub.sdl:
subConfiguration "vibe-d" "libasync"
$ dub build
[...]
Using Linux EPOLL for events
vibe-d:utils 0.7.27: building configuration "library"...
vibe-d:data 0.7.27: building configuration "library"...
vibe-d:core 0.7.27: building configuration "libasync"...
Enhanced memory security is enabled.
Using Linux EPOLL for events
vibe-d:http 0.7.27: building configuration "library"...
Enhanced memory security is enabled.
Using Linux EPOLL for events
vibe-d:diet 0.7.27: building configuration "library"...
Enhanced memory security is enabled.
Using Linux EPOLL for events
vibe-d:mail 0.7.27: building configuration "library"...
Enhanced memory security is enabled.
Using Linux EPOLL for events
vibe-d:mongodb 0.7.27: building configuration "library"...
Enhanced memory security is enabled.
Using Linux EPOLL for events
vibe-d:redis 0.7.27: building configuration "library"...
Enhanced memory security is enabled.
Using Linux EPOLL for events
vibe-d:web 0.7.27: building configuration "library"...
Enhanced memory security is enabled.
Using Linux EPOLL for events
vibe-d 0.7.27: building configuration "libasync"...
Enhanced memory security is enabled.
Using Linux EPOLL for events
test ~master: building configuration "application"...
Enhanced memory security is enabled.
Using Linux EPOLL for events

And ldd test yields the fine version:
        linux-vdso.so.1 =>  (0x00007ffd4f7f8000)
        libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007f975bf42000)
        libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007f975bb00000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f975b8f7000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f975b6d9000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f975b3d1000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f975b1cc000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f975ae02000)
        /lib64/ld-linux-x86-64.so.2 (0x00005578007c2000)

But building the same project a second time it will be linked against libevent. dub build:
Performing "debug" build using dmd for x86_64.
vibe-d:utils 0.7.27: building configuration "library"...
vibe-d:data 0.7.27: building configuration "library"...
vibe-d:core 0.7.27: building configuration "libevent"...
vibe-d:http 0.7.27: building configuration "library"...
vibe-d:diet 0.7.27: building configuration "library"...
vibe-d:mail 0.7.27: building configuration "library"...
vibe-d:mongodb 0.7.27: building configuration "library"...
vibe-d:redis 0.7.27: building configuration "library"...
vibe-d:web 0.7.27: building configuration "library"...
vibe-d 0.7.27: building configuration "libasync"...
test ~master: building configuration "application"...
Linking...

And so ...
$ ldd test
        linux-vdso.so.1 =>  (0x00007ffd9cf8f000)
        libevent_pthreads-2.0.so.5 => /usr/lib/x86_64-linux-gnu/libevent_pthreads-2.0.so.5 (0x00007f214ae02000)
        libevent-2.0.so.5 => /usr/lib/x86_64-linux-gnu/libevent-2.0.so.5 (0x00007f214abbc000)
        libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007f214a952000)
        libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007f214a510000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f214a2f2000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f2149fe9000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f2149de1000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f2149bdd000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f2149812000)
        /lib64/ld-linux-x86-64.so.2 (0x00005605b45bd000)
        libevent_core-2.0.so.5 => /usr/lib/x86_64-linux-gnu/libevent_core-2.0.so.5 (0x00007f21495e8000)

I am running on Linux x64. DMD Version 2.069.2. Problem als persists with 2.070.0. Dub version 0.9.24

Building with the verbose flags shows that Dub is passing -version=VibeLibeventDriver

You can remove the ~/.dub/packages directory as a workaround.

This was caused by D-Programming-Language/dub#745 (fixed on master).


Feature_Request : Make a Different Login Page #26

Feature_Request : Make a Different Login Page #26

Title :Feature_Request : Make a Different Login Page
Issue No. : #26
Code Stack : HTML, CSS
Close #26

Video/Screenshots (mandatory)

https://github.com/apu52/Travel_Website/assets/147996970/3ac586e7-28a7-424f-acf7-7b987e15639b
Checklist:

[X] I have mentioned the issue number in my Pull Request.
[X] I have commented my code, particularly in hard-to-understand areas
[X] I have gone through the  contributing.md file before contributing

Additional context (Mandatory )
Are you contributing under any Open-source programme?

@Sneha6003 I will again like to request you to make a completely new branch in your forked repo & then make all the changes . Don't make it in the previous branch


SNAP-393: Disable SDFS settings in configuration step when user mirgate…

… to new instance.

LGTM


Need help understanding FFmpeg implementation in speech-recognition-from-url for livestream decode of audio stream

Hello, I am getting live stream audio which I want to convert into format readable my Sherpa_ONNX.
I followed the example code in speech-recognition-from-url.py which uses ffmpeg to convert the audio data using this command:
ffmpeg_cmd = [
"ffmpeg",
"-i",
args.url,
"-f",
"s16le",
"-acodec",
"pcm_s16le",
"-ac",
"1",
"-ar",
"16000",
"-",
]
process = subprocess.Popen(
    ffmpeg_cmd, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL
)

frames_per_read = 1600  # 0.1 second

My first question is regarding this, does ffmpeg here convert the streaming data in real time to pcm_s16le format and does it need data to be saved in memory (on file) to do it? or does it work in real-time livestream without creating a file. Can you explain what impact frames_per_read  and samples.astype(np.float32) / 32768, here why we divide by 32768 how this code block works:
   data = process.stdout.read(frames_per_read * 2)
    if not data:
        break

    samples = np.frombuffer(data, dtype=np.int16)
    samples = samples.astype(np.float32) / 32768 <- What exactly is this line doing and why we divide by 32768
    stream.accept_waveform(16000, samples)

Second question is, I tried implementing similar logic in typescript although I am trying for a audio file for now instead of audio stream, my command looks like this:
const lsProcess = ffmpeg_child_process.spawn('ffmpeg',['-i', 'input.wav', '-f', 's16le', '-acodec', 'pcm_s16le','-ac','1','-ar','16000','new_try.wav','-y'],);
I am able to convert audio but when I try to decode it using online-client-websocket-decode-files.py I get this error:
raise Error('file does not start with RIFF id')
wave.Error: file does not start with RIFF id
Am I doing anything wrong, do I need to also implement the frames_per_read and samples.astype(np.float32) /32768?
Any help or clarity on how the code works will be very much appreciated. Thank you.

My first question is regarding this, does ffmpeg here convert the streaming data in real time to pcm_s16le format and does it need data to be saved in memory (on file) to do it? or does it work in real-time livestream without creating a file.

As you can read in the following code
https://github.com/k2-fsa/sherpa-onnx/blob/009ed2cd3095de14f611f6a3261d34a7ee96b4a2/python-api-examples/speech-recognition-from-url.py#L8-L9
The code also supports a live stream URL. So my reasoning is that it won't create a file. Otherwise, if the stream never ends, how large would be the created file ?
The processed data is pushed to a pipe in the RAM.

Can you explain what impact frames_per_read

If you want to read data from the pipe, you must decide how much to read. The more you read, the more time you have to wait until the requested amount of data is available.

samples.astype(np.float32) / 32768, here why we divide by 32768

The feature extractor expects that the input samples are in the range [-1, 1].

how this code block works:

It reads some data from the pipe
convert the data into the range [-1, 1]
send the data to the stream object for processing

const lsProcess = ffmpeg_child_process.spawn('ffmpeg',['-i', 'input.wav', '-f', 's16le', '-acodec', 'pcm_s16le','-ac','1','-ar','16000','new_try.wav','-y'],);

Please send us the file new_try.wav.

do I need to also implement the frames_per_read and samples.astype(np.float32) /32768?

Could you explain why you need this? You said you processed the new_try.wav file with online-client-websocket-decode-files.py, but that file does not need you to write any code. Could you tell us what extra steps you have done?
If you use files from us, you don't need to write any code. Everything has been done by us.
All you have done is to provide the data.
If the data does not have the requested format, you would get an informative error message.

Thank you for the response,
As per what you have said here let me explain the process which I am following:

Could you explain why you need this? You said you processed the new_try.wav file with online-client-websocket-decode-files.py, but that file does not need you to write any code. Could you tell us what extra steps you have done?

I have a Typescript Server which is getting Audio data in a format that is not compatible with Sherpa Onnx as it is in PCMU format and Sherpa ONNX expects PCM format
For this conversion I implement the following code to preprocess the data and convert it into PCM format in my TypeScript  server:

const lsProcess = ffmpeg_child_process.spawn('ffmpeg',['-i', 'input.wav', '-f', 's16le', '-acodec', 'pcm_s16le','-ac','1','-ar','16000','new_try.wav','-y'],);
I do not have new_file.wav but I can share its configuration according to FFmpeg logs:
nput #0, wav, from 'e5ee3a4f-d250-4901-b311-a12ea85216cd.wav':
Duration: 00:00:04.52, bitrate: 64 kb/s
Stream #0:0: Audio: pcm_mulaw ([7][0][0][0] / 0x0007), 8000 Hz, 1 channels, s16, 64 kb/s
File 'new_try.wav' already exists. Overwrite? [y/N] y
Stream mapping:
Stream #0:0 -> #0:0 (pcm_mulaw (native) -> pcm_s16le (native))
Press [q] to stop, [?] for help
Output #0, s16le, to 'new_try.wav':
Metadata:
encoder         : Lavf60.16.100
Stream #0:0: Audio: pcm_s16le, 16000 Hz, mono, s16, 256 kb/s
Metadata:
encoder         : Lavc60.31.102 pcm_s16le
[out#0/s16le @ 0x7291500] video:0kB audio:141kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
size=     141kB time=00:00:04.52 bitrate= 256.1kbits/s speed=1.64e+03x
has context menu
Its raw audio input format is pcm_mulaw according to FFmpeg logs and I followed the ffmpeg code which you had in the speech_recognition_from_url.py file to convert it to PCM s16le
const lsProcess = ffmpeg_child_process.spawn('ffmpeg',['-i', 'input.wav', '-f', 's16le', '-acodec', 'pcm_s16le','-ac','1','-ar','16000','new_try.wav','-y'],);
here input.wav is the raw audio coming from the Typescript server in PCMU format, and new_try.wav is the converted audio in PCM format. I wanted to test if it is readable my Sherpa_ONNX hence I used online_client_websocket_decode_files.py to test if the converted file is compatible, but that is when I get the error:
raise Error('file does not start with RIFF id')
wave.Error: file does not start with RIFF id
My end goal is to implement streaming transformation of audio bytes similar to what you had done in the speech_recognition_from_url.py file, but I am testing it on wav file first before trying to implement ffmpeg for streaming transformation of data to sent to streaming_server.

I do not have new_file.wav

wanted to test if it is readable my Sherpa_ONNX hence I used online_client_websocket_decode_files.py to test if the converted file is compatible

If you don't have new_file.wav, could you tell us what is the input file to online_client_websocket_decode_files.py?
Please post the input file.

The input to online_client_websocket_decode_files.py is new_file.wav, how can I post a wav file here, I can only post only screenshots here.
I can share a link of the file which is in my public repo: https://github.com/tempops/audiofile/blob/3549f746d784cf3317e3e36dff965193ca3f43b8/new_try (1).wav

Please see the above screenshot and make sure you provide a file that has the expected format.

Hello,
Thank you for pointing this out. I ended up using FFmpeg to convert the data and stream it to the server.


Does not respond to OS quit event

Using Linux with i3 as a window manager, Win-Shift-q closes every window but not Mattermost's; Ctrl-q works fine. Perhaps this is related to #88 or #8
MM version: 1.1.1

I don't know about i3, but $mod+Shift+q seems to be just closing a window. So I think that it's related to #8 and #90. And as I had written at #90, the way to tell 'quit' and 'close' is necessary. If 'quit' performs as 'close all windows', I might not be able to handle that.
btw, why is #88 related?

Correct; mod+shift+q closes the window. I considered the lack of system tray icon as relevant because without one, a closed window would be impossible to reopen, but the application would still be running.

Thanks @rpglover64  for the feedback,
A related ticket #178 has been resolved and closed, which should fix the issue presented here.
However, if there is a further improvement you would like to see, I invite you to help contribute your suggestion in Mattermost feature idea forum so it can be discussed, upvoted and considered for a ticket accepting pull requests?
If you do add it to the feature idea forum, please include a link back to this GitHub Issue. If you're interested in implementing, please say so and we'll prioritize the review.


⚠ Bofrost.es has degraded performance

In f4db754, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1707 ms

Resolved: Bofrost.es performance has improved in d9ca9cc after 6 minutes.


fix more bugs and misprints

Hi,
I checking sharpdx in PVS-Studio and fix bugs.
Thanks

Thanks this is merged.


Bug) FindHIP.CMake should include the compile definitions into HIP_HCC_FLAGS.

Regular usage of add_definitions doesn't work when you're using FindHIP.CMake to invoke HIPCC.
Something like this should be added in order to catch all of the compile definitions.
GET_DIRECTORY_PROPERTY(MY_DEFINITIONS COMPILE_DEFINITIONS)
IF (MY_DEFINITIONS)
  FOREACH(_item ${MY_DEFINITIONS})
    LIST(APPEND HIP_HCC_FLAGS "-D${_item}")
  ENDFOREACH()
ENDIF()

https://github.com/ROCm-Developer-Tools/HIP/pull/558


Implement Chaos over IP (using standard IP protocol 16)

Uses a Chaos "hardware trailer". Does not yet implement RUT broadcasts on IPv6.


mark_parent_class_method_used method traverses inheritance tree up indefinitely

We have such code in our legacy application:
module Public
  class ApplicationController < ApplicationController
    ...
  end
end

rails_best_practices-1.18.0/lib/rails_best_practices/core/methods.rb:57 #mark_parent_class_method_used method traverses inheritance tree up indefinitely. Ruby and Rails, however, handling situation just fine, assuming, that Public::ApplicationController inherits from ::ApplicationController
/Users/maksar/.rvm/gems/ruby-2.2.4/gems/rails_best_practices-1.18.0/lib/rails_best_practices/core/methods.rb:58:in `block in mark_parent_class_method_used': stack level too deep (SystemStackError)                                                    |
	from /Users/maksar/.rvm/gems/ruby-2.2.4/gems/rails_best_practices-1.18.0/lib/rails_best_practices/core/methods.rb:58:in `each'
	from /Users/maksar/.rvm/gems/ruby-2.2.4/gems/rails_best_practices-1.18.0/lib/rails_best_practices/core/methods.rb:58:in `find'
	from /Users/maksar/.rvm/gems/ruby-2.2.4/gems/rails_best_practices-1.18.0/lib/rails_best_practices/core/methods.rb:58:in `mark_parent_class_method_used'
	from /Users/maksar/.rvm/gems/ruby-2.2.4/gems/rails_best_practices-1.18.0/lib/rails_best_practices/core/methods.rb:60:in `mark_parent_class_method_used'
	from /Users/maksar/.rvm/gems/ruby-2.2.4/gems/rails_best_practices-1.18.0/lib/rails_best_practices/core/methods.rb:60:in `mark_parent_class_method_used'
	from /Users/maksar/.rvm/gems/ruby-2.2.4/gems/rails_best_practices-1.18.0/lib/rails_best_practices/core/methods.rb:60:in `mark_parent_class_method_used'
	from /Users/maksar/.rvm/gems/ruby-2.2.4/gems/rails_best_practices-1.18.0/lib/rails_best_practices/core/methods.rb:60:in `mark_parent_class_method_used'
	from /Users/maksar/.rvm/gems/ruby-2.2.4/gems/rails_best_practices-1.18.0/lib/rails_best_practices/core/methods.rb:60:in `mark_parent_class_method_used'
	 ... 7704 levels...
	from /Users/maksar/.rvm/gems/ruby-2.2.4/bin/rails_best_practices:22:in `load'
	from /Users/maksar/.rvm/gems/ruby-2.2.4/bin/rails_best_practices:22:in `<main>'
	from /Users/maksar/.rvm/gems/ruby-2.2.4/bin/ruby_executable_hooks:15:in `eval'
	from /Users/maksar/.rvm/gems/ruby-2.2.4/bin/ruby_executable_hooks:15:in `<main>'

@maksar I suggest you defining Public::ApplicationController as
module Public
  class ApplicationController < ::ApplicationController
    ...
  end
end

@flyerhzm That is exactly what I did to fix the error. Further more, I think construct class A < A should not be used. However, since ruby handles such situation well, probably RBP should too (at least have protection against eternal recursion).


EarlGreyExampleSwiftTests broken in Xcode 7.3

Do the EarlGreyExampleSwiftTests run correctly for you in Xcode 7.3? I have followed the readme and only the Objective C tests run correctly. Swift just has lots of errors.

@bootstraponline, thanks for bringing this to our attention. As per our investigation - the warnings mentioned in https://github.com/google/EarlGrey/issues/52 can be removed by following the suggestions - turning __FILE__ to #file, __LINE__ to #line and Selector to #selector.
For the build breakages, in GREYActions and GREYMatchers, we were able to remove them by moving the @end from after the c-function definitions at the very end of the file, to right before where we define #if !(GREY_DISABLE_SHORTHAND). Essentially, we need to move the C-functions outside the interface definition. Similarly with the typedef in GREYElementMatcherBlock. Please do test these out and tell us if you still see these issues.
We're planning to send in a Pull Request for these and add them to the next release framework for CocoaPods.

Thanks for the quick response! I will try the fixes tomorrow and let you know how it goes.

I confirm that the recommended fixes resolve the build problems. The swift tests are now passing.

Awesome. We'll send a fix for this along with some tests before closing this issue. Thanks for confirming this @bootstraponline.

Fixed in https://github.com/google/EarlGrey/pull/67.


Missing deckgl package.

Issue with deck.gl Visualizations in Observable Framework
Problem
The deck.gl visualizations in the Observable Framework do not work locally out of the box.

Steps to Reproduce

Clone the repository.
Run:npm install
npm run dev
Visit: http://<IP_ADDRESS>:3000/lib/deckgl.

Proposed Fix
Install deck.gl as a dependency:
yarn add deck.gl

Remaining Problem
Even after the fix, using deck.gl in custom apps with @observablehq/framework may still result in the following error (even after install deck.gl):
Protocol error Error: Not found: /lib/deckgl
    at enoent (file:///Users/....../node_modules/@observablehq/framework/dist/error.js:14:24)
    at LoaderResolver.loadPage (file:///Users/...../node_modules/@observablehq/framework/dist/loader.js:55:13)
    at hello (file:///Users/..../node_modules/@observablehq/framework/dist/preview.js:349:32)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async WebSocket.<anonymous> (file:///Users/..../node_modules/@observablehq/framework/dist/preview.js:372:11) {
  code: 'ENOENT'
}

Hope this fix + updating my app using @observablehq/framework will resolve that issue

Happy to fix it but lack the access rights.

Im wrong, missed this layer:
const topo = import.meta.resolve("npm:visionscarto-world-atlas/world/50m.json");
const world = fetch(topo).then((response) => response.json());
const countries = world.then((world) => topojson.feature(world, world.objects.countries));


Refactor some tests to use assertAll

As I mention in a discussion in another thread, I suggest to avoid many assertions in one test or at least do it all together to avoid not testing something useful because the previous assertion fails.
To improve that, I decide to add assertAll to those tests that needed it and split negative tests to clarify the goal of each test and simplify them.
WDYT? @tombentley @SamBarker @k-wall

I'm a big fan of "one assertion" per test, but I don't really see a lot of value in assertAll. I have always interpreted the one assertion to mean assert one behaviour, not as a single assertion statement.
When writing and structuring tests I really like the BDD pattern of Given, When, Then for example (not using the actual API):
public void clusterShouldHaveTwoBrokers() {
//Given
var clusterBuilder = new ClusterBuilder().withNumBrokers(2);

//When
var cluster = clusterBuilder.build();

//Then
assertThat(cluster.numBrokers)).is(2);
assertThat(cluster.boostrapServers).contains(",")

But that's not really the style of tests we have here. The tests here are much more stateful and I think they are trying to assert pre-conditions as well as actual behaviour.

I agree with there are some pre-coditions and I could re-do this part, but using assertions for preconditions is not the best way to do it following the testing principles. Anyway, we could go with it.
I see the benefits of assertAll in the way that if the first assertion fails, we can continue testing the rest of assertions to be sure what is really failing in the test. If we don't use it, when the first assertions fails, the test stops there and nothing else is tested.

This is the first time I've encountered assertAll (:scream:) and to be honest I don't immediately see the value of it. Sure I guess knowing that two assertions failed might be useful in some cases. But often enough (though not really in this case) I'm testing something stateful where is matters when an assertion holds in the lifecycle of an object. And I'm not a huge fan of all the extra () -> noise.
All that said, I might just be very far behind the times in the best practices of writing tests (feel free to educate me if this is the case!). So if you both think this is valuable then I'm happy for it to be merged.

I'm testing something stateful where is matters when an assertion holds in the lifecycle of an object

Hmm yeah about that...
I really don't like tests that do that, test state after state. I think they encourage tests which try to verify too much in one go. The suite as a whole should verify all the behaviour. So to that end I think if there is merit to testing the "state transitions" then each one should be handled explicitly rather than in a test which goes through the full life-cycle. That largely stems from the one "assertion" per test practice. I use "" around assertion as that comes back to the point of this PR, which I'll come back to. I'm not completely against multi step tests, as they have their place, but I have a high bar for accepting them (i.e. they come in useful when dealing with code which wasn't built using TDD).
I'm un-convinced by assertAll, I can see that it can have benefits in seeing all failures. Going back a few years the team I was on practised "one assertion per test" and we often wrote custom hamcrest matechers or assert  methods and I can see assertAll filling the same niche but I don't see the value in just dropping it in to wrap multiple unrelated assertions.

If no one is happy with this change I can close it, no problem. It was a suggestion only. Thanks for your feedback!


Add support Avro files

Add support of Avro files https://avro.apache.org/docs/1.2.0/spec.html

Added conversion of CSV files to AVRO


New Layout Available

This UI Upgrade Includes
Main Page
Trading Page
Support Page
Coin Listing Page
Wallet Page
etc
https://imgur.com/c9il6a0

Go To Link To See Live Version
https://doge-bit.com

Link not work

Link not work

Link not work

Would you like a working link as ill set one up for you now?

Link not work

Would you like a working link as ill set one up for you now?

Was a nice looking exchange. Might get back into this one just for fun.

Was a nice looking exchange. Might get back into this one just for fun.

I can send over the files if you wish bud
Get Outlook for Androidhttps://aka.ms/ghei36

From: freakcoderz<EMAIL_ADDRESS>Sent: Thursday, January 21, 2021 9:10:57 PM
To: 3s3s/opentrade<EMAIL_ADDRESS>Cc: AltCoinBuilders<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [3s3s/opentrade] New Layout Available (#355)
Was a nice looking exchange. Might get back into this one just for fun.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/3s3s/opentrade/issues/355#issuecomment-764943908, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AN25JEQ4HVDMYUVBP4JR65DS3CJ6DANCNFSM4OD5AE3A.

I can send over the files if you wish bud
Get Outlook for Androidhttps://aka.ms/ghei36

From: freakcoderz<EMAIL_ADDRESS>Sent: Thursday, January 21, 2021 9:10:57 PM
To: 3s3s/opentrade<EMAIL_ADDRESS>Cc: AltCoinBuilders<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [3s3s/opentrade] New Layout Available (#355)
Was a nice looking exchange. Might get back into this one just for fun.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/3s3s/opentrade/issues/355#issuecomment-764943908, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AN25JEQ4HVDMYUVBP4JR65DS3CJ6DANCNFSM4OD5AE3A.

Sure. I need to work out additional pairs too.

Sure. I need to work out additional pairs too.

Mine as additional pairs already just needs to add eth framework to it :)
Get Outlook for Androidhttps://aka.ms/ghei36

From: freakcoderz<EMAIL_ADDRESS>Sent: Friday, January 22, 2021 5:00:15 PM
To: 3s3s/opentrade<EMAIL_ADDRESS>Cc: AltCoinBuilders<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [3s3s/opentrade] New Layout Available (#355)
Sure. I need to work out additional pairs too.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/3s3s/opentrade/issues/355#issuecomment-765553889, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AN25JEWLD6Y5KEKB64OLCSTS3GVJ7ANCNFSM4OD5AE3A.

Mine as additional pairs already just needs to add eth framework to it :)
Get Outlook for Androidhttps://aka.ms/ghei36

From: freakcoderz<EMAIL_ADDRESS>Sent: Friday, January 22, 2021 5:00:15 PM
To: 3s3s/opentrade<EMAIL_ADDRESS>Cc: AltCoinBuilders<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [3s3s/opentrade] New Layout Available (#355)
Sure. I need to work out additional pairs too.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/3s3s/opentrade/issues/355#issuecomment-765553889, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AN25JEWLD6Y5KEKB64OLCSTS3GVJ7ANCNFSM4OD5AE3A.

Nice

Nice

would like to see it too

would like to see it too

would like to see it too

Feel free to find my email on the information section and send me an email and i can assist


Data Science vs. Machine Learning

Both are awesome but there is a large overlap between these two:

https://github.com/okulbilisim/awesome-datascience
https://github.com/josephmisiti/awesome-machine-learning

What would be the key differentiators?
Any plan to merge the two awesome repos?

Awesome-datascience repo is designed for data science application guide and structured for a data science project creation process. There are 6 different sections to build a data science project. Additionally, Data science life cycle is prepared for whole data science application process according to step by step challenges. If you look at "Apply algorithm to fix problem" part is just for Data mining and machine learning process. All in all, if you want to merge awesome-machine-learning repo with https://github.com/okulbilisim/awesome-datascience/blob/master/DataScience-Life-Cycle.md#apply-algorithm-to-fix-problem section we are open.

BTW, awesome-datascience not only a list but also a guide for whom try to learn this topic. We will set webinars and courses soon :)


Remote agent

I have a request for a remote agent,  This can be a docker one that can be loaded on any operating system.
The remote agent polls a tcp port on the uptime kuma server, e.g. 3010.  It measures the latency and then posts it to the server using JSON.  That post can include additional metrics such as CPU usage, hard disk health, wifi signal strength, etc.
When a new monitor is created on the server it generates a random unique key.  That key is used as a parameter in the remote agent and is embedded in the JSON posting the metrics.
This will allow monitoring of systems behind a firewall such as work from home users.  Even servers and desktops in a normal work environment being monitored by a cloud vps.

@amastelek Push based notifications are available in the mightly image tag, see here: #279
The metrics data sounds more of a "Health and Status", instead of an Uptime Status which is Uptime-Kuma

still since we have push now.. we can actually monitor if a host runs out of memory or has a high cpu load. I think we should gather simple commands/scripts to make this happen (without the need of an agent).

Hello,
I continue this topic to talk about the agent. The push notification is a very good feature to monitor not exposed service, but you should have access to this service and have to edit it to send the request.
For this case, I think an agent is necessary. For exemple, I will take a drone-runner service, within a Docker container, with is a service written with Go. I would like to monitor this service but it is not exposed, and I do not want edit the container (even if I could because the code is open source).
For this case, I imagine this feature. I made the POC with bash
IP=$(docker inspect drone-runner | jq '.[].NetworkSettings.Networks.bridge.IPAddress' -r)
PORT=$(docker inspect drone-runner | jq '.[].NetworkSettings.Ports' | jq 'keys []' -r | cut -d "/" -f 1)

URI="http://$IP:$PORT/healthz"

CODE=$(curl -s -o /dev/null -w "%{http_code}" $URI)

if [[ $CODE == 200 ]]; then
  curl https://UPTIME_KUMA_URL/api/push/lB95pTNgVI?msg=OK &> /dev/null
fi

I could edit this project to create the agent but I do not really know if this pattern is ok and how to implement this feature. Let's discuss about it :)
TL;DR :  How to monitor not exposed service without the ability to edit this service to add a push notification ?

@Tchoupinax
I think you should create agents in a seperate Agent-Repository where we as a community gather different agents for different usecases (add a little readme for every usecase/agent) and just run it using crontab or windows service etc..

and I must say I really see a big potential in all sizes of companys.. as they often have "monitoring for everything" but in the end it's never as good as a specific monitor for specific usecases.

Is this repository already exists ?
I do not really understand your reasoning, uptime-kuma is a monitoring for everything : it accepts HTTP, TCP, Ping... a lot of protocol and it supports a lot of service for notifications. The purpose of agent is just to extend the capability  for not exposed service. The push notification feature is the right way, but incomplete as my sense... The complete this, the project misses agents, in my sens always; I'm asking and expose my idea because I do not know if it is good.
I can create an agent like the code I published, but I could be integrate to the project as a native feature and I think it could be better for configuring and handle all your monitoring. We have just to improve a little the push notification feature, make an agent and modify the UI to specify an agent.
If I understand your idea : why uptime kuma supports so many notifcation type ? Kuma could only send a webhook and tells "if you want to notify on XX, please use our agent for this usecase". But it is simpler to select your notification type on the list and click on "valid".
I really need think agent could be a good part of this project but I would love to read your explanation if you think it is not the case.

I don't see enough improvement in bloating uptime-kuma for this.
you could just create a few installer-scripts where you put in the url and auto-create a crontab.
there are limitless amounts of agents.. so including them in the UI is quite useless as soon as the numbers build up

I don't see enough improvement in bloating uptime-kuma for this.
you could just create a few installer-scripts where you put in the url and auto-create a crontab.
there are limitless amounts of agents.. so including them in the UI is quite useless as soon as the numbers build up

I need to monitor servers that are hidden from the network where Kuma is running. However I have, what we call, a jump server that has access to these hidden servers. The jump server can see both the hidden and Kuma. In this case a passthrough agent would be nice...

Hello,
I continue this topic to talk about the agent. The push notification is a very good feature to monitor not exposed service, but you should have access to this service and have to edit it to send the request.
For this case, I think an agent is necessary. For exemple, I will take a drone-runner service, within a Docker container, with is a service written with Go. I would like to monitor this service but it is not exposed, and I do not want edit the container (even if I could because the code is open source).
For this case, I imagine this feature. I made the POC with bash
IP=$(docker inspect drone-runner | jq '.[].NetworkSettings.Networks.bridge.IPAddress' -r)
PORT=$(docker inspect drone-runner | jq '.[].NetworkSettings.Ports' | jq 'keys []' -r | cut -d "/" -f 1)

URI="http://$IP:$PORT/healthz"

CODE=$(curl -s -o /dev/null -w "%{http_code}" $URI)

if [[ $CODE == 200 ]]; then
  curl https://UPTIME_KUMA_URL/api/push/lB95pTNgVI?msg=OK &> /dev/null
fi

I could edit this project to create the agent but I do not really know if this pattern is ok and how to implement this feature. Let's discuss about it :)
TL;DR : How to monitor not exposed service without the ability to edit this service to add a push notification ?
I think this feature could create an agent that register to the master when starting. On the dashboard, we can choose a method and specify an agent if we need it to make the check (e.g. PING this url with Agent #1). For performing the action, in the case an agent has been specified, the master will contact the agent to execute the query instead of executing itself. When the agent received the answer, it forward it to the master.
What do you think ?

Great solution, don't need a agent in a local computer, with a simple script we can do it.
For example, I adjust your script to this:
UK="UPTIMEKUMA PUSH URL"
SERVER=SERVERIP
PORT=SERVERPORT
netcat -z ${SERVER} ${PORT}
echo $?
if [ $? -eq 0 ]
then
curl $UK &> /dev/null
echo $UK
fi
I will run this over local RPI computer and deploy it in local network for monitoring local servers and send to Cloud.

Noe exactly remote agents but I've created a small script on top of uptime kuma to do "replication" at database layer through restic https://github.com/Sid-Sun/replicator-kuma there are a few quirks though

still since we have push now.. we can actually monitor [...]

This part of the issue is resolved.
I think we should track the rest of the issue in #84 instead, being the more poplar issue and contentwise a duplicate
=> closing as a duplicate


Supporting Different Inputs

Is there a way to support different input types in individual cells? Like selects or checkboxes?

Looking at the code, I don't think there is an easy way to do this. The editor value is a function called at initialization time. I looked through the PRs here and didn't see anything offhand that does this. If there was a way to defer calling the editor function until you actually needed it, it might be possible to then craft a function that takes into account what you actually clicked and provide the appropriate editor at use time.
I could really use this functionality so I will play around with it and see if I can come up with something. I see this issue is almost three years old, and this repo looks kind of abandoned? So if you solved this problem with something else, I'd love to see it.

I have a version that does this in my branch: https://github.com/tungstencarbide/editable-table
wc-table-editor.js is a partial re-write with a little bit of cleanup and it should work as a drop-in replacement for the original. The README shows you how to specify a callback function that allows you to change the editor control. It Works For Me™ for my uses, but it's not been exhaustively tested.
HTH.


Sliders don't work in audio demo

You forgot to initialize your memory to zero d00d.

Personally I would use = { 0 }; to be a bit more safe / future-proof, but I fixed it this way to maintain consistency with your choices.

This should be fixed with 4a25a8b120c3712fe99daa841932505ad11c8807


fix: mutating webhook configuration test

This PR adds test mentioned in #15

Overall it looks great.  @DomFleischmann's comment is a good one to address but the test itself is perfect

@ca-scribner Let me know if there is anything else


修改nz-table的API错误

PR Checklist
Please check if your PR fulfills the following requirements:

[x] The commit message follows our guidelines: https://github.com/NG-ZORRO/ng-zorro-antd/blob/master/CONTRIBUTING.md#commit
[ ] Tests for the changes have been added (for bug fixes / features)
[x] Docs have been added / updated (for bug fixes / features)

PR Type
What kind of change does this PR introduce?

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[x] Documentation content changes
[ ] Application (the showcase website) / infrastructure changes
[ ] Other... Please describe:

What is the current behavior?

Issue Number: #174
What is the new behavior?
Does this PR introduce a breaking change?
[ ] Yes
[x] No

Other information

Coverage remained the same at 40.369% when pulling 5918b7bcd097332105da22c614a53abc74492f61 on bokire:bokire into 8bbc82a6587e9e625761f04697e73da6a8ffa68f on NG-ZORRO:master.

please read https://github.com/NG-ZORRO/ng-zorro-antd/blob/master/CONTRIBUTING.md#commit, thanks for your pr.

Hi @bokire
the format of commit other than the title
I'll close the PR for the time being.
Please comment if I'm missing something.


Update CONTRIBUTING.md to help contributors get started

Description
Currently, the CONTRIBUTING.md file is missing information on how to run the project in the development environment. The current steps are not very helpful to new contributors.
Some helpful information:

Pre-requisites to running the project
How to run the frontend
How to run different workers
How to Contribute, etc.

Hey, could I work on this one? I'm not 100% sure on how the workers work, but everything else I should be able to write on.


Route builder / tester

This would depend on completion of #51

Builder mode

Based on an idea with Discord user t1a2l

Similar to the vanilla route tool (that green line that you draw which defines what segments a road name is applied to).
It would ignore lane arrows / connectors, and vehicle restrictions, etc. It would respect 1-way roads.
You set the start point, draw the route to the end point, optionally moving a middle point to give some additional control over where the route goes.
Once happy, you click a button and TM:PE makes that route viable. For example, ensuring lanes are connected, etc.
Additional buttons could represent the vehicle restriction categories. So you could toggle the restrictions on/off along the route. Would need tri-state icons for the restrictions (eg. a mixed state to denote 'currently not all segments are the same state').
Tester mode

This is based on, and obsoletes, #96

Some sort of toggle (maybe one of those blue buttons that appear next to build bar, eg. the curved road button, that sort of thing) would switch to test mode.
In test mode, you draw from start to finish, optionally moving center point for some extra control over route, and then you can see where the route breaks.
For example if there's a 1-way road going wrong way the line in that area would be red = this is a problem.
If you select specific vehicle restriction, rather than toggling restriction like in build mode, it checks that vehicle can use those segments. Any segments restricted to that vehicle type would show red (or the line would be red, whatever).
Likewise, any issues with lane arrows/connectors, traffic lights, or other rules would turn the line red.
Result is a really easy way to see where problems are on the route.

Tagging #262


call refreshAccessToken and return 401 invalid_grants

Hi,
I have two projects deployed to gcloud, one is for development and another one is for UAT.And the two projects have same source code, just client id and client secret are different.
I store the refresh token in the database, and will check the access token is expired or not each time. If the access token is expired, than I use the refresh token stored in the database to refresh access token. But the two projects have different results.
The DEV runs great. But UAT return 400: invalid_grants if no user login the website for a long time(4-8 hours).
I do not known where there are some limits that multiple projects have the access to the same user`s google drive or refresh token would be invalid if no user use it for a long time.
Here is my code:
// @params (refreshToken) stored in database
function Auth (refreshToken) {
  this.refreshToken = refreshToken
}

Auth.auth = new googleAuth()
Auth.oauth2Client = new Auth.auth.OAuth2(client_id, client_secret, redirectUris)

Auth.prototype.Oauth2Client = function (cb) {
  if (!Auth.oauth2Client.credentials || !Auth.oauth2Client.credentials.refresh_token) {
    Auth.oauth2Client.setCredentials({
      refresh_token: this.refreshToken
    })
  }

  var expireDate = Auth.oauth2Client.credentials.expiry_date
  var currentTime = (new Date()).getTime()

  if (!expireDate || expireDate <= currentTime) {
    Auth.oauth2Client.refreshAccessToken(function (err, tokens) {
      if (err) {
        cb(err, null)
      }
      if (tokens) {
        Auth.oauth2Client.setCredentials(tokens)
        cb(null, Auth.oauth2Client)
      }
    })
  } else {
    cb(null, Auth.oauth2Client)
  }
}

Thank you.

I'm not sure what the issue could be. Are you still experiencing this issue?

This issue has been solved.

@flytofuture @jmdobry I am not able to refresh token. It gives me invalid_grant error. Please Help

Same problem....


Use Cargo for builds

Continuing the work from: https://github.com/exercism/xrust/pull/26
@pminten I also included your bash script and fixed that compilation error.
Closes #18

Looking into the failure...

Looking into the failure...

Looking into the failure...

And we are green!

Green! Woot :) Nice work!! :+1:


Remove stdarch_arm_crc32 feature as it is now stable

Fixes #565

@s3bk could you please review this? thanks a lot.

ping @mrobinson , can this be merged please.

Please update crates.io with version 0.5.4

Great ! Thanks for the swift response.


🌱 Refactor and rename common part of transport controller

Summary
This PR moves the generic transport controller code that was in cmd/ to pkg/transportcontroller/.
Eventually (when there are not other big PRs against it in flight) the other part of the generic transport controller code, which is currently in pkg/transport/generic_transport_controller.go, should also move to pkg/transportcontroller/.
Related issue(s)
Fixes #1750

/cc @nirrozenbaum
/cc @pdettori
/cc @vMaroon

@MikeSpreitzer I'm ok with the idea in the PR but would like to request a few changes in the structure:

no need to keep the deprecated files under cmd. no one is using this (other than me) and I will fix the pointer in that ocm-plugin as soon as this PR will be merged. keeping deprecated is even more confusing than what it was before.
let's just rename and move to pkg as you suggested. you don't break anything with this change cause nothing is running transport atm. we don't even break ocm plugin repo cause it's pointing to specific version of KS in go.mod (version before this PR).
the separation to pkg/transport and pkg/transportcontroller is strange. please let's put everything under pkg/transport.
once everything is under pkg/transport just change the functions in generic_transport_controller to lowercase so it's not exposed, e.g., func Run(...) -> func run(...) and func NewTransportController(...) -> func newTransportController(...). Then we can keep using function Run(...) in base_controller. calling it from outside as suggested in this PR looks like - transportcontroller.MainForTransport() - using transport twice is redundant, it's in the package name.
after this change it will be called transport.Run(..) or alternatively transport.Main(..).
I strongly prefer calling it Run, since as you stated, this is not executable under cmd and therefore it's not main and this also might be confusing.

I deleted the deprecated stuff.
I changed NewTransportController to newTransportController.
I changed MainForTransport to GenericMain. There is still a problem with Run, in that a controller typically has a Run func --- and this one is no exception (genericTransportController::Run). Really we are using the word "controller" with two meanings here, one is the whole executable and one is the specific Go language object constructed by newTransportController. I think having the word "main" in the command func helps to direct the reader's thinking.

I merged package transportcontroller into transport.

ok. there is a point in the the fact that a typical controller has Run function.
I can also live with GenericMain as the function name.
What if we put what was previously under cmd/transport into pkg/transport/cmd, would that work?
it creates a clear separation between the "cmd-alike" part and the pkg classic part. wdyt?

That package structure works for me too.

The force-pushes make the latest package structure change and squash.

it didn’t refresh so ignore the last comment.
/lgtm
Thanks!

/approve


Feature request - refactor UA parsing to separate module

I'm really keen to use this module's functionality, but I only really need the code from https://github.com/rguerreiro/express-device/blob/master/lib/device.js#L48-L106 that maps UA strings to device types.
It'd be really useful if that mapping was separated out into its own module, which this one can then depend on, so the mapping can be used standalone without need for the (unnecessary in my case) coupling to express.

As a side note, I'd be happy to do the work on this, but I figured you might want to retain ownership of the child module since you've done all the good work to get it this far.

@jimmybergman - any chance you might be open to sharing https://www.npmjs.com/package/device for this purpose?

Sure, if done without changing the (tiny) API, then this sounds exactly like what I wanted when depending on express-device in https://github.com/jimmybergman/device

@jimmybergman I've thrown together some code at https://github.com/lennym/device/tree/port-express-device-detection-code that does this
Note: I've changed the license in the package to credit @rguerreiro as the code is all his originally.

@lennym Thanks, I merged this and published as 0.2.0

That's something that already crossed my mind. Many use express-device only for the purpose of finding the device type and do not want to use all the express stuff. Maybe it's time to move forward and do just that

Happy to put together a PR that uses device (a.k.a. the code ported out of this project by me yesterday) since I know what I left out. Give me ten mins...

Presto! #37

Well...I don't have ownership of the "device" package and the current version of "device" depends from "express-device". Maybe I should create another package with another name?

The current version (as of yesterday) doesn't have the dependency anymore (see https://github.com/jimmybergman/device/blob/master/package.json)
I'd ask @jimmybergman to share ownership (which I imagine he'll be happy to do, since he was building on top of your code before, and was happy to merge code crediting you), but if you want to fork and publish a new package yourself then that's cool too. I just thought device was a good package since a) it has the best name, and b) was doing exactly what was required.

Well then...the ball is on @jimmybergman 's side :)

Absolutely no problem for me if device moves to beeing a package in this repo, any other repository handled by @rguerreiro or some other solution meaning that the code isn't in two places :-).
For now I added @rguerreiro as owner of the device package.

Yesterday I made a little experiment called WIMD (stands for what is my device?). But I agree that calling it device is the best option. But do you agree that I change, a tiny bit, the API that you made in the device package? You can check my WIMD repo

Well, I don't have a hard opinion and it is not a problem for myself, I was just thinking of not annoying users of the existing package. Although the old device package doesn't have a lot of users (it has some hundred downloads per month).
I.e. I would prefer if the existing API could be kept as well as the new one (i.e. making sure that require('device').useragent_is('useragent_string') returns like your parser.get_type(), but I have absolutely no problem with having the main API be like in the WIMD repo.

+1 for keeping the current API in place for backwards compatibility, and extending accordingly.

Great! When I have some time off I'll publish the new device package with all of your suggestions. Thanks for all the input

Done! New 0.3 version already published.

@rguerreiro @jimmybergman You guys are awesome!


Updater stops responding.

The Updater, when I went to run it today, stopped responding. It gets just past the section in the screenshot and just stops responding.

I ran the non-gui updater and it works fine, so it's not the DS card.

Did you happen to click around in other apps and windows? This happened quite often to me as soon the the updater loses focus as it seems...

Did you happen to click around in other apps and windows? This happened quite often to me as soon the the updater loses focus as it seems...

It just stops doing anything. I've left it running for 10 minutes without touching it and nothing. Task Manager shows zero activity and clicking in the app causes Windows to figure out it's not responding, so you're right about that.

Is this issue still happening with v1.6.1?


BUG :  Week/Resource visualization there is a display problem

Hello, in the Rapla(2) Week/Resource visualization there is a display problem, the resources displayed on the left do not match while it works in Rapla(1.8)? Query problem? The Day/Resource display is good, no offset. Thank you
Hallo, in der Wochen-/Ressourcen-Visualisierung von Rapla(2) gibt es ein Anzeigeproblem, die links angezeigten Ressourcen stimmen nicht überein, obwohl es in Rapla(1.8) funktioniert? Problem mit der Abfrage? Die Tages-/Ressourcenanzeige ist gut, keine Verzögerung. DANKE
Bonjour, dans la visualisation Rapla(2) Semaine/Ressource il y a un problème d'affichage, les ressources affichées à gauches ne correspondent pas alors que cela fonctionne dans Rapla(1.8)? Problème de requète ? L'affichage Jour/Ressource lui est bon, pas de décalage. Merci

Hello, i cant read the resourcenames in the screenshot correctly. Do you mean the order in which the resourcenames are listed in the Boxes is not the same as in Rapla 1.8?
The resourcename in the column should appear also on the right side. But if the display is not that wide it can be capped when there are multiple resources in the box.

In the gray column is the list of resources used. For each resource (1 per line) corresponds in the table to its right a colored rectangle for each day of use. In version 1.8 the colored rectangles are well aligned with the name of the resource in the left column. In version 2.0 everything is mixed up, the list on the left (gray column) no longer corresponds to any colored box. There is a display BUG, a simple query error in the Week/Resource view.

"Do you mean that the order in which the resource names are listed in the boxes is not the same as in Rapla 1.8?"
Yes...

Hello, i cant read the resourcenames in the screenshot correctly. Do you mean the order in which the resourcenames are listed in the Boxes is not the same as in Rapla 1.8? The resourcename in the column should appear also on the right side. But if the display is not that wide it can be capped when there are multiple resources in the box.

Do you mean that the order in which the resource names are listed in the boxes is not the same as in Rapla 1.8?
Yes...
In the gray column is the list of resources used. For each resource (1 per line) corresponds in the table to its right a colored rectangle for each day of use. In version 1.8 the colored rectangles are well aligned with the name of the resource in the left column. In version 2.0 everything is mixed up, the list on the left (gray column) no longer corresponds to any colored box. There is a display BUG, a simple query error in the Week/Resource view.


Merge refs/heads/temp-branch-6404091285 into alpha

Merge refs/heads/temp-branch-6404091285 into alpha

This PR has not been merged into alpha. Please fix the issues and re-open the PR.

This PR has not been merged into alpha. Please fix the issues and re-open the PR.


Add queue ttl

Context:

Currently gang jobs are all submitted in a single batch to Armada and are rejected as a submit check if they aren't of a sufficient size
This meant that you would either get all the jobs for a gang to yield a result or it is rejected
In order to support elasticity between a min size and max size for gangs, we need to be able to submit gang jobs across batches
With this it is possible that the gang size is not satisfied and jobs remain in the queue indefinitely
Hence, we are adding the ability to set a TTL to indicate the max time a job can remain in the queued state in Armada forever

(Failing checks for airflow-integration-tests and python-client-integration-tests are cause by permissioning: "pull access denied for armada-airflow-operator-builder, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.")


Improving data coverage

acronym | status | source |   |   |
-- | -- | -- | -- | -- | --
DHS | fallback | https://raw.githubusercontent.com/GSA/code-gov-harvester/master/data/fallback/DHS.json | https://www.dhs.gov/code/json | https://github.com/usdhs |
DOJ | fallback | https://raw.githubusercontent.com/GSA/code-gov-harvester/master/data/fallback/DOJ.json | https://www.justice.gov/digitalstrategy | https://github.com/usdoj |
EPA | fallback | https://raw.githubusercontent.com/GSA/code-gov-harvester/master/data/fallback/EPA.json | https://edg.epa.gov/data.json | https://github.com/USEPA |
NSF | fallback | https://raw.githubusercontent.com/GSA/code-gov-harvester/master/data/fallback/NSF.json | https://www.nsf.gov/digitalstrategy/ | https://github.com/nsf-open |
NARA | fallback | https://raw.githubusercontent.com/GSA/code-gov-harvester/master/data/fallback/NARA.json | https://www.archives.gov/digitalstrategy | https://github.com/usnationalarchives | https://www.archives.gov/developer
DOI | NULL | NULL | https://github.com/usinterior |   |
DOC | NULL | NULL | https://github.com/CommerceGov | https://github.com/usnistgov | https://github.com/NOAA-GFDL
DOS | NULL | NULL | https://www.state.gov/digital-government-strategy/ |
USAID | NULL | NULL | https://www.usaid.gov/usaid-digital-strategy | https://github.com/USAID |
NRC | NULL | NULL | https://www.nrc.gov/public-involve/open/digital-government.html | https://www.nrc.gov/developer.html |
OPM | NULL | NULL | https://www.opm.gov/blogs/OpenOPM/digital-government-strategy/ |
USGS | NULL | NULL | https://github.com/usgs/code-json-generator |   |
NSA | NULL | NULL | https://code.nsa.gov/ | https://github.com/nationalsecurityagency
EOP | NULL | NULL | https://raw.githubusercontent.com/EOP-OMB/code_json/master/code.json |
HUD | OK | https://www.hud.gov/sites/documents/CODE_INVENTORY.JSON |   |   |
USDA | OK | https://www.usda.gov/sites/default/files/documents/code.json |   |   |
DOL | OK | https://www.dol.gov/code.json |   |   |
DOT | OK | https://www.transportation.gov/sites/dot.gov/files/docs/code.json |   |   |
TREASURY | OK | https://s3.amazonaws.com/static.treasury.gov/jsonfiles/code.json |   |   |
VA | OK | https://www.va.gov/code.json |   |   |
NASA | OK | https://raw.githubusercontent.com/nasa/Open-Source-Catalog/master/code.json |   |   |
GSA | OK | https://open.gsa.gov/code.json |   |   |
SBA | OK | https://www.sba.gov/code.json |   |   |
SSA | OK | https://www.ssa.gov/code.json |   |   |
CFPB | OK | https://www.consumerfinance.gov/code.json |   |   |
DOD | OK | https://code.mil/code.json |   |   |
ED | OK | https://www2.ed.gov/code.json |   |   |
DOE | OK | https://www.energy.gov/sites/prod/files/2019/07/f64/code-07-17-2019_0.json |   |   |
HHS | OK | https://www.hhs.gov/code.json |   |   |
FEC | OK | https://www.fec.gov/code.json |   |   |

@IanLee1521 have y'all look into collecting the user/organizations for US federal agencies?
@CalvinIsch, could you look into finding users/organizations for US federal entities on GitHub?
@saracope What's the process to touch base with agencies for updating their code.json or reviving those?
A few special cases

EPA has both OSS projects and datasets on the same JSON file (currently it is not registered)
NSA has a list of OSS projects and a link to code.gov, but no code.json
USGS has a repository to to generate the code.json
EOP has a code.json not registered
DOJ has link to Drupal contributions

Hi @Nosferican -- I haven't specifically, but mostly because there is already https://government.github.com/community/ which has that data.
I did run github.com/llnl/scraper against the U.S. entries on that site (at the time) and posted those results on the pull request: https://github.com/LLNL/scraper/pull/3
Is that what you were thinking, or something else?

The GitHub Government Community collection is a crowd-sourced initiative, but it isn't curated per se... For example, you have some organizations that are definitely not U.S. Federal dept/agencies (e.g., @radiofreeasia is under U.S. Federal, but is an NGO). We were thinking of querying the name of U.S. department and agencies and some combinations (e.g., US $name, acronym) against the GH Torrent organization name or GraphQL API.
For obtaining a list of U.S. federal dept/agencies and other entities we were thinking of using A-Z Index of U.S. Government Departments and Agencies which has a directory based on the U.S. Government Manual supplemented with entities that "directly serve the public" (e.g., USDA National Agricultural Statistics Service NASS). Ideally M-16-21 should have the dept/agency heads report monitor it, but I suspect it isn't occurring as diligently as it could be.
That proposal seems like the best approach considering the lack of access to an exhaustive list of (at least public) U.S. federal dept/agencies and the different organizational levels.
For example,

National Science Foundation (NSF)

Social, Behavioral & Economic Sciences (SBE)

National Center for Science and Engineering Statistics (NCSES)

Science & Engineering Indicators Program

It is unclear at what levels would a GitHub organization be set up. It is unclear which levels would show up in budget / organizational databases.
Some of the heuristics we might use are to require the GitHub organization to have listed a website with a .gov/.mil domain. Not all federal government website use those domains though (e.g., goarmy.com). Also not require organizations to be verified (usually requires the including some metadata at the website which is usually only reserved at the department level).


basic R lab notes

students got a bit confused about x and y being different lengths - which is helpful from an educational opportunity... but maybe could first show adding them with the same length and make the different length addition a bonus question? or this is out -of -date cause I pulled the wrong one idk :P

oh also maybe instead of script in the lecture we should say r chunks too? like for comments - I also wonder if we should move this to the Rstudio lecture- people were confused by it early on this time

also I think this may be the only lecture left for adding a summary of the previous day https://github.com/jhudsl/intro_to_r/issues/169 . So I am going to leave a note here and close that issue


eas device:create gives never ending gibberish screen

Build/Submit details page URL
No response
Summary
eas device:create gives never ending gibberish scree

n
Managed or bare?
bare
Environment
waiting for command to complete
Error output
No response
Reproducible demo or steps to reproduce from a blank project
eas device:create gives never ending gibberish screen

finally I have to Ctrl+C end the command and hers the eas diagnostics log
EAS CLI 0.43.0 environment info:
System:
OS: macOS 11.5.2
Shell: 3.2.57 - /bin/bash
Binaries:
Node: 17.2.0 - /usr/local/bin/node
npm: 8.1.4 - /usr/local/bin/npm
Utilities:
Git: 2.30.1 - /usr/bin/git
npmPackages:
@expo/webpack-config: ^0.16.13 => 0.16.15
expo: ~43.0.2 => 43.0.4
react: 17.0.1 => 17.0.1
react-dom: 17.0.1 => 17.0.1
react-native: 0.64.3 => 0.64.3
react-native-web: 0.17.1 => 0.17.1
npmGlobalPackages:
eas-cli: 0.43.0
expo-cli: 5.0.3
Project workflow: managed

Can you show the beginning of the gibberish?

The root cause of this issue is that EAS CLI indirectly depends on a package called colors. The author of that package intentionally published new versions in bad faith (see original post) and npm has since pinned colors@latest = 1.4.0 to fix this. You can fix this issue on your own computer by uninstalling and reinstalling EAS CLI.
There are two dependencies in EAS CLI that used colors. One has removed colors and the other has pinned its use of colors to 1.4.0.
We have also verified that EAS CLI does not depend, directly or indirectly, on any other packages owned by the person who published the malicious updates.


[Bug]: BaseSynthesizer does not work with TextNodes as nodex parameter

Bug Description
BaseSynthesizer expects Node objects (passed in via nodes param) to have a "node" attribute (like NodeWithScore does).
This is, however, not the case for TextNode, Documents, etc.
https://github.com/jerryjliu/llama_index/blob/f8c95479bb9e502eb1eb4f938b942cb2551d6f08/llama_index/response_synthesizers/base.py#L131 is the link to the faulty line.
assuming that EVERY node object has a node attribute - which it does not.
Version
0.8.35
Steps to Reproduce
from llama_index.schema import Node, Document, NodeWithScore
from llama_index.response_synthesizers import ResponseMode, get_response_synthesizer
from llama_index.indices.service_context import ServiceContext
from llama_index.node_parser import SimpleNodeParser

parser = SimpleNodeParser.from_defaults()

content = ["I am a funny text", "I should be in context"]
nodes = [Node(text=t) for t in content]

response_synthesizer = get_response_synthesizer(
    response_mode=ResponseMode.ACCUMULATE,
    service_context=ServiceContext.from_defaults(llm=llm)
)

response_synthesizer.synthesize("All your base are belong to us.", nodes=nodes)

results in
AttributeError: 'TextNode' object has no attribute 'node'
Wrapping nodes a second time as "NodeWithScore" fixes this but.
from llama_index.schema import Node, Document, NodeWithScore
from llama_index.response_synthesizers import ResponseMode, get_response_synthesizer
from llama_index.indices.service_context import ServiceContext
from llama_index.node_parser import SimpleNodeParser

parser = SimpleNodeParser.from_defaults()

content = ["I am a funny text", "I should be in context"]
nodes = [Node(text=t) for t in content]
l2_nodes = [NodeWithScore(node=n, score=1.0) for n in nodes]

response_synthesizer = get_response_synthesizer(
    response_mode=ResponseMode.ACCUMULATE,
    service_context=ServiceContext.from_defaults(llm=llm)
)

response_synthesizer.synthesize("All your base are belong to us.", nodes=l2_nodes)

runs successfully but is rather stupid
Relevant Logs/Tracbacks
No response

@andreas-kopecky The proper input here is NodeWithScore actually, if you follow the typing here
https://github.com/jerryjliu/llama_index/blob/f8c95479bb9e502eb1eb4f938b942cb2551d6f08/llama_index/response_synthesizers/base.py#L116
And NodeWithScore is defined here
https://github.com/jerryjliu/llama_index/blob/f8c95479bb9e502eb1eb4f938b942cb2551d6f08/llama_index/schema.py#L423

Thanks for pointing out the typing hint - you should really update the synthesizer docs on your webpage though in that case since this uses lists of Nodes (i.e. TextNode) in every example.

@andreas-kopecky just a single example uses Node by accident. Every other example I could find uses NodeWithScore
Will update accordingly


Client port

Signed-off-by: Luke Hinds<EMAIL_ADDRESS>

Is there a Makefile that can be reused to generate the client stubs?

Is there a Makefile that can be reused to generate the client stubs?

Yes, I did plan to do that. I was going to talk with you both about how we share the openapi.yml from fulcio into here.

Is there a Makefile that can be reused to generate the client stubs?

I wanted to chat to you about this, as in how we handle the openapi.yml in this repo. I thought we could either:
Could we do this as a follow up patch? Makes sense to do this as the same time as docker compose files.


Loop on all the slides

Hello,
I'd like to suggest a new feature: ability to create a loop with all the slides so you can swipe indefinitely, it will take the elements from the beginning (or from the end, depending on the direction).
An example of something similar I'm trying to build is the carrousel at http://google.ai/ (below "Our Vision") where you can partially see the previous and the next slide of current one.

Hey, I believe that this issue is a duplicate of #181.

It can be implemented as we are speaking, but the API could be improved. I would love your feedback on that point.


Write a ASDF file and publish to ultralisp

I'd love to have this on ultralisp.

Quicklisp would be even better, methinks.
One issue is that you’d not want to immediately call (run-powerlisp), but rather leave that for the invoker.

Quicklisp it's useful indeed, the process of submitting that is a little slow.


updating details filled in jwt token

Signed-off-by: Sanjay Nathani<EMAIL_ADDRESS>
What this PR does / why we need it:
This PR does following

adds user details in the jwt token being generated

make the email and name parameters optional during update

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
Checklist

[ ] Does this PR have a corresponding GitHub issue?
[ ] Have you included relevant README for the chaoslib/experiment with details?
[ ] Have you added debug messages where necessary?
[ ] Have you added task comments where necessary?
[ ] Have you tested the changes for possible failure conditions?
[ ] Have you provided the positive & negative test logs for the litmusbook execution?
[ ] Does the litmusbook ensure idempotency of cluster state?, i.e., is cluster restored to original state?
[ ] Have you used non-shell/command modules for Kubernetes tasks?
[ ] Have you (jinja) templatized custom scripts that is run by the litmusbook, if any?
[ ] Have you (jinja) templatized Kubernetes deployment manifests used by the litmusbook, if any?
[ ] Have you reused/created util functions instead of repeating tasks in the litmusbook?
[ ] Do the artifacts follow the appropriate directory structure?
[ ] Have you isolated storage (eg: OpenEBS) specific implementations, checks?
[ ] Have you isolated platform (eg: baremetal kubeadm/openshift/aws/gcloud) specific implementations, checks?
[ ] Are the ansible facts well defined? Is the scope explicitly set for playbook & included utils?
[ ] Have you ensured minimum/careful usage of shell utilities (awk, grep, sed, cut, xargs etc.,)?
[ ] Can the limtusbook be executed both from within & outside a container (configurable paths, no hardcode)?
[ ] Can you suggest the minimal resource requirements for the litmusbook execution?
[ ] Does the litmusbook job artifact carry comments/default options/range for the ENV tunables?
[ ] Has the litmusbooks been linted?

Special notes for your reviewer:

/lgtm

/preview

/preview

/preview


Is the entire genome reconstruction feasible?

Hi @anyuzx,
Thank you for this wonderful tool.
In the example, the tutorial showed reconstructing the 3D conformation for only one chromosome using HiC contact map. I was wondering if HIPPS-DIMES could do it for the whole genome instead of one chromosome based on HiC data.
Thank you. Hope you could answer it.
Best,
Xingnan

Hi Xingnan, unfortunately, the model has not been tested for reconstructing the multiple chromosomes. If you provide the contact map that contains multiple chromosomes, in principle, the model will generate structures. However, we have not tested whether the reconstructed structures are reasonable or not.

Thank you for your reply. I will try it.


Publishing version 1.2.0 to Confluent Hub

I was wondering if it would be possible to get version 1.2.0 available on confluent hub? I was using version 1.1.2 which uses an old version of the google big query library with a NPE bug. I was able to work around this by downloading the 1.2.0 release version and building the connector locally. But it would be nice to avoid having this step on my end.

I just checked and saw that 1.2.0 and 1.3.0 are published. Thank you.


Chatbot issues with single quote/apostrophe in the question body

Having played with the codes on this page, I realized when you type a question that has a single quote/apostrophe in its body, the bot will respond that his bot code has issues.
One way to go around this is to modify this line:
string questionJSON = @"{'question': '" + question + "'}";
to this line:
string questionJSON = "{backslash"question backslash": backslash""+ question+"backslash"}";
backslash" apparantly escapes the double quote in a JSON string. Please correct me if I am wrong as I am fairly new to C#.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 3041fcf6-3b37-8d88-4d16-73ad0f841b78
Version Independent ID: 76c146ee-7c51-9486-9a76-cac90ca9b674
Content: LUIS and QnAMaker - Bot Integration - Azure Cognitive Services
Content Source: articles/cognitive-services/QnAMaker/Tutorials/integrate-qnamaker-luis.md
Service: cognitive-services
GitHub Login: @diberry
Microsoft Alias: diberry

@Charlie-Chat-Bot
Thanks for the feedback! We are currently investigating and will update you shortly.

#in-progress


cli panics

Describe the bug
tf-grid-cli panics when deploying name gateways without specifying a node
To Reproduce
Steps to reproduce the behavior:

run tf-grid-cli deploy gateway name -n anyname --backends http://example.com

Expected behavior
It should either succeed or fail but not panic.

still facing the same issue

works fine now

verified


logs allow you to click on deleted file; alert is unhelpful

Steps

Visit a project with a file (or upload one)
Delete that file
Look at the logs and scroll to any log that is about that file except the log saying the file was deleted

Outcome

The log that says the file was deleted does not link to the file
All other logs do link to the a file page
The file page is titled with a series of numbers
There is an alert that says the file does not exist
The revisions say the file was not found

Expected outcome

The file should not be linked to in the logs. If the file must be linked to, the detail page should still include the file's title and the alert should read, "This file was deleted on [Date]." The revisions should also include that message.

Based on how log works, this is expected behavior. Before the log is deleted, any logs about that file include the url link to that file which is already stored in the database. When the file is deleted, it just generated a new log but doesn't modify any previous log.

An improved message of "this file was deleted on [date]" is useful, and the file should continue to be linked in the logs.

I like that approach - @nanchen47 How does that sound? Difficult to change?

@JeffSpies, thoughts?
what do we do about the file title on the file's page?

@lbanner easy to do. Can be an intern task.


Is there a way to run Drats against a cf deployment with windows cell?

Hi:
When we run drats test against a cf-deployment setup with windows cell enabled, we see errors like
failed to find instances for deployment cf: couldn't find jobs: finding scripts failed on windows2016-cell/0bca09ab-c6ae-4f7e-b9e2-156b41a7f0a9: 'sudo' is not recognized as an internal or external command,

Is there a way to exclude windows cell? what is the right approach running it?
With many thanks.

Related issue: https://github.com/cloudfoundry-incubator/bosh-backup-and-restore/issues/15

Hi @Kaixiang
Currently, cf-deployment with windows cells is not supported by bbr. And there is no way to skip an instance in the deployment when running bbr deployment backup/restore.
Best wishes
Josh and @alamages

Hello @Kaixiang
We have a new 1.2.4 BBR which works with windows VMs (it properly skips them without failing).

Good to know, Thanks!


Lightning Talk on Strawberry

i watched the excellent talk on GraphQL and strawberry at EuroPython 1019.  This library makes a lot of sense to me.  But sadly people want 5 minute lightning talks.  Has anyone created one.  I want to add it to PythonLinks.info.

@PythonLinks I did one here: https://youtu.be/TAhBIzOjo2c?t=1047 hope that's good enough :)

I'll close this for now, let me know if there's any other question!


Styling Loss When Rendering the parsed pdf file to my view

I have a pdf file with with bold heading, tables, etc.
When i'm rendering the parsed file through pdf-reader into my html, all the texts are having same font-weight. Also, loss of table structures happening. Can you please help me out...what am i doing wrong??
My code is -->
In my Jobs controller, i have a method
def viewjobapplication
io = open('http://s3-ap-southeast-1.amazonaws.com/static.referhire.com/rhdoc/res/prfl/8e3c37921f1e451cc84fbd76257d95985b154d21/Guruprasad_G_SAP_BO_-_Copy.pdf')
@reader = PDF::Reader.new(io)
end
and in my viewjobapplication.html.erb file, i'm accessing it like this way -->
<% @reader.pages.each do |p| %>

<%= p %>

<%end%>
i've attached the images also for original file and the rendered view in browser...please have a look

Thanks for filing this issue.
pdf-reader is not intended to be a complete PDF to HTML conversion tool, so it's not surprsing that the PDF tables and formatting are lost. You may need to look for another tool that meets your needs.


CMS-353: Data migration for nature and culture

Jira Ticket:
CMS-353
Description:

This script migrates the nature and culture field into 4 new fields
Only parks with a parsable nature and culture section will be migrated
To run the migration script, you'll need to delete a strapi migration and restart strapi
2024.08.22T00.00.20.nature-and-culture.js
The line of code that removes the old data from the natureAndCulture field has been commented out in the migration so QA can compare the data

Hi @molund , reminder to remove the comment on the line of code that removes the old data from the natureAndCulture field for PROD deployment. I think it wasn't removed for TEST environment as well, but that's okay.