GPU VideoReader not working

🐛 Describe the bug
from torchvision.io import VideoReader
import urllib.request
import matplotlib.pyplot as plt
%matplotlib inline

urllib.request.urlretrieve("https://downloads.videezy.com/system/protected/files/000/004/210/4.mp4", "/tmp/cat.mp4")

used_timestamps = sorted(np.random.choice(np.arange(0, 8, 0.1), 10, replace=False).tolist())

video_reader_cpu = VideoReader("/tmp/cat.mp4", device="cpu", num_threads=4)

images_cpu = []
for seek in used_timestamps:
    video_reader_cpu.seek(seek)
    frame = next(video_reader_cpu)
    images_cpu.append(frame["data"].permute(1,2,0))

video_reader_gpu = VideoReader("/tmp/cat.mp4", device="cuda")

images_gpu = []
for seek in used_timestamps:
    video_reader_gpu.seek(seek)
    frame = next(video_reader_gpu)
    images_gpu.append(frame["data"].cpu())

for i1, i2 in zip(images_cpu, images_gpu):
    plt.figure()
    plt.subplot(121).imshow(i1)
    plt.subplot(122).imshow(i2)

When seeking to specific timestamps in the video and trying to extract the closest image frames, the cpu implementation of VideoReader works exactly as expected. However, the gpu implementation outputs progressively more corrupted versions of a single frame, with halo effects of other frames getting more prevalent in the latter frames.
Versions
Collecting environment information...
PyTorch version: 1.11.0
Is debug build: False
CUDA used to build PyTorch: 11.3
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.6 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.27

Python version: 3.8.12 (default, Oct 12 2021, 13:49:34)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-4.15.0-163-generic-x86_64-with-glibc2.17
Is CUDA available: True
CUDA runtime version: 11.3.109
GPU models and configuration:
GPU 0: NVIDIA RTX A5000
GPU 1: NVIDIA RTX A5000
GPU 2: NVIDIA RTX A5000
GPU 3: NVIDIA RTX A5000
GPU 4: NVIDIA RTX A5000
GPU 5: NVIDIA RTX A5000
GPU 6: NVIDIA RTX A5000
GPU 7: NVIDIA RTX A5000

Nvidia driver version: 495.29.05
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.2.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.2.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.2.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.2.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.2.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.2.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.2.0
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] mypy==0.931
[pip3] mypy-extensions==0.4.3
[pip3] numpy==1.21.2
[pip3] pytorch-lightning==1.6.0rc1
[pip3] torch==1.11.0
[pip3] torchelastic==0.2.2
[pip3] torchmetrics==0.7.3
[pip3] torchtext==0.12.0
[pip3] torchvision==0.13.0a0+1db8795
[conda] blas                      1.0                         mkl
[conda] cudatoolkit               11.3.1               ha36c431_9    nvidia
[conda] ffmpeg                    4.3                  hf484d3e_0    pytorch
[conda] mkl                       2021.4.0           h06a4308_640
[conda] mkl-service               2.4.0            py38h7f8727e_0
[conda] mkl_fft                   1.3.1            py38hd3c417c_0
[conda] mkl_random                1.2.2            py38h51133e4_0
[conda] mypy                      0.931                    pypi_0    pypi
[conda] mypy-extensions           0.4.3                    pypi_0    pypi
[conda] numpy                     1.21.2           py38h20f2e39_0
[conda] numpy-base                1.21.2           py38h79a1101_0
[conda] pytorch                   1.11.0          py3.8_cuda11.3_cudnn8.2.0_0    pytorch
[conda] pytorch-lightning         1.6.0rc1                 pypi_0    pypi
[conda] pytorch-mutex             1.0                        cuda    pytorch
[conda] torchelastic              0.2.2                    pypi_0    pypi
[conda] torchmetrics              0.7.3                    pypi_0    pypi
[conda] torchtext                 0.12.0                     py38    pytorch
[conda] torchvision               0.13.0a0+1db8795          pypi_0    pypi

This seems to be a very serious bug, as many video learning code relies on the GPU decoding. Could someone please help?
@mostafarohani I'm also curious, how do you manage to compile torchvision with GPU video decoding support? I installed ffmpeg with GPU support and also tried conda install -c conda-forge ffmpeg, but it keeps saying ffmpeg is missing bfs.h and skips installing the GPU decoder.

@mostafarohani @LinxiFan we're currently assigning all GPU decoding issues as low-pri as there was a shift in priorities and we're a bit short on people. I'll definitely look at this but probably not before the end of the month. Sorry about that :(
@prabhat00155 do you perhaps know what this could be about? If so, maybe I can take a look at it sooner if I have a decent starting point?

I'm also curious, how do you manage to compile torchvision with GPU video decoding support?

@LinxiFan the instructions are here; we're utilising NVC rather than on FFMPEG for hardware-accelerated decoding.

Is our CI actually running the tests for the GPU decoder? Looking at recent CI runs it seems like the tests are always skipped.

@NicolasHug nope: see #5147

The current implementation of GPU decoder enables you to seek once in the video and read frames from there. If you seek multiple times, the subsequent outputs may be frames closely related to the first seek. The reason being that multiple frames could have been decoded and queued up for return, after seek and demuxing.
The numerical values between CPU and GPU decoders are slightly different due to different colour spaces being used.
Could you compare the results you get after one seek operation only between GPU decoder and pyav and report any differences? Example usage

@prabhat00155 in this case, I would propose to drop all enqueued frames after a seek. Seeking multiple times on the same file is a very common operation, and in its current state the decoder is indeed not really working as expected

Thanks for the responses everyone, and I agree with @LinxiFan that this is a very serious bug.
In my use case, I can't afford to just fetch all the frames (there are thousands); only a subset of 15-30 are annotated, and the annotations are not spaced evenly apart; thus I need to do the seeking like described above. On the other hand, the CPU version is so slow that it blocks my model training. Our current hack is to export all the frames as jpg as preprocessing and then do model training. The con of course is the dataset size ballooned from 100GB to 5TB, which has caused a lot of issues, like not being able to store all the data on the server disk.
A prompt fix on this would be much appreciated <3

@LinxiFan , to install, what i did was in my dockerfile
FROM pytorch/pytorch:1.11.0-cuda11.3-cudnn8-devel

ENV DEBIAN_FRONTEND=noninteractive
ENV NVIDIA_DRIVER_CAPABILITIES compute,utility,video

COPY Video_Codec_SDK_11.1.5 /tmp/Video_Codec_SDK_11.1.5
RUN git clone https://github.com/pytorch/vision.git ~/torchvision
RUN cd /root/torchvision && \
    TORCHVISION_INCLUDE=/tmp/Video_Codec_SDK_11.1.5/Interface \
    TORCHVISION_LIBRARY=/tmp/Video_Codec_SDK_11.1.5/Lib/linux/stubs/x86_64 \
    TORCH_CUDA_ARCH_LIST=8.6+PTX \
    FORCE_CUDA=1 /opt/conda/bin/python setup.py install
RUN pip uninstall -y torchvision

i also add -e NVIDIA_DRIVER_CAPABILITIES=compute,utility,video \ in my docker run script


Don't query pasteboard contents until paste:

Currently canPerformAction: causes a read of the pasteboard, which is annoying on iOS 14. The system alerts the user "Pasted from [x]" every time the menu is queried.
Instead we can check the types without querying the contents.
Signed-off-by: Adlai Holler<EMAIL_ADDRESS>

@manuroe Updated with signoff & changelog.


Can this module support a standard http-digest exchange which uses WWW-Authenticate?

The README says:

You should define a custom header in the server side in order to avoid the browser form and use your custom login form.

This seems to imply that is the WWW-Authenticate header is present, the usual browser prompt will appear?  However, if the server does not use WWW-Authenticate then arguably it isn't using standard HTTP-Digest authentication, but a non-standard variation of it.
Also, I'm not clear if this is required just for the sign-in URL, or all URLs (I would assume the latter, but that makes the whole API exposed by the server non-standard, not just one URL, which would allow the rest of the API to stick to the standards and thereby interoperate with other clients besides the AngularJS UI using this module).
Can you clarify: can this module be used with standard http-digest with WWW-Authenticate, or does some or all occurrences of the WWW-Authenticate header need to be prevented for it to work?
Thanks!

Unfortunately, avoiding the default handling of the HTTP authentication seems not possible. So, when the browser parses the response with a status code 401 and a standard header, it always displays the standard form. There are two possibilities:

Return a generic 400 error instead of 401.
Return a generic (custom) header for the authentication.

Either options produce a non-standard behavior in both client and server side. I personally suggest to handle the authentication with OAuth, but if you can work in the server code and change the header you can use this small library to mask the HTTP authentication and improve the user experience with a "standard" login form.
Anyway, if you know a better solution to avoid the displayng of the standard browser form, your help is very appreciated.

Thanks, no there don't seem to be any very nice solutions to this. OAuth2 is arguably not very nice either! Anyway, I've submitted a documentation patch which attempts to clarify this point: wu-lee/documentation


Refresh patch to fix build issue

Fix following:
Could not apply patch. Please fix repos and patches. (patch path: /work/patches/poky/0001-fit-image-multiconfig.scarthgap.patch, repo: poky, patch entry: fit-image-multiconfig, vcs output: )

This is weird. It fails same on my setup locally after fix it works. Now it's failing on CI :)


html tags, proper nouns

First, I need to congratulate the team for your work, especially Nils is imho one of the best devs in the NLP community. Sentence-transformers and this NMT translator repo have been very helpful to us in Contents.com.
I use the Opus-mt right now it is great and I noticed that it even keeps hmtl tags. But sometimes it makes a mistake of generating ">/strong>", ×/strong> or --/strong> instead of  (strong as example, same for other tags like h3 or li (as far as I know)). I solved it by simply replacing: text = text.replace('×/', '</').replace('>/', '</').replace('--/', '</'). It is a very simple thing, Just wanted to make you know.
I wonder if you are aware of a translator model that is very good at keeping the proper nouns like people, cities, company nouns ect unchanged, even when made by 2 or more words? I could use NER but it could decrease speed and tbh I don't find the free NER libraries enough reliable.
Thank you!

Hi @tpilkati
Thanks for the compliment. Happy to hear that you find the projects useful :)
Yes, the models were not trained with HTML tags. So it is no big surprise, that they fail here to generate valid HTML tags. I think it is best to remove HTML tags for translation and then later to add it again if needed.
I am not aware of such a model. The models learn based on the training data, and if proper nouns are translated there (e.g. France and Frankreich), then the model will learn it from there and translate these too. However, it has challenges when there are new proper nouns, there, the translation can sometimes be a bit odd.
Best
Nils

@nreimers: is there by chance a token or some kind of marker that can be use to mark untranslatable chunks?
I.e. something like:
<IGNORE>Dinsdale Piranha</IGNORE>And now for something completely different!


Add configuration option to suppress project name.

All of our project notifications are sent into separate Slack channels
making the addition of the project name redundant in the notification.
This patch adds a configuration option to suppress the project name from
the notification.
fixes #44

This change is

Looks like this made it in at some point when it moved to sentry-plugins.


Send keys option is not working.

Hii Adams,

We used your image(python2) to run the selenium test cases.
image
In single file, we wrote two test cases
1: One with send keys,
2: Another one without sending keys.
The test case without using send keys is passed and another one with sending keys is failed.
Test_case file link
This is the final output

test_google_with_send_keys (testing.ExampleTests) ... ERROR
test_google_without_send_keys (testing.ExampleTests) ... ok

======================================================================
ERROR: test_google_with_send_keys (testing.ExampleTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/tests/testing.py", line 19, in test_google_with_send_keys
    input_box.send_keys('apple', Keys.RETURN)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webelement.py", line 349, in send_keys
    'value': keys_to_typing(value)})
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webelement.py", line 493, in _execute
    return self._parent.execute(command, params)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 256, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 194, in check_response
    raise exception_class(message, screen, stacktrace)
WebDriverException: Message: unknown error: an X display is required for keycode conversions, consider using Xvfb
  (Session info: chrome=57.0.2987.98)
  (Driver info: chromedriver=2.28.455506 (18f6627e265f442aeec9b6661a49fe819aeeea1f),platform=Linux 3.13.0-38-generic x86_64)

----------------------------------------------------------------------
Ran 2 tests in 15.282s

FAILED (errors=1)

Can you please suggest how can we resolve this issue.Thanks!

I got it working  by replacing the line
chromium --no-sandbox $@ &
with
chromium --no-sandbox --headless $@ &
in the file
https://github.com/mark-adams/docker-chromium-xvfb/blob/master/images/base/xvfb-chromium
Hope that helps!


UI layout clean up and bug fix

fix #67
This PR resolves all the issues listed in issue 67

move dex, trade pages to trade tab
fix excess padding between top navigation and page content layout
fix hidden side nav buttons
fix order page content alignment bug
add total value info modal etc

Apart from the few UI designs issue I have raised, the code looks good to me!


Fix: Retrysend broken when passing socket to agent

Fixes issue #202

Thank you for finding the issue :) Could you check if adding this._closing = false to the _doClose() function solves the issue? To me it looks like it's the "deinitialization" problem, which leaves an invalid state, while the code in the init is supposed to run only once. IMO the fix should reflect that.
Also did you find a way to consistently reproduce the bug? It would be nice to have a test to catch this in the future :)

Thank you for finding the issue :) Could you check if adding this._closing = false to the _doClose() function solves the issue? To me it looks like it's the "deinitialization" problem, which leaves an invalid state, while the code in the init is supposed to run only once. IMO the fix should reflect that.

The init is with the code provided meant to be run every time a new request is created, at least it is called each time a new request is created, so fixing the bug there (and there it actually is located) makes the most sense for me. The cleanup in this stack was written the way it is, so it's definitely not a bug of the deinitialization, since this behaves as intended by the author. The init does not behave as intended (when using externally provided sockets). One could discuss whether the way, this stack handles init and deinit could be done in another way, but that's not the point here. This "single-line-bugfix" solves the issue.

Also did you find a way to consistently reproduce the bug? It would be nice to have a test to catch this in the future :)
Yeah, provide an external socket , create a request to a server that exists, let it finish and then create a request to a sever that doesn't exist, the second request's promise will leave you wait forever. But I don't know whether this can be automated.

Are there plans to merge this PR? I am eagerly waiting for the npm package being updated with this bugfix, so that I can update my project to use the new version of the node-coap package.

I would like to have a test case for this issue. So far all my attempts to reproduce it worked as expected.
For example:
const coap = require('coap');
const dgram = require('dgram');

let coapTiming = {
  ackTimeout:0.25,
  ackRandomFactor: 1.0,
  maxRetransmit: 3,
  maxLatency: 2,
  piggybackReplyMs: 10
};
coap.updateTiming(coapTiming);

const server = coap.createServer();
server.on('request', (req, res) => {
  res.end();
});
server.listen(7892);

const sock = dgram.createSocket('udp4');
sock.bind();

const agent = new coap.Agent({ socket: sock });

const req = coap.request({ agent: agent, port: 7892 });
req.end();
req.on('response', (res) => {
  console.log('Req1 response');
  setImmediate(() => {
    const req2 = coap.request({ agent: agent, port: 7893 });
    req2.end();
  });
});

does emit timeout.
Looking into the agent, I can see that the _closing flag should not be set (to true) when an external socket is used:
  if (!this._opts.socket)
    this._closing = true

How is _closing flag set in your case?

Without having a look at the code in detail, your idea seems to be valid, too.
I fixed it in
Agent.prototype._init = function initSock(socket)
since, the original code (I guess at the time, when no external sockets where used) set
this._closing to false again there.
Your solution is valid if this._closing is only used for the purpose of closing the socket. But I have the code not in mind right now, so I can't say whether setting this._closing to like you suggest has any interference.

Yeah, provide an external socket , create a request to a server that exists, let it finish and then create a request to a sever that doesn't exist, the second request's promise will leave you wait forever. But I don't know whether this can be automated.

Could we focus on reproducing this in a test case? The problem is still confusing me.

Any updates here?
Testcase can be set up using a CoAP server that is killed after the first sucessful request, so a subsequent request should cause a timeout which it doesnt in the old code.

Testcase can be set up using a CoAP server that is killed after the first sucessful request, so a subsequent request should cause a timeout which it doesnt in the old code.

server.js:
const coap = require('coap');

let counter = 0;

const server = coap.createServer();
server.on('request', (req, res) => {
  res.end('' + counter++);
});
server.listen(7892);

client.js:
const coap = require('coap');
const dgram = require('dgram');

let coapTiming = {
  ackTimeout:0.25,
  ackRandomFactor: 1.0,
  maxRetransmit: 3,
  maxLatency: 2,
  piggybackReplyMs: 10
};
coap.updateTiming(coapTiming);

const sock = dgram.createSocket('udp4');
sock.bind();

const agent = new coap.Agent({ socket: sock });

function doRequest(res) {
  if (res) {
    console.log('Got response', res.payload.toString());
  }

  const req = coap.request({ agent: agent, port: 7892 });
  req.end();
  req.on('response', doRequest);
  req.on('timeout', () => { console.log('Timeout') });
  req.on('error', () => { sock.close() });
}

doRequest();

When I run only client.js I get a timeout.

This and all my previous attempts to reproduce the problem based on your description seems to have failed. Could you provide an example code that causes the issue or even better a complete test case for travis?

I realize the "bug" will occur when there is no explicit calling of updateTiming(). I think there should be some sensible defaults?
// server.js
const coap = require('coap');

let counter = 0;

const server = coap.createServer();
server.on('request', (req, res) => {
	res.end('' + counter++);
});
server.listen(7892);

// client.js
var coap = require('coap');

(function loop() {
	console.log('loop');
	var req = coap.request('coap://localhost:7892');
	req.on('response', () => setTimeout(loop, 1000));
	req.on('timeout', () => console.log('Timeout'));
	req.on('error', () => console.log('Error'));
	req.end();
})();

Steps:

start server
start client
kill server

@Apollon77 Can you reproduce this problem? I can't reproduce it locally..

No I can‘t but the code change makes sense logically and when reading the issue and the Pr the fix seems to be proofed by users. Additionally it does not really break things so risk is ok.
I learned that I can not repro anything and so I review things a bit different in these cases and more do a logical due dilligence. ;-)
Yes ideally each issue also adds a test that breaks without the test and then fixing it ... but sometimes this is just not possible.

@mcollina What do you thingk on how to handle such edgy cases like this?

Unfortunately I do not have time to dig in.

I meaned more the general "approach" question ... ok to accept it if there was a proof by usersand code "makes sense" also if we can not really repoduce it? I would accept it ion such situations

@Apollon77 OK，I agree with it. And if someone approve this change would result in another bug, we revert it.


Financials positive

Requires dill: pip install dill
Test logic is in the docstring for the financials property.
The test is run with the negative test in test_financials.py, or it can be run separately: python -m yfinance.ticker -v

Whoops, accidentally made this PR for the wrong repo. Please ignore!


⚠ Jenkins Plugins Artifactory has degraded performance

In 6d2b249, Jenkins Plugins Artifactory (https://get.jenkins.io/plugins/artifactory/) experienced degraded performance:

HTTP code: 200
Response time: 1169 ms

Resolved: Jenkins Plugins Artifactory performance has improved in 3b387e0 after 19 minutes.


Add real text blurbs/subtitles to map modals

Right now (in the q4_revamp branch currently), we are using placeholder text for the map modal subtitles. Update this with real text.

Instead of adding these, we discussed and we're going to remove the placeholders -- not enough room and the main content will go instead into a "user guide" section.


Unable to use globals in functions in other files?

See https://github.com/FredTheDino/PuckNet/commit/5d6523a15ca323cc90cb6024a31c340dc69f58b3 for workaround.

Failed to reproduce - closing tomarrow if nothing new is added.

This commit (applied on top of the commit linked above) causes the following panic:
thread 'main' panicked at 'index out of bounds: the len is 2 but the index is 2', sylt/sylt-typechecker/src/typechecker.rs:371:26


Fix Python misspelling

Hi!
I was reading through some of these specs and caught this small misspelling present in 4 files.

The build failures look unrelated; unfortunately I do not have permission to retry the build.


Resolve typo in GetModifierName

The modifier record checked the volatile attribute twice in succession.

Thanks Jeremy!

Yes I did :).


feature: unbuffered token stream

Now this should be quite easy at least for the llama.cpp backend: https://github.com/go-skynet/go-llama.cpp/pull/28 thanks to @noxer's contribution ( :heart: ) now it's just a matter of wiring things up in the SSE callback here in the server

[x] go-llama.cpp
[ ] gpt4all.cpp
[ ] gpt2.cpp
[x] rwkv.cpp

the go-gpt4all-j backend has support as well for unbuffered token stream, so we should (almost) all good as 2 out of 3 backends supports it till now

implemented for llama.cpp backend!

This is the most exciting feature for my use case! I'm wondering, do you have already planned how the API will support this? Thanks!

This is the most exciting feature for my use case! I'm wondering, do you have already planned how the API will support this? Thanks!

LocalAI is following the OpenAI specs. Therefore tokens are pushed via SSE (server-sent events) streams. It currently works already for llama.cpp models, like: vicuna, alpaca, wizardlm, and alikes.

I am getting memory issues with LocalAI on 16 GiB of RAM even in the smaller chat model.
This only happens on "stream: true". I guess this is related to buffering being enabled in the request?
If I make a single POST request without "stream: true", the request works properly and I receive a proper answer.
I am using GPT4All models.


fixes link to gh repo (master -> main)

Inside this doc on github link in every sample

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: tesar-tech sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

Thanks, @tesar-tech!


Missing Files when building configuration editor executable

When building a configuration editor some VIs appear to be missing.
This might be the editor has some miss linking.

Files are acutally in the folder in the machine at:
C:\Program Files (x86)\National Instruments\LabVIEW 2017\vi.lib\NI\DCAF\Tag Editor Core\about menu\acfg Files
But the project is expecting them at a different location.

Those files should be moved out of the library, and move to the always include, In case of custom editors that should be replace with the about information of the custom editor.


Doesn't seem to work with WorldEnvironment

I added a WorldEnvironment to my scene and hooked the sky_updated signal up with this function:
func _on_Sky_texture_sky_updated():
	$Sky_texture.copy_to_environment($WorldEnvironment)

I get the following error:
Invalid get index 'background_sky' (on base: 'WorldEnvironment').

This happens in sky_texture.gd line 65.

Just found out I need to pass $WorldEnvironment.environment. Could have thought of that :grin: Could you add a line to the readme explaining how to call copy_to_environment using a WorldEnvironment?
func _on_Sky_texture_sky_updated():
	$Sky_texture.copy_to_environment($WorldEnvironment.environment)

Oops, yeah world environment is just a container node for the actual environment resource. I'll make a note of it in the asset thanks :)


Tempfiles generated for config, params, pre-run don't get read in properly

When running launch or pipelines add in a Python script and passing in configs, params, pre-run scripts as tempfiles to the tw commands, the command itself will be invoked correctly. For example:
def launch_pipeline(fetchngs: FetchngsLaunch) -> None:
    # Create tmp config file
    config = f"""plugins {{
        id<EMAIL_ADDRESS>
    }}
    """

    # Create a temporary pre-run script
    prerun = f"export NXF_VER={fetchngs.nxf_ver}"

    # Create a temporary parameter file
    params_dict = {
        "input": "https://raw.githubusercontent.com/nf-core/fetchngs/1.11.0/tests/sra_ids_test.csv",
        "outdir": fetchngs.outdir(),
    }

    with tempfile.NamedTemporaryFile(
        mode="w", suffix=".config", delete=False
    ) as config_file, tempfile.NamedTemporaryFile(
        mode="w", suffix=".json", delete=False
    ) as params_file, tempfile.NamedTemporaryFile(
        mode="w", suffix=".sh", delete=False
    ) as prerun_file:
        config_file.write(config)
        json.dump(params_dict, params_file)
        prerun_file.write(prerun)

        # Launch the pipeline
        sp.launch(
            "nf-core-azure-fetchngs-test",
            "--name",
            fetchngs.name(),
            "--workspace",
            workspace,
            "--revision",
            fetchngs.fetchngs_version,
            "--params-file",
            params_file.name,
            "--profile",
            "test",
            "--config",
            config_file.name,
            "--pre-run",
            prerun_file.name,
        )

Will return:
DEBUG:root: Running command: tw pipelines add https://github.com/nf-core/fetchngs --name fetchngs-withSlash-1110-nf-21041-nf-val-113 --workspace scidev/testing --revision 1.11.0 --params-file /var/folders/f5/l1wj3_814_9g_s_37b0_950c0000gn/T/tmptn5mx8rn.json --profile test --config /var/folders/f5/l1wj3_814_9g_s_37b0_950c0000gn/T/tmpg5foy1wy.config --pre-run /var/folders/f5/l1wj3_814_9g_s_37b0_950c0000gn/T/tmp87

However, the files themselves will not be generated correctly on the Platform. If the tw command generated and returned on the console is ran itself manually, the files are created correctly:
$ tw pipelines add https://github.com/nf-core/fetchngs --name fetchngs-withSlash-1110-nf-21041-nf-val-113 --workspace scidev/testing --revision 1.11.0 --params-file /var/folders/f5/l1wj3_814_9g_s_37b0_950c0000gn/T/tmptn5mx8rn.json --profile test --config /var/folders/f5/l1wj3_814_9g_s_37b0_950c0000gn/T/tmpg5foy1wy.config --pre-run /var/folders/f5/l1wj3_814_9g_s_37b0_950c0000gn/T/tmp878m0gau.sh

On the platform:

Can't seem to reproduce this in a separate, similar Python script carrying out the same functionality. Going to close for now.


When transfering repository and database transaction failed, rollback the renames

Fix #14821

This probably also needs a panic handler too.
My suspicion is that putting the return in a defer would be better:
e.g.
func TransferOwnership(doer *User, newOwnerName string, repo *Repository) (err error) {
   repoRenamed := false
   wikiRenamed := false

   defer func() {
       recErr := recover()

       if recErr != nil || err != nil {
          // handle moving stuff back
          // If there is an error moving stuff back this needs a clear log.Critical(...) so administrators know that they need to do something
      } ()
      ...
   }

Then you can just err = sess.Commit() and return err
Something like that.

This probably also needs a panic handler too.
My suspicion is that putting the return in a defer would be better:
e.g.
func TransferOwnership(doer *User, newOwnerName string, repo *Repository) (err error) {
   repoRenamed := false
   wikiRenamed := false

   defer func() {
       recErr := recover()

       if recErr != nil || err != nil {
          // handle moving stuff back
          // If there is an error moving stuff back this needs a clear log.Critical(...) so administrators know that they need to do something
      } ()
      ...
   }

Then you can just err = sess.Commit() and return err
Something like that.

It's running beyond on http router, and I think we have already recovery middleware there.

yes but if there is a panic after the initial rename has occurred but then you're not going to rename back unless you use a recover deferral scheme as I suggest above.

See https://github.com/lunny/gitea/pull/14


Restyled login and registration page

Closes #214
I have changed the styling of login and registration pages by adding background image, border-radius and hover effect on links.I have also shifted the login button to more right. Kindly review my pr .

[x] My code adheres to the established style guidelines of this project.
[x] I have conducted a self-review of my code.
[x] I have included comments in areas that may be difficult to understand.
[x] I have made corresponding updates to the project documentation.
[x] My changes have not introduced any new warnings.

Screenshots

Original
Updated

![Original screenshot]

![Updated screenshot]

@apoorvaron review

@apoorvaron kindly review and merge the pr

@apoorvaron ?

@shruuti321 my exams are going on. Will review your PR on/after 19th july

@shruuti321 Collaborator's need some changes, make it lighter and do something to make it better.

Light as in? And could you suggest what changes to better this?

@shruuti321 light blue shade*

Something like this?

@apoorvaron  These are the final changes I have made in the login page. I have highlighted the texts and made them bold, increased the font size .Kindly review and approve so that I can replicate these changes for the registration page as well

@apoorvaron please review

@apoorvaron  may i know what went wrong with the styling?

??

@apoorvaron may i know what went wrong with the styling?

@shruuti321 Other collabs and Friends denied and idk what changes should i tell you but this is not going good


Issue With Form Dont Finish sending email and dont display sucess massage

Issue With Form Dont Finish sending email and dont display sucess massage.

Looks to be an issue with setup. Please provide more information, specifically what the response looks like from the POST to sendmail.

When I Submit Its Just Like Above Picture Sending.. But No Im getting

But How ever thanks For Reply.....:)

I get the same issue as prasathrp and get this in the chrome console:
jquery.min.js:6 POST http://www.example.co.uk/library/sendmail.php 500 (Internal Server Error)

contact-form.js:77 Uncaught TypeError: Cannot read property 'message' of undefined

Who's your hosting provider? Might be the same issue as https://github.com/jonmbake/bootstrap3-contact-form/issues/24. The message, Cannot read property 'message' of undefined usually means valid JSON was not returned in the response. I can update the code to catch the exception and provide a more meaningful error message... I just have to know where the error is propogating from. @Backbone666 What does the reponse body from the the http://www.example.co.uk/library/sendmail.php request look like? Would it be possible to get a stacktrace from the PHP exception being thrown on the server?

Hi Jonmblake.
I too am having the issue with "sending" not completing. But only some of the time, its not happening every time i try and send a contact form.
The google console shows the following -

Hope you can help
Cheers

Just to add.. ive found the php error log (finally after a log of moaning to my hosting provider)
It contains the following when a stuck send happens
join() [function.join]: Invalid arguments passed in /home/autowiiv/public_html/library/sendmail.php on line 46
[14-Oct-2016 18:15:25 America/New_York] PHP Warning:  Cannot modify header information - headers already sent by (output started at /home/autowiiv/public_html/library/sendmail.php:46) in /home/autowiiv/public_html/library/sendmail.php on line 9


Bug - When reloading the page the task state changes to incomplete

When the app gets reloaded all the completed tasks go back to be in the incomplete state. Please  @avinash201199 assign me to this issue and make it part of Hacktoberfest

Don't forget to support your pr with screen shot

Hey, @avinash201199 Can I work on this bug (When reloading the page the task state changes to incomplete)?
Please assign this issue to me.

@ankitmittal99 I'm already working on it


[BUG] config file is the config file for floodgate mod

Describe the bug
I just installed mod version 7.1.0 on fabric and when I check the config folder /config/RandomTP/config.yml I see the config for the mod floodgate which is an addon mod for geyserMC
To Reproduce
Steps to reproduce the behavior:

Install GeyserMC & Floodgate
Run Server to generate their config stuff
Install RandomTP v7.1.0 fabric
Restart server to generate RandomTP config files
check /config/RandomTP/config.yml

Expected behavior
The config file for RandomTP
Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

RandomTP version: 7.1.0
Minecraft Version: 1.19.2

Additional context
RandomTP still works, but I won't be able to change any config options for it.

For now, the fix is to temporarly remove floodgate from the mod folder and regenerate the RandomTP config. Once the config is regenerated and looks normal, re-add floodgate

This is extremely weird and I will take a look into it. Thank you for reporting it.


Update install.sh to use new easy_install url

Just came across this problem today, hope this gets merged.

Merged in a2a7d4710407be0e242e7db2b0e66dcc0082c88a, thanks!


Add SSE to SimpleTable

Closes #376
Adding optional SSE to AWS::Serverless::SimpleTable.  See #376 for more info.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@jfuss, I believe I've addressed all your concerns.
SimpleTable did not have any supported globals, so I've added SSESpecification as the only one.

SimpleTable did not have any supported globals, so I've added SSESpecification as the only one.

Thanks for adding that.
I am taking a look through the PR again now :)

@oharaandrew314 I will do my best to pull this in today. I need to figure out how to we will communicate features that are not yet deployed but are updated here. Staging it here until I have clarity on that.

Yay!  So, is there now a process for handling merged versus releases features?

Yes. We'll cut a release branch and not merge to master until the feature is available.


Basic Documentation

Set up basic documentation, and CI for hosting documentation.
The documentation largely contains existing documentation present in the wiki reworded and moved around a little.
[edit by Rachel to add Anant's nice summary from his email to me for the record]
I've included almost everything you've put in the wiki, and reworded
things a little in a few places, and moved some things around. I
decided to drop the "Basic Architecture" subsection seeing as it was
only explaining the basic architecture of any CNN (convolutional
layers, pooling layers, and fully connected layers).
I've also taken the liberty of including an additional section
explaining how and why GCNNs are a good idea for tasks like this.
The only things "pending" are documentation of the examples (for
AtomicGraphNets) and documentation pertaining to the interfaces and
functions the package provides (for both AtomicGraphNets and
ChemistryFeaturization), but I don't feel that is as important for
AtomicGraphNets as it is for ChemistryFeaturization.
For most of the equations, and matrices, I used LaTex instead, and
imported whatever images I needed into a separate directory. I assume
that would mean that we could delete the "img" folder entirely, unless
you have something else in mind for those assets.

Merging this into my new add_docs branch so I can make a few tweaks before deploying. Thanks again @thazhemadam for the awesome work setting this up! 😁


Update the year on all pages

Currently, there is 2021 on the page https://dao.ubq.fi/
Please, update year tp 2022 on all pages

Fixed on https://dao.ubq.fi/


Call to undefined function Deployer\env()

Deployer version: v7.0.0-rc.8
Deployment OS: Ubuntu

I've been running into this problem for a while now, I can no longer use env() in my deploy.php file. In my case, I'm using it to get env variables from Github Action (in this case to get certain Slack credentials) in the deployment script. What am I missing to solve this problem?
Error message:
Run php vendor/bin/deployer.phar deploy -v
 Error  in deploy.php on line [13](https://github.com/platformdroneracingnl/race-event-registration/runs/5841728614?check_suite_focus=true#step:7:15):

  Call to undefined function Deployer\env()

Error: Process completed with exit code 1.

Github Actions flow
- name: Run Deployment
  env:
    APP_SLACK_WEBHOOK: ${{ secrets.APP_SLACK_WEBHOOK }}
    RELEASE_VERSION: ${{ github.event.release.tag_name }}
  run: php vendor/bin/deployer.phar deploy -v

Deploy.php
<?php

namespace Deployer;

require 'recipe/laravel.php';
require 'contrib/slack.php';
require 'contrib/npm.php';

set('application', 'PDRNL - Racing events');
set('ssh_multiplexing', true); // Speed up deployment
set('repository', 'git@github.com:platformdroneracingnl/race-event-registration.git');

set('slack_webhook', env('APP_SLACK_WEBHOOK')); // This is R13
set('release', env('RELEASE_VERSION'));
set('slack_success_text', 'Version *{{release}}* has just been successfully deployed on the live server.');
set('slack_failure_text', 'The deployment failed, check the build logs to see what went wrong.');

Use getenv()

Yes, I tried, but the variable is not coming through.
I don't know if this has anything to do with Deployer being a phar these days?

You're absolutely right, I was fooled by Fish shell variable resolution.
getenv does the job. Thanks!

As an extra confirmation, I changed everything to getenv() and it works again 😄


Stone

Its like wood, but for castles

Money can now buy stone, but it still can't buy happiness.


UHD Ripping Not Possible (Hash File Download)

Hello,
I am trying to migrate my existing Windows MakeMKV experience to this docker container. I am not able to rip UHD Blu-ray discs in the container however. I was able to rip the test disc fine in Windows with MakeMKV auto-downloading the necessary hash file.
I have the flag checked in the MakeMKV preferences to allow an internet connection so it should auto-download the hash file to the config directory.

Can you share the log/error message you are getting ?

Below is the MakeMKV log. There are no warnings or errors in the docker container log.
MakeMKV v1.15.3 linux(x64-release) started Using LibreDrive mode (v06.2 id=96929CF53B04) Using direct disc access mode Loaded content has table, will verify integrity of M2TS files. Saved AACS dump file as /config/data/[DISC I AM TRYING TO RIP].tgz The volume key is unknown for this disc - video can't be decrypted Failed to open disc
I am able to rip the disc without any issue on Windows. Preferences > Online updates > Enable Internet access is checked.
LibreDrive is showing as enabled with all the flags reporting "Yes"

I am also having this issue though I think it is the current linux build of makemkv causing this and not specifically the containerized version as a linux laptop I've been using to rip stuff for months has been fine up until the recently while the same discs on my windows laptop open and rip without issue.

Still not working on 1.15.4
Debug log started at Mon Dec 14 23:29:07 2020 , written by MakeMKV v1.15.4 linux(x64-release)
Using 262272KB for read cache.
001005:0000 MakeMKV v1.15.4 linux(x64-release) started
001004:0000 Debug logging enabled, log will be saved as file:///config/MakeMKV_log.txt
001003:0020 DEBUG: Code 0 at "9'%z"H?E?~Q3Xii.LoDl6:29393631
001003:0020 DEBUG: Code 0 at &GNRJ"}}c+r}L5Q :29407201
SDF  v07d: ASUS_BW-16D1HT_3.10_211901041014_KLMJ4HD3200
SDF  v07d: ASUS_BW-16D1HT_3.10_211901041014_KLMJ4HD3200
001011:0000 Using LibreDrive mode (v06.2 id=0FA242DD4D0B)
003007:0000 Using direct disc access mode
005085:0000 Loaded content hash table, will verify integrity of M2TS files.
DEBUG: Code<PHONE_NUMBER> at &GNRJ"}}c+r}L5Q :121264786
001003:0020 DEBUG: Code 0 at &GNRJ"}}c+r}L5Q :121267040
003332:0000 Saved AACS dump file as file:///config/data/MKB20_v72_FLASH_GORDON_-_4K_UHD_A25D.tgz
003303:0000 The volume key is unknown for this disc - video can't be decrypted
001003:0020 DEBUG: Code 0 at ZS4lT#MS%(ZQ 'X:29396069
001003:0020 DEBUG: Code 0 at ZS4lT#MS%(ZQ 'X:29395149
005010:0000 Failed to open disc

Still not working on 1.15.4

**> 003332:0000 Saved AACS dump file as file:///config/data/MKB20_v72_FLASH_GORDON_-_4K_UHD_A25D.tgz

003303:0000 The volume key is unknown for this disc - video can't be decrypted**

the volume key is not in the database, submit the dump and wait for mike to add it!

I have submitted a number of times. I was told there was a problem in the Linux 1.15.3 version and to wait for .4. Just letting you know it was not fixed.

I have a one UHD disc that I have this same issue. Without libcurl installed into the docker, neither 1.15.3 nor 1.15.4 are able to retrieve the UHD updates. MakeMKV does not flag in its logs when it is unable to make the connection to download the updates, but does show in the log when it does:
[makemkvcon] Loaded content hash table
[makemkvcon] Downloading latest HK to /config/data ...

I get this with any UHD disk. Regular BR disks work fine.

libcurl has been added, can you try the latest image ?

Yes! You rock, thanks!!

The latest build also worked for me.

👍


Deploy

This is an automatically generated release PR!

I will wait for https://github.com/artsy/metaphysics/pull/3238 to be merged and then deploy both schema updates together.


funky css issue in ticket form - events

refreshed and this happened:

@piamancini That grid is the new layout that @xdamman just released. https://github.com/opencollective/frontend/pull/712
I'm hoping my recent css fix will resolve the funky css issue after I release it today.

Yes I made the change to show the list of tickets as a grid because on the events page we don't show them in the sidebar. Maybe @cuiki can take a quick pass at it to make those look better.
Also happy to revert this small change. It's literally just two lines of CSS. Lmk.

yes please, let's revert until we have  a better design for it

What about this (without stretching the height of the tickets and making sure they align)

Versus:

I think that if we want to re-design this, we should involve @cuiki and do a more comprehensive  re-design.
I don't see how this changes anything and it seems like we have higher priorities atm.

Hi everyone, for this particular case, I think that what @xdamman did is a good improvement vs the stacked elements like we had (which works on mobile but not on desktop).
This will eventually have to adopt the new styles and we can take that opportunity to redesign the layout if needed.

@piamancini this is not a redesign. This is two lines of CSS to make a quick small improvement.
@cuiki thanks for your input. If you have small improvements to suggests, please send them away. It's not a priority but as we fix bugs on those pages, we may as well make small CSS changes that sometime can go a long way.

I hear you. I share that frustration. We still haven't found the right process. On one side we have the homepage that is super well done and well implemented but it took 3 months to get it done (and it's not fully finished yet since it's still lacking the video section at the bottom), and on the other side we have everything else that we have been doing that is made of a lot of patches here and there and not well thought out user flows.
Maybe we can find someone externally who can help us figure out how to manage this process?
I'm obviously not the right person for this type of process thinking.


HV-1863 HV-1900 Better (and tested) support for records

https://hibernate.atlassian.net/browse/HV-1863
https://hibernate.atlassian.net/browse/HV-1900

I merged this, thanks. Gonna work on pushing it to the other branches now.

@gsmet Can you give an indication regarding when this change will be released 🙂 ? Thanks

@gsmet @yrodiere Any chance for a new release? :-)

@Katalystical I will push one today.

@Katalystical sorry I got a bit sidetracked. 6.2.4.Final is available on Maven Central now.


A bug in get rest_06_options_chain.py

rest_06_options_chain.py example
info = client.search_secdef_info_by_conid(
conid=ibit_contract['conid'],
sec_type='OPT',
month=options['months'][0],
strike=strikes['call'][0],
right='C'
).data
uses the first letter of the string months, in my case D (DEC24)

Hey @levshinD thanks for reporting the issue. I've just pushed an update to that example on master branch, let me know if it works for you now 👍


Code block encapsulated two times

As can be seen in the previous image, the result of creating a code block is two boxes instead of one. The right way a code block should be parsed is shown in Jekyllnow official website.
I'm very new to jekyll. In fact, I started with jekyllnow just with the purpose to create a personal blog. I'll try to solve the problem myself, but a little help or orientation would be much appreciated.

After some time looking over the problem I could come out with a quick solution. As you can see in the picture below, when kramdown parses the code block it creates div and pre tags both with the "highlight" class. This class is responsible for styling the code block. Either there should be one of the two tags or only one of them should be marked as "highlight".

As per the html of the block code here, I would dare to say that at some moment kramdown started including one more <div> when parsing a block code. This resulted in the error exposed here.
To solve the problem, I changed the highlights css to affect the pre tag instead of the highlights class (in _highlights.scss). Now, the code block is styled in the pre tag, of which there's only one.


upgrading to 5.0.0 all Script/Program Devices now Fail

As above, when running with the 5.0.0 Jar all my devices which are using Script blocks to (mostly) execute Python scripts.
everything was fine on the last version.
I can see this in the logs which looks suspect:

WARN
Error reading the file: data\group.db - Does not exist or is not readable. continuing...
com.bwssystems.HABridge.dao.GroupRepository

That is just a warning telling you there is no group file as you haven't set them up.
As far as the script issues, I assume you are using the exec function to call a local script. Turn on debug for com.bwssystems.HABRidge.plugins.exec and post your log.

Any update?

Check your antivirus. You may need to add an exclusion for the Ha-bridge folder

Sorry, been on vacation.
I did not change any anti virus settings, however after putting 5.1.0 on, everything is working again.
Not sure what it was in 5.0.0 that broke it.


[RFC] Should provide the setting for enabled?

I wonder if we can provide the setting for visible? In some scenes, I want to disable all listeners when I drag an element or do something else. We could provide enable, disable, and isEnabled to resolve these problems. #215

I got it. But if I use a special scope. I have to record the current scope to reset scope when I want to disable it. It doesn't feel intuitive.


About Create Snapshot Into Hive

hello,
I've configed hdfs and hive connection, and when i create a snapshot into Hive, it works well, but it appears error when i check the table in Hive:

And when i check the snapshot created in the snapshot list of metatron:
console_error.txt
If it was a BUG, i think the bug source should be the process of creating snapshot into Hive.

@BookXHD Can you tell me which version of hadoop and hive you are using?

@kyungtaak ok, hadoop v3.0.0 and hive v2.1.1, and both deployed in CDH 6.1.1

@kyungtaak And here is my test data:
small4.xlsx

@BookXHD Thank you for your information. My colleague can confirm it.
@joohokim1 Can you check this issue?

@kyungtaak @BookXHD Yes, I'm on it.

@BookXHD @kyungtaak In my environment, creating and viewing the snapshot from small4.xlsx works fine.
So, I'm going to build the same environment to produce the bug, it might take some time.

@joohokim1 thank you, and by the way, what is the your environment which could work fine ?

@BookXHD It's Apache Hadoop 2.7.3 + Apache Hive 2.3.0. They are from their own official sites. I ran on MacOS Mojave. May I ask the OS where you tested? (I guessed it's linux, so I'm going with a CentOS docker.)

@joohokim1  I tested on MacOS Mojave too.

@BookXHD Could you tell me the senario more detail? You just upload the excel file and create snaphot?
Is there any apply rules?

@eldorado0402  yes, i just upload the small4.xlsx as a dataset, and without applying any rules, i create  a snaphot in hive.And it is all goot up to this step, but when i check the hive table or the snaphot, they are all not good.

@BookXHD I tested the senario in my local pc including hadoop 3.0.0. But the problem was't happended. I will try to do it again in cdh 6.X and share the result. Thanks

@eldorado0402 i can location the detail of this error now, it occurs when using the OCR(Textfile is not working either, with all of messy code), i found that 2 things:
1.the error"ArrayIndexOutOfBoundsException"is a kown typical error with OCR, but aftering search a lot, i still can not solve it.
2.My hive could use MR or Spark, but not Tez, will it make influence?

@BookXHD You mean CSV-based Hive does work?

@BookXHD
Sorry, we tried best. I hope we can see this issue in further development soon.
Closing.


Translate for a player-"group" in Dnd5 3.0.2

User Experience
If I use a player-group-actor (don't know i this is only for dnd5 3.02) on a large scale map, like a country, Polyglot does not translate for the individual players of the group. The individual player actors need to be on the map to give them the tyranslation.
Would be nice if it does also work on the group-actor.
Have you read the Wiki?

[X] Yes

None of your players have a character assigned to them? https://github.com/mclemente/fvtt-module-polyglot/wiki#actor-selection

Players have a character assigned to them, but on this map I use a players group. Rather not have 5 people running around in the city. :-). Polyglot is working if i put the characters themself on the map, but not if i use the player-group icon. As a gamemaster I can see the languages the characters have, even if I use the player-group icon. But the players do not see the translation if I use the group-icon.
By the way. I like Polyglot very much. Thanks.

Then their Group actor's ownership permission is set to Owner and, as that is the only owned actor they have on the map, they will read and send messages as that actor.
That is how Foundry handles owned tokens on a scene, Polyglot isn't doing anything.
If you want them to be able to read and send messages as their actors and still be able to move the Group's token, add their actors somewhere on the scene and make walls around them so they don't move.
The alternative would be to reduce the ownership permission to at least Observer and move the token yourself.

I have just updated the wiki with a tip on how to handle that behavior.
You could add the players' actors to the map, wall them off with ethereal walls, and then set them as hidden. https://github.com/mclemente/fvtt-module-polyglot/wiki/Understanding-Core-Behavior


[JBMAR-221] Test checks if default constructor of non-serializable parent POJO is called in deserialization process

JIRA issue: JBMAR-221

POJO is initialized on client side using it's parent constructor.
One of the fields of the object is set to different value.
Remote stateless EJB is called and the object is used in the EJB call.
Field values received by the server are being checked to see if the default constructor of the non-serializable parent was called on EJB server side.

Do not merge now, waiting for the fix to be merge.

Why is the test in the JBMAR PR insufficient?

I did not know about the test from PR, we can close this.


fix: bump nvd3 plugin for pie chart label type

SUMMARY
Fixes https://github.com/apache-superset/superset-ui/issues/697
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TEST PLAN
CI
ADDITIONAL INFORMATION

[x] Has associated issue: https://github.com/apache-superset/superset-ui/issues/697
[ ] Changes UI
[ ] Requires DB Migration.
[ ] Confirm DB Migration upgrade and downgrade tested.
[ ] Introduces new feature or API
[ ] Removes existing feature or API

@ktmud @mistercrunch This is to update nvd3 plugin in superset for pie chart label type.

@stuarthu you need to rebase to fix a conflict in the lock file.

@stuarthu you need to rebase to fix a conflict in the lock file.

done


How to handle unused code in a shared package within a monorepo

Hello, guys!
The question is as follows: currently, when adding any unused imports to the packages/shared module in the monorepo repository, knip does not raise any complaints about them (see the image below). Is there a way to make knip complain about getRandom?

Live repo:
https://github.com/what1s1ove/whatislove.dev
Knip config:
/** @type {import('knip').KnipConfig} */
let config = {
	workspaces: {
		'apps/certifications-whatislove-dev': {
			entry: [`src/index.jsx`],
		},
		'apps/whatislove-dev': {
			eleventy: {
				config: `eleventy.config.js`,
				entry: [`src/data/**/*.js`],
			},
			entry: [`src/scripts/index.js`],
		},
		'packages/shared': {
			entry: [`src/index.js`],
		},
	},
}

export default config

Project structure:
apps/
  certifications-whatislove-dev/
    index.js
    package.json
 whatislove-dev/
    index.js
    package.json

packages/
  shared/
    index.js
    package.json

knip.config.js
package.json

So you'd need to use --include-entry-exports, but the issue you linked to shows that you know that. Furthermore, I'd really need a minimal repro to look into (not screenshots or a scenario that I need to imagine).

So you'd need to use --include-entry-exports, but the issue you linked to shows that you know that. Furthermore, I'd really need a minimal repro to look into (not screenshots or a scenario that I need to imagine).

Hey @webpro !
I'll share the repository with you a bit later. I'll also share the solution I was able to come up with. It's a bit strange, but it works. Perhaps you can provide some insights on how to improve it.

I don't have a minimal repro currently, but enabling that flag in our repo gave me a billion false positives, unfortunately :(
I would say it doesn't follow source maps, is the problem? Entry file used to detect code usage in the package is src/index.ts, but entry file to the package is dist/index.js (and that's gitignored). I don't think knip understands the relationship between the two?

I don't have a minimal repro currently, but enabling that flag in our repo gave me a billion false positives, unfortunately :(
I would say it doesn't follow source maps, is the problem? Entry file used to detect code usage in the package is src/index.ts, but entry file to the package is dist/index.js (and that's gitignored). I don't think knip understands the relationship between the two?

Yes, I had the same issue. I had to use the --no-gitignore flag, add an entry for dist/index.js, and it worked with several exceptions.

Hey @webpro!
I have prepared a test repository - https://github.com/what1s1ove/knip-playground-ts/tree/main
So, once again, the problem I want to solve is - when using a typescript monorepo, when there are unused things in packages/shared, knip should complained about them.
In the example repository, the packages/shared exports two functions, foo and bar. The function foo is used in apps/app, while the function bar is not used anywhere. I want knip to complain about the unused function bar.
As already mentioned, to detect unused items from packages/shared in other applications, the --include-entry-exports flag should be used. However, it's not as simple as that. And as I managed to find out, the issue is not with the flag itself but with how we import items from this module.
Since npm cannot import non-js files, it means that we need to convert all files in packages/shared from .ts to .js. In other words, perform a regular build and make the entry point for this package not src/index.ts but build/index.js.
// packages/shared/package.json
{
	"name": "@kp/shared",
	"main": "build/index.js",
	"types": "build/index.d.ts",
}

Accordingly, if knip has the entry set to src/index.ts, it will complain about all files as unused because other applications import items from packages/shared specifically through packages/shared/build/index.js as the entry.
It means we need to specify the entry for knip for packages/shared as build/index.js, and everything should work. But I couldn't make it work either, as knip continued to complain about unused files that were actually used in other applications.
The only thing that helped me solve the problem was to specify the entry point exactly as build/index.d.ts, while ignoring all other d.ts files and all files in the repository.
// knip.js
'packages/shared': {
	entry: ['build/index.d.ts'],
	ignore: ['src/**/*', 'build/helpers/**/*.d.ts'],
},

Only with this configuration, I was able to make knip complain about the unused function bar when using npx knip --include-entry-exports.

The issue primarily stems from the fact that Knip is about source code, not build artifacts.
Adding build artifacts as entry files to the configuration does not "fix the chain", especially considering they're .gitignored (and also source maps would not get interpreted).
Feel free to bend and flex Knip any way you like, but I just can't support this use case atm for multiple reasons, and I'm not sure if that will ever be the case.

Hey @webpro!

especially considering they're .gitignored

Exactly! That's why I had to use the --no-gitignore flag.
During my latest experiment with integrating knip into another project, I made a slight improvement to the solution. By specifying the main key in package.json as the path to the built file build/index.js, and the types key as the path to the source TypeScript file src/index.ts, everything works beautifully.
Shared folder - https://github.com/what1s1ove/knip-playground-ts/blob/main/packages/shared/package.json
Knip config - https://github.com/what1s1ove/knip-playground-ts/blob/main/knip.js
Once again, you rightly noted that this solution won't suit everyone, as I mentioned earlier in the post, since the builds and workflows of all monorepo projects vary, unfortunately. There are so many tools nowadays to achieve the same thing...


add a target image into the dataset generation command

i.e. add an image url for the target. This should not be in the captcha though!

Why? What is the benefit from this?


[5.5] Cast to object before json_encoding

Both Support\MessageBag and Eloquent\Model hold their data as an associative array, and their JSON representation is the simple the "json_encode-ing" of that array. It's an implicit contract that a key-value structure is returned.
However, when that array is empty, the resulting JSON is [] instead of the expected {}. Thus, that contract is broken, from the point of view of the consumer of the json string.
Consider  (related and following up to #21605 ) a Vue component to which I'm passing a model and the validation errors.
<user-form :model="{{ $user }}"
           :errors="{{ $errors }}"
></user-form>

While the component expects an object, if the error bag (or, less likely, the model) is empty, an empty array is effectively passed. This change ensures the resulting JSON represents an object.

This is a breaking change.

I'll ignore the fact that you decided to send a PR that may break existing 5.5 installations because you're unhappy with the data format in your specific use case, and instead ask you if forgot to write tests, or just forgot to add those test files to the commit?

@sisve Well, I tried to provide reasons as to why this could be considered the breaking of a contract (it's  a different data type from the perspective of a JS client for instance), so it's not about "my specific use case", is it.
Speaking of breaking, I do realize this may be potentially breaking, so I will re submit it to a different branch.

@alepeino when you say it fails for a use case, it is better to provide a test that shows the use case.

Send it to Master

@Dylan-DPC @deleugpn will do, thanks!


[WIP] Prereg Prize

Post Draft-Reg additions

Add generic custom login/registration flows with 'campaigns'
Add Prereg splash page: . It's my understanding that much of this code is borrowed from the onboarder, but this is almost all work done by Lyndsy. QA has fairly thoroughly tested this page, so unless their are major errors or blockers in the code we can give this a lighter touch in CR.

views in website.prereg to support this
JS added:

`...js/pages/prereg-landing-page'
`../js/components/autocomplete' for autocomplete widgets on the prereg landing page

Add the 'Prereg Challenge' registration schema
Updates the registration editor (QA has thoroughly tested this):

Bugfixes and improvements to file picker
Add a contributor importer
Many small improvements/bugfixes

Add special PreregCallbackMixin class to handle sending a custom-branded email to Prereg applicants when their registration or embargo is made active
Update DraftRegistrationApproval class to actually implement on_complete and on_reject callbacks-- This code is unreachable until the admin interface goes live
Refactor registrationEmbargo.js into componentized registrationModal.js and add tests
Add scripts to auto-approve/reject draft registration approvals (testing only)

Replacing: https://github.com/CenterForOpenScience/osf.io/pull/4602

@felliott, replacing https://github.com/CenterForOpenScience/osf.io/pull/4602. I will still ping you when this PR is ready for review.

First-readthrough complete. I'll pause while a few remaining comments are addressed; passing back to @samchrisinger .
I'll pick this back up once responses are done, and once all unit tests are passing.

Functional/ style review:
The instructions portion of a prereg question should stand out more distinctly. Possibly remove the well and blockquote styles so that data.description text simple appears to be part of an unstyled white-background area at the top of the page.

A question is required, even when not relevant.
I'm not sure if this can be changed, but it's a bit disconcerting to the user. Thoughts?

The tab order goes Last question box --> Save as draft. I can't seem to tab to the "next page" button at all, which is awkward for users who interact with the form via keyboard controls.
(Eg, an accessibility concern)

I uploaded a file for an optional question, but did not click on it.
It showed up as selected in the UI anyway (which is good), but the "preview for submission" page said no file was selected (which is confusing)
Also, there should be no period in the "preview for submission" question; .: looks odd. (see second screenshot)

The purpose of the comments box is unclear on this page- some explanatory text might be helpful.
Also, the comments only appear on the first page of the registration preview screen. It's not very intuitive where to find these. (I'm not sure whether to suggest that this be its own item on the sidebar nav, or what to recommend- depends on the use case)

I went back and selected the optional analysis script. This representation is not a lot more helpful:

Showing the filename would be preferred.

Light functional/style pass complete.

I uploaded a file for an optional question, but did not click on it.
It showed up as selected in the UI anyway (which is good), but the "preview for submission" page said no file was selected (which is confusing)

I am still seeing this issue with the newest branch.

Style guide comments. I'm ok with resolving these in a followup (provided issue is tracked):
When deleting a draft from the registrations page,

The button should be read
The text should reference an action, not the word "ok"

https://centerforopenscience.github.io/osf-style/style.html#buttons

Confirming fix for filepicker bug works for me.
Tested on Firefox and Chrome, Mac OS.


Get full config from config_updates

I am creating a script to run a variety of experiments with different configurations. Currently I do this, by calling ex.run(config_updates={...}) in another Python script. I want to check whether an experiment with the same config has already been run, to avoid performing reruns in certain situations.
I can currently get the config of a run that has already been run from the output of the FileStorageObserver by loading config.json. However, I don't know how to get the final config of the experiment that I am about to run. Essentially, I need to figure out how the config_updates combine with the default settings to the final settings dict. Then I need to figure out whether they are the same.
Does sacred provide any functions that can help with this?

The solution of using ex.configurations[0](fixed=config_updates) to get the experiment settings dictionary seemed to work for me. Is this a recommended method?

Your solution (ex.configurations[0](fixed=config_updates)) only works in a specific special case. That is, you have an experiment without Ingredients, your experiment defines exactly one config scope, this config scope is not a named config, and you don't load any configuration files via the command line interface.
Maybe #322 helps. I modified the solution from there to match your use case:
# define your experiment...
ex = ...

# This creates a run object and builds the full configuration. This is exactly the same that is
# executed internally when you call ex.run(...)
run = ex._create_run(config_updates=config_updates, named_configs=...)

# Now, run.config contains the constructed config
# Check for duplicates but be aware that run.config now additionally contains the (random) seed.
if run.config ...:
    # Run the experiment by simply calling the run object
    run()

I think this is the best solution you can get, but it involves calling a protected method _create_run which is not so nice.

Thanks a lot. This is really helpful to know, especially as my experiment setups become more complex.

The solution of @thequilo is indeed the best way to do this. We should probably make _create_run a public method since it has so many usecases.


SOLR-16784 Badapple CollectionReloadTest

https://issues.apache.org/jira/browse/SOLR-16784

I am going to merge as a badapple for now, and maybe someday if I learn more about how all this works, I can dig into the cause!   Thanks for doing this @janhoy


snapshot is a reserved word in redshift and needs to be quoted

sqlmesh.core.engine_adapter.base - DEBUG - Executing SQL: CREATE TABLE IF NOT EXISTS sqlmesh._snapshots (name VARCHAR(MAX), identifier VARCHAR(MAX), version VARCHAR(MAX), snapshot VARCHAR(MAX)) (base.py:761)
But since 'snapshot' is a reserved word in Redshift, this query fails with a syntax error. We must quote reserved words when using them in column names.

FYI I just ran into this bug

fixed by https://github.com/tobymao/sqlglot/commit/8440ea4565a626fb65197e8142b99430efbe54ea


Driver Drowsiness Detection

💥 Proposal
In this project, I'll be making a Driver Drowsiness Detection System built using Python, Open CV, AND Dlib.
Since countless people drive on the highway day and night. Taxi drivers, bus drivers, truck drivers, and people traveling long-distance who suffer from lack of sleep.
Due to this, it becomes very dangerous to drive resulting in the majority of accidents happening due to the drowsiness of the driver.
So, to prevent these accidents I will build a system that will alert the driver whenever they feel sleepy.
This project aims to build a safety technology that will detect if the driver’s eyes are closed for a few seconds and will alert the driver when drowsiness is detected.

Please assign this project to me and allow me to contribute my hard work to GSSoC'23.

@transponster27 - you can go ahead! We are assigning you 21 days for this project, after which it will be assigned to someone else if not completed. All the best!
Name the file as: algorithm_dataset.ipynb and link it in the readme of the labeled directory as algorithm - dataset.


Cocktail Sort

Description
Documentation on Cocktail Sort
Domain
DSA
Type of Contribution
Documentation
Code of Conduct

[X] I follow Contributing Guidelines & Code of conduct of this project.

/assign

Closing the issue since deadline to open new PRs is over. Thanks for all your contributions.


Inline private variable

Sure thing! Also... err... there's still low hanging fruit (mostly cleanup work) in rules_scala. I'd argue that it improves maintainability of the codebase and definitely isn't cheating.

I figured you were joking, I just wanted to poke back 😄.


Replace _Reactor to ReactorKit.Reactor

Codecov Report

Merging #79 into master will not change coverage.
The diff coverage is n/a.

@@         Coverage Diff          @@
##           master   #79   +/-   ##
====================================
  Coverage      93%   93%
====================================
  Files           6     6
  Lines         100   100
====================================
  Hits           93    93
  Misses          7     7

Impacted Files
Coverage Δ

Sources/ReactorKit/Reactor.swift
82.5% <ø> (ø)
:arrow_up:

Sources/ReactorKit/Stub.swift
100% <ø> (ø)
:arrow_up:

Sources/ReactorKit/View.swift
100% <ø> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0e71698...055a972. Read the comment docs.

Wow I like it :heart:


repair index --read-all-packs creates empty index record - feature or bug?

Output of restic version
restic 0.17.0 compiled with go1.22.5 on linux/amd64
What backend/service did you use to store the repository?
local file and Microsoft Onedrive
Problem description / Steps to reproduce
when running restic -r  repair index --read-all-packs on relatively small repositories, the above command creates a new index file and an empty index file with the contents '{"packs":[]}\n' (length
55 bytes)

Expected behavior
I expect one index file, particularly when the second file is logically empty.

Actual behavior
I see tho files listed.

Do you have any idea what may have caused this?
restic repair index rebuild process

Did restic help you today? Did it make you happy in any way?

I see tho files listed. - spelling mistake: 'tho' should be 'two'

This is just a minor cosmetic issue that also only shows up for very specific pack file counts. Empty indexes are ignored while loading them.
If someone wants to take a stab at this, the relevant code section is at https://github.com/restic/restic/blob/76d56e24d615b2460247bf184ed4c0906d8e39eb/internal/repository/index/master_index.go#L448 where the index should only be sent towards saveIndex if it is not empty.

I had a look at the relevant code; I added a few log records. This is the log output:
See attached file log_extract.log.
The section in repository/index/master_index.go, function Rewrite, regarding the building of the new Index is in my opinion unnecessary, since repo.Flush() has already done that successfully. Only the task of removing the old indices needs to be done.
log_extract.log

here is the fix: git diff internal/repository/index/master_index.go
--- a/internal/repository/index/master_index.go
+++ b/internal/repository/index/master_index.go
@@ -456,6 +456,9 @@ func (mi *MasterIndex) Rewrite(ctx context.Context, repo restic.Unpacked, exclud
worker := func() error {
for idx := range saveCh {
idx.Finalize()

                  if len(idx.packs) == 0 {

                          return nil

                  }
                  if _, err := idx.SaveIndex(wgCtx, repo); err != nil {
                          return err
                  }

I attach modified file internal/repository/index/master_index.go as 'internal/repository/index/master_index.txt'.
master_index.txt - I called it ...txt

Please open a PR with the changes. Issues aren't well suited to discuss code changes.
Instead of returning, please just call continue. That way there's less risk of unexpected behavior if for some reason an empty index is sent to saving and  the index is not the last one (not sure what we'd have to break for that to happen  though)


Create the missing Algorithms in C++

Follow CONTRIBUTING.md

I would like to work on this. TIA

I would like to implement Binary Search Tree Algorithms. Please assign this issue to me.

I would like to work on this implementing a modified binary search, please assign this issue to me. TIA

I would like to work on the implementation of the XOR swap,Please assign this task to me. TIA

I would like to work on this.

@BarnikRay @sourabbr @kab11 Thank you for your interest 👍 Please check the C++ folder for missing algorithms.

i would like to add  some thing

@afAditya  Please check the C++ folder for missing algorithms.

I would like to work on this

Can I work on this issue?

@DebadityaPal @vaishali614 Thank you for your interest 👍 Please check  C++ folder for missing algorithms.

I would like to work on this

@giatuongtran9  Please check C++ folder for missing algorithms.

I added factorial in it.

I would like to work on this.

@Thuva4 Hi :) Can I add C++ AVL Tree for Range Tree data structure?

I just add Factorial Algorithm for C++


Bump Elixir version to 1.11

Because of https://github.com/elixir-sqlite/ecto_sqlite3/pull/84 this library needs Calendar.strftime/3 which is only available in Elixir 1.11+

Oh thanks for taking care of this for me 😄


Update bios for 2023 students

Please add new bios to the about/bios.md page

Hi @Ruwaidar just a reminder to finish this up ASAP.

Do let me know if anything needs to be changed


[Feat]: Adding Loader

What feature?
We can show a loader to the user whenver the user changes the pages and when the website loads for the first time.
Add screenshots
Add screenshots
Code of Conduct

[X] I agree to follow this project's Code of Conduct

You can work on this after completing the previously assigned issue.

I would like to work on this issue plz assign me!! @JasonDsouza212


Update build.gradle

Thanks for the PR. I didn't know people are actually using this :) It's basically PoC-level code I threw together..


Add [KuCoin kucoin.com] #exchange #memo-required GAJ4BS…BQJW2P

Address: GAJ4BSGJE6UQHZAZ5U5IUOABPDCYPKPS3RFS2NVNGFGFXGVQDLBQJW2P
Tags: exchange memo-required
Title: KuCoin  Domain: kucoin.com
Requested by @rakibsaleh01

Hi ,some how i put wrong memo .can you please send  it to<PHONE_NUMBER>

Hello @Rakibsaleh01
Stellar Expert is just a blockchain explorer and does not have any control of transactions on the block chain.
You will need to contact KuCoin kucoin.zendesk.com/hc/en-us/requests/new they will also need to know the transaction ID which can be found by clicking on the operation number next to the transaction.
Looking towards the future with you,
Rob

Hello @Rakibsaleh01,
Please feel free to re open this issue if you have new information to let us know about.
Looking Towards The Future With You,
Rob


Need assistance with a timestamp containing miliseconds

I have access log in the following format:
$datetime|$remote_addr:$remote_port|"$scheme://$host$request_uri" $status $body_bytes_sent "$http_referer" "$http_user_agent" "$http_x_forwarded_for" "$upstream_addr" "$upstream_status" "$upstream_connect_time" "$request_time"

Sample:
2018-09-24 18:56:01:121|<IP_ADDRESS>:29390|"http://requested_URL" 206 24387 "-" "user-agent" "<IP_ADDRESS>" "<IP_ADDRESS>:80" "206" "0.001" "0.656"

I almost made it work, but can't get how to proceed miliseconds.
Here is what I have tried:
'%d %T:%^|%h:%^|"%r" %s %b "%R" "%u "%^" "%^" "%^" "%^" "%T"' --date-format=%Y-%m-%d --time-format=%k:%M:%S - shows almost all statistics but no time distribution

'%d %t:%^|%h:%^|"%r" %s %b "%R" "%u "%^" "%^" "%^" "%^" "%T"' --date-format=%Y-%m-%d --time-format=%k:%M:%S:%L

and
'%d %t|%h:%^|"%r" %s %b "%R" "%u "%^" "%^" "%^" "%^" "%T"' --date-format=%Y-%m-%d --time-format=%k:%M:%S:%L

lead to
Token '12' doesn't match specifier '%t'
Token '12' doesn't match specifier '%t'

How I can skip miliseconds and get correct time distribution chart?

Can you please try with --time-format=%T:%3N? (haven't tested)

I have tried. Time distribution chart is empty :(

It looks like strftime() doesn't support the milliseconds. So you will have to do some preprocessing and strip it out.  Please build from development so you can properly parse a log containing pipes on it.
This works for me:
sed 's@\(..\):\(..\):\(..\):\(...\)@\1:\2:\3@' access.log | goaccess --log-format='%d %t|%h:%^|"%U" %s %b "%R" "%u" %^ %^ %^ %^ "%T"' --date-format=%Y-%m-%d --time-format=%T --http-method=no --http-protocol=no

Oh, so there simple way to teach parser that it should ignore miliseconds field?
Frankly speaking I was expecting --time-format=%k:%M:%S and %t:%^ should do the trick, why it doesn' work?
One more doubt about pipes - for me only time distribution chart doesn't work, but other things seem to be ok, what's exactly the problem about pipes?

There's no simpler way. If this becomes an issue for folks, I can work on a fix directly in goaccess. I'm not sure so far people are recording the ms part though. Is there any reason why you are recording the ms field? are you running nginx/apache?
%t:%^ is only grabbing the hour part and discards everything else, thus it doesn't solve the issue.
A bug was reported not too long time a go regarding pipes as delimiters. This was fixed upstream. If you don't build from development, it's likely it won't parse each field correctly or miss some panels as in your case.

Wow, logs containing millisconds, thats hardcore.
i manage servers with over 32 million requests a day and frankly never saw the need for using milliseconds. I am curious as to why you log this way too.

It's defaul configuration provided by solution's vendor.
Yes, it's Nginx log.
If you guys insist it's overhead - I will change it :)

@zoulja Please try the following before building from development:
# apt-get install gettext autopoint

@allinurl, sorry for bothering you, but I have SUSE, in isolated environment, so it's not  that simple.
I understand resources of one person is not enough to support all distros, so let's close this, I will stay on 1.2

Try:
# zypper in gettext-tools

Closing this. Feel free to reopen it if needed.


The detection results on rviz are very stuck

Thank you for open sourcing this excellent work!
When I use the Velodyne16 lidar data in gazebo to test the algorithm, the detection results on rviz are very stuck, as shown in the video below:
https://github.com/ethz-asl/dynablox/assets/55040440/9e55e986-3820-4e57-bc50-7767bc61212a
In addition, the terminal keeps printing (I ignore it for now):

Failed to find match for field 't'.
Failed to find match for field 'reflectivity'.
Failed to find match for field 'ambient'.
Failed to find match for field 'range'.

Some changes are as follows:

Modify the lidar hardware parameters in "default.yaml":

sensor_horizontal_resolution: 2048
sensor_vertical_resolution: 64
sensor_vertical_field_of_view_degrees: 33.22222
sensor_horizontal_resolution: 720
sensor_vertical_resolution: 16
sensor_vertical_field_of_view_degrees: 30

Modified the lidar topic and tf in "play_doals_data.launch".

 <!-- <remap from="pointcloud" to="/os1_cloud_node/points" />
 <remap from="~pointcloud_corrected" to="/pointcloud" />
 <param name="odom_frame_id" value="map" />
 <param name="lidar_frame_id" value="os1_lidar" /> -->

 <remap from="pointcloud" to="/velodyne_points" />
 <remap from="~pointcloud_corrected" to="/pointcloud" />
 <param name="odom_frame_id" value="odom" />
 <param name="lidar_frame_id" value="velodyne" />

What else do I need to modify? I would be grateful for any help!

The CPU of my computer is i7-10750H, which is about the same performance as AMD-4800U. I downloaded the small_town_simulation sequence in DOASL for testing, but it is still stuck. The results are as follows:
https://github.com/ethz-asl/dynablox/assets/55040440/13714770-7816-40a1-b84f-3e75f7c3fcba

I downloaded hauptgebaeude_sequence_1 in DOASL for testing. This sequence performed very smoothly without any lag, which is so strange! The video is as follows:
https://github.com/ethz-asl/dynablox/assets/55040440/4cc09052-9b37-43af-b7fd-69e42edf6263

Hi @504115681
Many thanks for your interest! That is very strange, we haven't really had problems being 'stuck'. Few pointers that may hoepfully help you resolve your issues:

The visualization (including the entire point cloud) is published after a frame is processed, so if no new visualizations appear the system is still processing or waiting to process the next frame.
Some irregularities can arise if the max integration distance is not set and a very large extent pointcloud is received, as the entire volume will be integrated before moving to the next frame. This can be expensive and take some time (see computation time results for fixed and open max integration range in the paper)
It could also be related to the tf's not being received, as the system waits for the transform of the pointlcouds to arrive.
In the past I had issues with ROS subscribers getting slow for long queue sizes, in your videos you can see the small time lag of the published (most recent processed) pose by dynablox and the simulated robot pose, indicating that not all frames are being processed in real-time. However, this lag was usually negligible in our experiments.

W.r.t. Failed to find match for field 't': This is printed by the lidar undistortion, not by dynablox, and is not an issue (unless you need the cloud to have these fields later)

Hi @504115681 Many thanks for your interest! That is very strange, we haven't really had problems being 'stuck'. Few pointers that may hoepfully help you resolve your issues:

The visualization (including the entire point cloud) is published after a frame is processed, so if no new visualizations appear the system is still processing or waiting to process the next frame.
Some irregularities can arise if the max integration distance is not set and a very large extent pointcloud is received, as the entire volume will be integrated before moving to the next frame. This can be expensive and take some time (see computation time results for fixed and open max integration range in the paper)
It could also be related to the tf's not being received, as the system waits for the transform of the pointlcouds to arrive.
In the past I had issues with ROS subscribers getting slow for long queue sizes, in your videos you can see the small time lag of the published (most recent processed) pose by dynablox and the simulated robot pose, indicating that not all frames are being processed in real-time. However, this lag was usually negligible in our experiments.

W.r.t. Failed to find match for field 't': This is printed by the lidar undistortion, not by dynablox, and is not an issue (unless you need the cloud to have these fields later)

Thank you for your help! Its a problem with TF, everything works fine now!

Hi @504115681 ,
I am experiencing the same issues when running the small_town_simulation sequence. Other DOALS sequences run well.
How did you solve the tf problem?
In my case it seems to be caused by lidar_undistortion with the warning Waited 2.500s, but still could not get the TF from os1_lidar to map.


Enable lll line length linting, fix errors

Signed-off-by: Daniel Farrell<EMAIL_ADDRESS>

It seems like the embedded YAML auto-generation is causing long lines for lines with links.


Error At Percentile

🚀 Feature
A regression metric that measures the error of the model at a certain percentile (e.g, in a price prediction model, 95% of the houses will be within 10,000$ of the projected price).
Motivation
I personally used this metric in two very different projects, so I figured it could be useful to other people. It's a useful metric when trying to estimate risks from using the model in real- life settings, since it helps to quantify the amound of cases where the prediction will go horribly wrong.
Pitch
I already wrote the code and I'm using it, so this should be pretty straightforward :)
Alternatives
Additional context

Hi @YovelRom, seems like a great addition to the regression package.
Would you be up for sending a PR?

@YovelRom are interested in contributing this metric? :rabbit:

After taking a second look at this, I have not been able to find any references or open-source implementation of this metric. Closing the issue.


swapping issue

Describe the bug
many attempts to swap eth - bull or vise versa but can't go through
Steps to reproduce
Steps to reproduce here.
Additional context
Additional context here.

What do you mean by not being able to swap? Does the transaction hang at pending?

Closing this issue due to lack of response from the issue author.


url without http

would be nice if there would be an option to validate a url without http(s) at the beginning

That would make a useful option. Without it, you could use urllib from the standard library to drop the scheme from the given URL (urlunparse(('',) + urlparse(url)[1:]).lstrip('/')). As a utility function:
# Python 3.x

from urllib.parse import urlparse, urlunparse

def drop_scheme(url):
    """Return URL string without scheme (e.g. 'http://').

    Example:

    >>> drop_scheme('http://www.example.org')
    'www.example.org'
    >>> drop_scheme('ftp://www.example.org')
    'www.example.org'
    >>> drop_scheme('example.org')
    'example.org'
    >>> drop_scheme('http://www.example.org/~username/index.html')
    'www.example.org/~username/index.html'
    >>>
    """
    parsed = urlparse(url)
    return urlunparse(('',) + parsed[1:]).lstrip('/')

if __name__ == '__main__':
    import doctest
    doctest.testmod()

You could then use this approach:
>>> validators.url('http://' + drop_scheme(url))
True
>>>

well, i already have a workaround for this, maybe yours is even better. anyway, implementing this would be nice!

I'm sorry but a URL must begin with a scheme. Ref RFC 3986.


Timeout APIConnectionError during Large File Upload (200MB) - ECONNRESET

Describe the bug
When attempting to upload a large file (200MB) to the Azure OpenAI API using the OpenAI Node.js SDK, the request fails with an ECONNRESET error. The error seems to occur when the upload exceeds a certain size, as smaller files upload without issue.
https://platform.openai.com/docs/api-reference/files/create
To Reproduce
 const file = await openai_client.files.create({
      file: fs.createReadStream(path.join(__dirname, './uploads/testfile.pdf')),
      purpose: 'assistants',
    });

Expected behavior
The file should upload successfully without a connection reset error because there is a description in api documentation like -  Individual files can be up to 512 MB, and the size of all files uploaded by one organization can be up to 100 GB.
Additional context
Additional Info:
The issue only occurs with larger files (200MB+).
Smaller files  upload successfully.
Node.js version: 20
OpenAI SDK version: V2

APIConnectionError: Connection error.
at AzureOpenAI.makeRequest (...\node_modules\openai\core.js:304:19)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async runThread (...\fileTestOpenai.js:23:16)
at async main (...\fileTestOpenai.js:19:5) {
status: undefined,
headers: undefined,
request_id: undefined,
error: undefined,
code: undefined,
param: undefined,
type: undefined,
cause: FetchError: request to https://...openai.azure.com//openai/files?api-version=2024-07-01-preview failed, reason: read ECONNRESET
at ClientRequest. (...\node_modules\node-fetch\lib\index.js:1501:11)
at ClientRequest.emit (node:events:517:28)
at TLSSocket.socketErrorListener (node:_http_client:501:9)
at TLSSocket.emit (node:events:529:35)
at emitErrorNT (node:internal/streams/destroy:151:8)
at emitErrorCloseNT (node:internal/streams/destroy:116:3)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
type: 'system',
errno: 'ECONNRESET',
code: 'ECONNRESET'
}
}

Hi @SuMyatHlaing30 Thanks for opening this report! I am not able to repro this issue using API version 2024-08-01-preview and a resource in the Sweden Central region. Could you please share details about the resource and the API version you're using. I would also encourage you to open a support ticket. To open a support ticket, please refer to the guide here. In the meantime, please feel free to reach out if you have any other questions!

Hi @deyaaeldeen
I am using resource :

US Region
deployment = "gpt-4o"
API Version = "2024-07-01-preview"

When I upload file in Azure Openai Studio, showing file  size should be less than 200 MB.

@SuMyatHlaing30 I am not entirely sure what did you mean by "US Region". Please checkout the availability table in https://learn.microsoft.com/azure/ai-services/openai/concepts/models#assistants-preview for more information on what regions to use with what models.

are there any updates?


build(flux): update flux to v0.166.0

There is another PR open for a more recent version of Flux, that has fixes for the issues present in this PR.
https://github.com/influxdata/influxdb/pull/23348


Migrate samples frmo binance tools repo

https://github.com/tiagosiebler/binance-tools

Done already, but can make a topic for all the examples we should add to the awesome crypto examples


Overflow bug when running tf.linalg.diag on colab

Issue type
Bug
Have you reproduced the bug with TensorFlow Nightly?
No
Source
binary
TensorFlow version
2.13.0
Custom code
Yes
OS platform and distribution
No response
Mobile device
No response
Python version
No response
Bazel version
No response
GCC/compiler version
No response
CUDA/cuDNN version
No response
GPU model and memory
No response
Current behavior?
Due to large integer list element
Standalone code to reproduce the issue
import tensorflow as tf
import os
import numpy as np
try:
  diagonal_0_0_0_0 = 1111
  diagonal_0_0_0_1 = 1112
  diagonal_0_0_0 = [diagonal_0_0_0_0,diagonal_0_0_0_1,]
  diagonal_0_0_1_0 = 1121
  diagonal_0_0_1_1 = 1122
  diagonal_0_0_1 = [diagonal_0_0_1_0,diagonal_0_0_1_1,]
  diagonal_0_0 = [diagonal_0_0_0,diagonal_0_0_1,]
  diagonal_0_1_0_0 = 1211
  diagonal_0_1_0_1 = 1212
  diagonal_0_1_0 = [diagonal_0_1_0_0,diagonal_0_1_0_1,]
  diagonal_0_1_1_0 = 1221
  diagonal_0_1_1_1 = 1222
  diagonal_0_1_1 = [diagonal_0_1_1_0,diagonal_0_1_1_1,]
  diagonal_0_1 = [diagonal_0_1_0,diagonal_0_1_1,]
  diagonal_0 = [diagonal_0_0,diagonal_0_1,]
  diagonal_1_0_0_0 = 2111
  diagonal_1_0_0_1 = 2112
  diagonal_1_0_0 = [diagonal_1_0_0_0,diagonal_1_0_0_1,]
  diagonal_1_0_1_0 = 2121
  diagonal_1_0_1_1 = 2122
  diagonal_1_0_1 = [diagonal_1_0_1_0,diagonal_1_0_1_1,]
  diagonal_1_0 = [diagonal_1_0_0,diagonal_1_0_1,]
  diagonal_1_1_0_0 = 2211
  diagonal_1_1_0_1 = 2212
  diagonal_1_1_0 = [diagonal_1_1_0_0,diagonal_1_1_0_1,]
  diagonal_1_1_1_0 = 2221
  diagonal_1_1_1_1 = 2222
  diagonal_1_1_1 = [diagonal_1_1_1_0,diagonal_1_1_1_1,]
  diagonal_1_1 = [diagonal_1_1_0,diagonal_1_1_1,]
  diagonal_1 = [diagonal_1_0,diagonal_1_1,]
  diagonal = [diagonal_0,diagonal_1,]
  name = "None"
  k = -92233720368
  padding_value = 0
  align = "RIGHT_LEFT"
  out = tf.linalg.diag(diagonal=diagonal,name=name,k=k,padding_value=padding_value,align=align,)
except Exception as e:
  print("Error:"+str(e))

### Relevant log output

```shell
Error:{{function_node __wrapped__MatrixDiagV3_device_/job:localhost/replica:0/task:0/device:CPU:0}} Encountered overflow when multiplying<PHONE_NUMBER>2 with<PHONE_NUMBER>, result: -1 [Op:MatrixDiagV3]

Not an issue, that's an error that is being returned to the user instead of performing bad computations.
Please consult again the SECURITY.md guide, please don't spam with low quality issues.


LC0068 - False Positive - Record Identical to Page Record

We have an API Page for the Dimension Value (SourceTable = "Dimension Value").
In this API Page we read the parent Dimension Value (we search for a previous Begin-Total).
The FindLast results in a warning.
In my opinion this is a false positive, since we are in the context of this table and the user already has enough permissions to "open" the API Page.
I think at least a read should always be possible on pages that have the same sourcetable as the record variable.

Agree, thanks!

Please retest with the most recent pre-release version

Working now as expected


updated tx.proto

closes #773
Signed-off-by: nitin mewar<EMAIL_ADDRESS>

closing as #775 merged


Reintroduce the uhfqa.qa_delay function as uhfqa.qas.adjusted_delay

With the refactoring the function qa_delay was removed from the UHFQA driver.
This function is not really part of the sequencer generator, but mostly a patch of the behavior of the /DEV…./QAS/n/DELAY node to make it more reasonable. (ideally that would be part of the firmware, but that is not possible because of multiple reasons.
Since the delay node is part of the qas node we should add it there.

@YakBizzarro I still need to write tests for it but wanted to get you in feedback first

LGTM, thanks!


SmartLink should propagate c++ detection along library nodes

Describe the Feature
SmartLink should propagate c++ detection along library nodes
Required information

Link to SCons Users thread discussing your issue.
https://discordapp.com/channels/571796279483564041/612459756246007829/715705725804085329

Version of SCons
3.1.2

Version of Python
3.6

Additional context
If you have a .Program that is made up only of C source files, and it depends on a StaticLibrary node that is made of C++ files (but, say, exports a C ABI like a malloc replacement), then SmartLink will decide to use the C compiler to drive the link of the Program. This will fail though, because the necessary C++ runtime bits weren't included to satisfy the needs of the C++ compiled files.
This only affects projects which build .Programs out of StaticLibrary nodes. However, that is a fairly common setup, so probably worth fixing.

That thread isn't something publicly viewable. Unfortunately.
It'd be sufficient to just say bdbaddog asked on discord for me to file this issue.


Publish 7.4.2

Publish 7.4.2

@bot-gradle test and merge

OK, I've already triggered a build for you.

OK, I've already triggered a build for you.

OK, I've already triggered a build for you.


Implement/759 upgrade settings commandlet

Fixes: #759
Implements:

all devon.properties are renamed to ide.properties. In case both files exists in the same directory, devon.properties gets merged into ide.properties. ide.properties has priority here.
duplicated variables in the same ide.properties file are being removed
If variable DEVON_IDE_CUSTOM_TOOLS exists, it is converted into a custom-tools.json file in settings
if settings/devon folder exists, it is renamed to settings/templates, if settings/projects folder exists, it is renamed to settings/repositories
recursively traverses all files in settings/workspace and settings/*/workspace and replacse all known IdeVariables from CURLY into ANGLED syntax and legacy variable names are replaced with their new names.
for all .xml files that do not contain the XML namespace of our merger,  we print a warning with the filepath saying that this file is most-likely outdated and print a link to a documentation page about this settings migration feature
documentation page with small instructions for those who migrate from devonfw-ide to IDEasy
if a replacement-patterns.properties is found in a workspace folder a warning will be logged if the file is not empty and the file will be deleted.
if settings/ide.properties does not contain IDE_VARIABLE_SYNTAX_LEGACY_SUPPORT_ENABLED=false it will be added.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>4
Details

0 of 0   changed or added relevant lines in 0 files are covered.
26 unchanged lines in 3 files lost coverage.
Overall coverage decreased (-1.5%) to 65.905%

Files with Coverage Reduction
New Missed Lines
%

com/devonfw/tools/ide/commandlet/RepositoryCommandlet.java
3
80.49%

com/devonfw/tools/ide/commandlet/CommandletManagerImpl.java
3
90.63%

com/devonfw/tools/ide/io/FileAccessImpl.java
20
62.65%

Totals

Change from base Build<PHONE_NUMBER>7:
-1.5%

Covered Lines:
6521

Relevant Lines:
9542

💛  - Coveralls