youtube-dl is downloading altered video/mp4 files

https://github.com/rg3/youtube-dl/issues/6251 seems to be an issue again.
When I run youtube-dl with --fixup never I still get files with the incorrect size. For example, I run:
youtube-dl --fixup never -f 134 -o video.mp4 https://www.youtube.com/embed/dTDYoJZyXuc
stat video.mp4
I see that the file size is 40827280, but the file size should be 40829316 according to YouTube video web requests and the YouTube manifest.
What can I do to get the unaltered video files from YouTube?

Have you tried to download the specific format manually, e.g. with using youtube-dl -f and the desired format id?
Because --fixup is an option for post-processing, it shouldn't interfere with the download itself.
Or maybe try --keep-video

-f 134 is served in segments. DASH manifest contains gaps:
<Initialization sourceURL="range/0-708"/>
<SegmentURL media="range/2745-161548"/>

If you subtract two boundary values you'll get the difference you have.

Yes that's what I've been trying. And I tried running with --keep-video,
but with the same result. I might just have to modify the source to get the
correct file sizes. Thanks for the response!
On Thu, Dec 3, 2015 at 6:37 AM, FGS<EMAIL_ADDRESS>wrote:

Have you tried to download the specific format manually, e.g. with using youtube-dl
-f and the desired format id?
Because --fixup is an option for post-processing, it shouldn't interfere
with the download itself.
Or maybe try --keep-video
—
Reply to this email directly or view it on GitHub
https://github.com/rg3/youtube-dl/issues/7737#issuecomment-161658092.

But YouTube still expects the file sizes to include those gaps, meaning that youtube-dl is not downloading the same files stored on the YouTube server. This bug will affect byte range requests (the offsets into the file) and file requests (which need accurate file sizes) creating issues for users that need the actual YouTube files.
Are there any options I can use to download the actual YouTube files even with the bytes that are not included in the mpd segment list?

It is incorrect not to download the byte range from 709-2744. Youtube-dl is not downloading those bytes leading to altered files as I mentioned above! That's a huge issue for those of us that rely on accurate downloads from youtube-dl.

Note that, at least currently, the file has the correct size:
$ cat Sintel\ Silent\ Video-dTDYoJZyXuc.info.json | jq .filesize
32298722
$ stat Sintel\ Silent\ Video-dTDYoJZyXuc.mp4
  File: 'Sintel Silent Video-dTDYoJZyXuc.mp4'
  Size: 32298722  	Blocks: 63088      IO Block: 4096   regular file
Device: 1000002h/16777218d	Inode: 4355654     Links: 1
Access: (0644/-rw-r--r--)  Uid: (  501/jaimemf93)   Gid: (   20/   staff)
Access: 2015-12-07 10:51:01.000000000 +0100
Modify: 2015-12-07 09:54:52.000000000 +0100
Change: 2015-12-07 10:51:01.000000000 +0100
 Birth: 2015-12-07 09:54:52.000000000 +0100

It is incorrect not to download the byte range from 709-2744

If Youtube doesn't list that range I don't think we should download it (it could be unavailable), if they listed it we would definitely download it.

Yes that is lucky! The YouTube MPD is now different than last time I checked!
When the MPD is in this format as it is now:
</BaseURL?

youtube-dl handles the download correctly. However, when the segment list is given explicitly, as was happening before, youtube-dl was missing the indexRange="709-2744". With this new MPD, you can see that youtube-dl was missing important bytes from the YouTube video files and handling the earlier MPD format incorrectly by omitting the index.
It seems that the YouTube MPD are not always complete in terms of information given. If there is a gap in the YouTube MPD segment list, the file should still be downloaded with the correct file size and byte range offsets!


site.author.name displays on top of page?

i understand it appears in the footer (includes), but i don’t see it in the head ‘n header files... i feel like it’s a debug output? no clue.

# config.yml
author:
  name: a willowolf of rathewolf
  email: '' # error without it?

note: the picture has an edited footer, without the ‘| escape’, so that my links now work. But the problem occurs with or without that.

@Rahil627 Providing us a link to your site's repository would help diagnosing the cause.

Nevermind. Found the repository via your screenshot.
The reason behind this is the HTML in site.author.name which is used under the hood in {% seo %} as well.
Since you're using a custom theme, it'd better altering the theme to your taste.
As of now, we don't intend on supporting HTML in site.author.name

oh wow, thanks so much for the investigation. I was going to reply with the repo after i cleaned out some old tests... but, then i ran into another problem: i can’t fork my own repo! 😩😂 sorry about that!
the theme is just minima, although i did edit a few lines...
okay, i’ll look out for the seo

@rahil627 I think you misunderstood.
It will be difficult for you to alter the output of {% seo %}. The easier way would be to avoid using HTML in site.author.name.

ohhh right, now i understand. Yeah, i’ll avoid hacking the author.name altogether... I only did it as a way to avoid using a custom theme... i’ve switched over now.


NYX - prevent/speed up file counting by Pilatus AreaDetector

new description:
AreaDetector counts all images collected in the current collection. The previous description was based on the incorrect assumption that it would count all images collected since the last reset.
The task is to speed things up at the end of a collection - either speed up the checking or disable it altogether.
below is original description:
AreaDetector counts all images collected since the last reset of the counter upon the completion of a data collection. This can take a long time when many collections have been performed.
This counter should be reset after every collection so that the number of images counted is only for the last collection.
PV is: XF:19IDC-ES{Det:Pil6M}cam1:ArrayCounter
NOTE: check whether this has already been done in LSDC, but just had not been implemented correctly in the flyer

From memory, this results in annoying waits at the end of collection before control is returned to the user, thus only labelled as "medium". Testing and feedback from Kevin will be useful to determine whether this should be increased to high.

Assuming this hasn't been done in LSDC, would you prefer it to be done there, in the flyer, or at the device level?

I don't have a strong opinion, but since image counting isn't used for the Eiger, it should be somewhere it is easy to remove when we change detectors. and as a data acquisition person, I would lean towards flyer or LSDC

we don't actually care about the counter being reset, and the counting only increments up based on the files created by the current collection.
also moving to low priority

no longer using this detector. closing.


Problems with running FAST.Farm

No matter how I change NY_Low and Y0_Low, it always reports the error Farm_Initialize:AWAE_Init:AWAE_UpdateStates:InflowWind_CalcOutput:CalculateOutput:IfW_FlowField_Ge
tVelAcc:Grid3DField_GetCell: Violation of GF wind array boundary: Y-direction grid is too small.
Y=510; Y boundary = [-500, 500]
InflowWind_CalcOutput:InflowWind_GetSpatialAverage:CalculateOutput:IfW_FlowField_GetVelAcc:Grid3DF
IfW_FlowField_GetCell: Violation of GF wind array boundary: Y-direction grid is too small. y=500.82; Y
Boundary = [-500, 500]
The values reported as errors also stay the same, why is this?
Your answers are much appreciated！

Dear @Yongbyon,
It looks like your TurbSim grid extends from Y= -500 to 500 m.  However, both your low-resolution domain and your high-resolution domains for turbines 6 and 8 have Y dimensions that extend beyond 500 m.  The TurbSim grid must fully cover the FAST.Farm domains.
FYI: Given the complexity of the FAST.Farm modeling guidance for ambient inflow generation and its compatibility with the low- and high-resolution domains of FAST.Farm (https://openfast.readthedocs.io/en/main/source/user/fast.farm/index.html), we generally recommend using the python scripts available in the OpenFAST_Toolbox (https://github.com/OpenFAST/openfast_toolbox/tree/main/openfast_toolbox/fastfarm) when setting up the ambient wind inflow and low- and high-resolution domains used by FAST.Farm.
Best regards,


[Bug] A bug in the initialize_ub function.

I find that TransformerEngine has supported the tensor parallelism (TP) communication overlap without the dependency of MPI. Therefore, I tried to use the tensor parallelism (TP) communication overlap in torchrun method. In this process, I found a bug in the initialize_ub function.
I ran my code but get the error as follows.
[rank1]:   File "/usr/local/lib/python3.10/dist-packages/transformer_engine/pytorch/module/base.py", line 149, in initialize_ub
[rank1]:     raise OSError(f"Invalid network interface: {ifname}") from err
[rank1]: OSError: Invalid network interface: eth
[rank7]: Traceback (most recent call last):
[rank7]:   File "/usr/local/lib/python3.10/dist-packages/transformer_engine/pytorch/module/base.py", line 144, in initialize_ub
[rank7]:     fcntl.ioctl(
[rank7]: OSError: [Errno 19] No such device

I found the code as follows.
https://github.com/NVIDIA/TransformerEngine/blob/bdea56fc023014eaf52a171047b641b3bfdded70/transformer_engine/pytorch/module/base.py#L130-L149
The function tries to get the IP address as the hostname if the ifname is not None. And the ifname is obtained from the environment variables NVTE_UB_SOCKET_IFNAME, NCCL_SOCKET_IFNAME, and GLOO_SOCKET_IFNAME.
In my environment, the NCCL_SOCKET_IFNAME is set to be eth. In fact, the machine has many network cards named eth0, eth1, eth2, and so on. But there is not a network card called eth. I try to use ifname = eth0 and the code above can run successfully.
I checked the NCCL documentation about the environment variables. It shows that when the value of NCCL_SOCKET_IFNAME is eth, it will use all interfaces starting with eth, e.g. eth0, eth1.
So I think the code need to use other methods, such as psutil or netifaces, to get the name of the network cards when NCCL_SOCKET_IFNAME is eth.
This is my test code:
import socket, fcntl, struct, os
ifname = os.getenv("NCCL_SOCKET_IFNAME")
ifname = 'eth0'
print('ifname:', ifname)
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
print('fileno:', s.fileno())
inet = fcntl.ioctl(s.fileno(), 0x8915, struct.pack("256s", ifname[:15].encode("UTF-8")))
print('inet:', inet)
hostname = socket.inet_ntoa(inet[20:24])
print('hostname:', hostname)
s.close()

@denera Could you take a look at it?

@wangzihe1996 socket.if_nameindex() in the standard Python distribution already provides a list of network interfaces without relying on external dependencies like psutil or netifaces. However, cluster configurations vary significantly across the user base and there is no way for us to know which of these listed network interfaces is the right one to use when trying to detect ranks on the same physical node.
That's why we set up the NVTE_UB_SOCKET_IFNAME variable to override both NCCL_SOCKET_IFNAME and GLOO_SOCKET_IFNAME. You can simply set NVTE_UB_SOCKET_IFNAME=eth0 to run your code without modifying anything in TE, and it will use ifname = 'eth0' to fetch hostname information on every rank.
That said, I will nonetheless add a check to make sure the ifname we detect from any of the 3 environment variables is one of the valid interfaces we can see from socket.if_nameindex(). That way, we can at least show the user a meaningful warning message and fall back on socket.gethostname() (which unfortunately fails on managed cluster services like Kubernetes).

@denera I'm glad you have made these changes which can give more informations to the developers such as me. Meanwhile, I have another question that which one should I choose if I have many RDMA network cards. We can assume that they are named eth0, eth1, ..., eth7 and each of them has an individual IPV4 address.

@wangzihe1996 The correct network interface is the one that returns the same hostname on processes/ranks that map to GPUs on the same physical node.
We can't make any general assumptions or guesses about this in TE, but your specific case looks like each node connecting to different groups of nodes via different RDMA network interfaces. If that's true, then any one of the eth interfaces will likely return the correct hostname information and it won't matter which you select.

@denera Thank you for your reply. I understand more details of this.


Re-run ansible playbook after first run breaks

- name: Add token to config.yaml
  ansible.builtin.lineinfile:
    dest: /tmp/ansible-config.txt
    line: "{{ original_token }}"
    state: present
    insertbefore: BOF
  when: previous_rke2_config.stat.exists and original_token | length > 0
  changed_when: false

 - name: Add server url to config file
  ansible.builtin.lineinfile:
    dest: /tmp/ansible-config.txt
    line: "{{ original_server }}"
    state: present
    insertbefore: BOF
  when: previous_rke2_config.stat.exists and original_server | length > 0
  changed_when: false

The following issue can be reproduced by running the playbook then updating the hosts.ini with another node and running the playbook again. The original_token and original_server vars are defined as null on the first server since it does not have the token and server address set in the /etc/rancher/rke2/config.yaml. Ansible throws an error because you can't get the length of  null. Remove length in both those tasks, and by default, the when statement will result as false since those vars return null. No need to get  length in that statement.

Can't create issue on latest branch and code has been modified from your post. Please reopen if you're still have issues.


Temporary failure in name resolution

Hi, I just wanted to try the tool because I think it is a really great idea but encountered a problem that I can't figure out. I have staged some changes, set my API key and executed gptcommit install in the repository. But when I enter git commit I get the following output:
Error: error sending request for url (https://api.openai.com/v1/completions): error trying to connect: dns error: failed to lookup address information: Temporary failure in name resolution

Caused by:
    0: error trying to connect: dns error: failed to lookup address information: Temporary failure in name resolution
    1: dns error: failed to lookup address information: Temporary failure in name resolution
    2: failed to lookup address information: Temporary failure in name resolution

When I try to make a request to the completions API using curl, it works.
Thanks in advance!

Thanks for reporting the issue!
What happens when you re-run the command with RUST_LOG=trace?

Hi, thanks for the fast reply!
This results in the following output (Bearer key is replaced with "sk-..."):
2023-02-02T08:34:56.286Z DEBUG [gptcommit::settings] Using home dir at /home/tfr
2023-02-02T08:34:56.286Z DEBUG [gptcommit::settings] Found config dir at /home/tfr/.config/gptcommit
2023-02-02T08:34:56.287Z DEBUG [gptcommit::settings] Applying config file at /home/tfr/.config/gptcommit/config.toml
2023-02-02T08:34:56.287Z DEBUG [gptcommit::settings] Applying config from  GPTCOMMIT__*: Ok({})
2023-02-02T08:34:56.287Z DEBUG [gptcommit] Settings: Settings { model_provider: Some(OpenAI), openai: Some(OpenAISettings { api_key: Some("sk-..."), model: Some("text-davinci-003") }), prompt: Some(PromptSettings { file_diff: Some("You are an expert programmer, and you are trying to summarize a git diff.\nReminders about the git diff format:\nFor every file, there are a few metadata lines, like (for example):\n```\ndiff --git a/lib/index.js b/lib/index.js\nindex aadf691..bfef603 100644\n--- a/lib/index.js\n+++ b/lib/index.js\n```\nThis means that `lib/index.js` was modified in this commit. Note that this is only an example.\nThen there is a specifier of the lines that were modified.\nA line starting with `+` means it was added.\nA line that starting with `-` means that line was deleted.\nA line that starts with neither `+` nor `-` is code given for context and better understanding.\nIt is not part of the diff.\nAfter the git diff of the first file, there will be an empty line, and then the git diff of the next file.\n\nDo not include the file name as another part of the comment.\nDo not use the characters `[` or `]` in the summary.\nWrite every summary comment in a new line.\nComments should be in a bullet point list, each line starting with a `-`.\nThe summary should not include comments copied from the code.\nThe output should be easily readable. When in doubt, write less comments and not more. Do not output comments that simply repeat the contents of the file.\nReadability is top priority. Write only the most important comments about the diff.\n\nEXAMPLE SUMMARY COMMENTS:\n```\n- Raise the amount of returned recordings from `10` to `100`\n- Fix a typo in the github action name\n- Move the `octokit` initialization to a separate file\n- Add an OpenAI API for completions\n- Lower numeric tolerance for test files\n- Add 2 tests for the inclusive string split function\n```\nMost commits will have less comments than this examples list.\nThe last comment does not include the file names,\nbecause there were more than two relevant files in the hypothetical commit.\nDo not include parts of the example in your summary.\nIt is given only as an example of appropriate comments.\n\n\nTHE GIT DIFF TO BE SUMMARIZED:\n```\n<FILE_DIFF>\n```\n\nTHE SUMMARY:\n\n"), commit_summary: Some("You are an expert programmer, and you are trying to summarize a pull request.\nYou went over every file that was changed in it.\nFor some of these files changes where too big and were omitted in the files diff summary.\nPlease summarize the pull request.\nWrite your response in bullet points, using the imperative tense following the pull request style guide.\nStarting each bullet point with a `-`.\nWrite a high level description. Do not repeat the commit summaries or the file summaries.\nWrite the most important bullet points. The list should not be more than a few bullet points.\n\nTHE FILE SUMMARIES:\n```\n<SUMMARY_POINTS>\n```\n\nRemember to write only the most important points and do not write more than a few bullet points.\nTHE PULL REQUEST SUMMARY:\n\n"), commit_title: Some("You are an expert programmer, and you are trying to title a pull request.\nYou went over every file that was changed in it.\nFor some of these files changes where too big and were omitted in the files diff summary.\nPlease summarize the pull request into a single specific theme.\nWrite your response using the imperative tense following the kernel git commit style guide.\nWrite a high level title.\nDo not repeat the commit summaries or the file summaries.\nDo not list individual changes in the title.\n\nEXAMPLE SUMMARY COMMENTS:\n```\nRaise the amount of returned recordings\nSwitch to internal API for completions\nLower numeric tolerance for test files\nSchedule all GitHub actions on all OSs\n```\n\nTHE FILE SUMMARIES:\n```\n<SUMMARY_POINTS>\n```\n\nRemember to write only one line, no more than 50 characters.\nTHE PULL REQUEST TITLE:\n\n") }), allow_amend: Some(false) }
🤖 Asking GPT-3 to summarize diffs...
2023-02-02T08:34:56.398Z DEBUG [gptcommit::summarize] summarizing file: sortingutils.cpp
2023-02-02T08:34:56.398Z DEBUG [gptcommit::summarize] summarizing file: sortingutils.h
2023-02-02T08:34:56.398Z DEBUG [gptcommit::openai] Sending request to OpenAI:
RequestBuilder { method: POST, url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("api.openai.com")), port: None, path: "/v1/completions", query: None, fragment: None }, headers: {"content-type": "application/json", "authorization": "Bearer sk-..."} }
2023-02-02T08:34:56.398Z DEBUG [gptcommit::openai] Sending request to OpenAI:
RequestBuilder { method: POST, url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("api.openai.com")), port: None, path: "/v1/completions", query: None, fragment: None }, headers: {"content-type": "application/json", "authorization": "Bearer sk-..."} }
2023-02-02T08:34:56.399Z DEBUG [reqwest::connect] starting new connection: https://api.openai.com/
2023-02-02T08:34:56.399Z DEBUG [reqwest::connect] starting new connection: https://api.openai.com/
2023-02-02T08:34:56.401Z DEBUG [gptcommit::openai] Sending request to OpenAI:
RequestBuilder { method: POST, url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("api.openai.com")), port: None, path: "/v1/completions", query: None, fragment: None }, headers: {"content-type": "application/json", "authorization": "Bearer sk-..."} }
2023-02-02T08:34:56.401Z DEBUG [reqwest::connect] starting new connection: https://api.openai.com/
2023-02-02T08:34:56.401Z DEBUG [gptcommit::openai] Sending request to OpenAI:
RequestBuilder { method: POST, url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("api.openai.com")), port: None, path: "/v1/completions", query: None, fragment: None }, headers: {"content-type": "application/json", "authorization": "Bearer sk-..."} }
2023-02-02T08:34:56.401Z DEBUG [reqwest::connect] starting new connection: https://api.openai.com/
Error: error sending request for url (https://api.openai.com/v1/completions): error trying to connect: dns error: failed to lookup address information: Temporary failure in name resolution

Caused by:
    0: error trying to connect: dns error: failed to lookup address information: Temporary failure in name resolution
    1: dns error: failed to lookup address information: Temporary failure in name resolution
    2: failed to lookup address information: Temporary failure in name resolution

Hmm, I'm not sure what the source of the error is here.
It could be something related to a rust library. I can't reproduce it from my linux virtual machine.
Let me know if you find any other clues here

Closing as we haven't heard a response from OP


CDS-80424 TAS artifact bundle

Harness Developer Pull Request
Thanks for helping us make the Developer Hub better. The PR will be looked at
by the CODEOWNERS.
What Type of PR is This?

[ x] Issue
[ ] Feature
[ ] Maintenance/Chore

If tied to an Issue, list the Issue(s) here:

https://harness.atlassian.net/browse/CDS-80424

House Keeping
Some items to keep track of. Screen shots of changes are optional but would help the maintainers review quicker.

[ ] Tested Locally
[ ] Optional Screen Shoot.

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://657cdc32b0fbca47c67b4e0c--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://657cdd871ade91455246e15c--harness-developer.netlify.app


fix soil/en/hydr/biogeo test

Purpose
closes #354

[x] I have read and checked the items on the review checklist.

bors r+


How to use acorn-jsx with tern?

I tell me if we could use acorn-jsx with tern. @RReverser @marijnh have you an idea?

Have a Tern plugin load the Acorn plugin and add a preParse hook that enables it in the parsing options. And also somehow patch Tern to now choke on expression nodes that it doesn't know about.

I will study how to do that when I will have time. Thanks @marijnh

Thanks @marijnh for your info.
@RReverser @marijnh I have started the POC with tern react plugin at https://github.com/angelozerr/tern-react and now I can initialize acorn with acorn-jsx https://github.com/angelozerr/tern-react/blob/master/react.js#L18
Now I have a problem with acorn walk. When I try to open completion. When type is JXSElement, the code base[type] (see https://github.com/marijnh/acorn/blob/master/src/walk/index.js#L23) is null.
So my question is how to customize acorn walk with JSXElement (like https://github.com/chtefi/acorn-jsx-walk have done)

So my question is how to customize acorn walk with JSXElement (like https://github.com/chtefi/acorn-jsx-walk have done). I have created this issue at acorn marijnh/acorn#329

Forget my comment, it was easy to do. Now I have a similar problem when walk.recursive is called. After debugging scopeGatherer fom infer.js doesn( define JSXElement.
Is it possible to customize scopeGatherer and inferWrapper?

Attached patch exports some objects that you might want to hook into from the infer module. Note that this may still not be very easy to work with, and I am not committing to keeping all the internal interfaces involved stable, but it might at least enable you to hack up some basic solution to this problem.

Thanks @marijnh! I will study that. I tell me just if I could extend scopeGatherer without copying/pasting your code.

@marijnh I have tried your patch and to use it I had modify your code by adding exports in https://github.com/marijnh/tern/blob/master/lib/infer.js#L1709
walk.recursive(ast, scope, null, scopeGatherer);
if (cx.parent) cx.parent.signal("preInfer", ast, scope)
walk.recursive(ast, scope, null, inferWrapper);
and https://github.com/marijnh/tern/blob/master/lib/infer.js#L995

Neither of those lines point to something that could be exported, so I have no idea what you mean.

If you see https://github.com/angelozerr/tern-react/blob/master/react.js#L46 I have added JSXElement in the scopeGatherer walk (otherwise tern will throw an error) like this:
infer.scopeGatherer = walk.make({
        JSXElement: function(node, scopes, c) {
          console.log(node)
        }
      }, infer.scopeGatherer);

in other words, I need to override existing scopeGatherer to add JSXElement.
Your code:
walk.recursive(ast, scope, null, scopeGatherer);

doesn't use my overrided scopeGatherer
If I do :
walk.recursive(ast, scope, null, exports.scopeGatherer);

It use my overrided scopeGatherer

Why aren't you just adding new properties to the existing object?

Why aren't you just adding new properties to the existing object?

Yes you are right! Why I have not think about that? Sorry for the noise. I will change my code.


logger: Properly accommodate IPv6 addresses in LOG_CONNEVENTS logs

Unfortunately, I introduced a bug in #790 that I didn't catch at the time since I was primarily testing with IPv4.
When the remote peer is connected over IPv6, the cast (struct sockaddr *) &c->request_addr truncates the struct which causes the inet_ntop output for rip to be incorrect. The proposed fix is to retain the struct sockaddr_in6 type defined here. c->request_addr_size abstracts away the size already (presumably to take care of IPv4/IPv6 differences).
Before this change:
IPv4:
$ nc -4 <IP_ADDRESS> 11211

ts=1629305951.994024 gid=6 type=conn_new rip=<IP_ADDRESS> rport=40534 transport=tcp cfd=29
ts=1629306074.986838 gid=7 type=conn_close rip=<IP_ADDRESS> rport=40534 transport=tcp reason=normal cfd=29

IPv6:
# hello from san francisco
$ nc -6 2604:5500:c382:f300:5cf1:bb25:1001:5964 11211

# note the truncated address
ts=1629306088.631154 gid=8 type=conn_new rip=2604:5500:c382:f300:: rport=57106 transport=tcp cfd=29
ts=1629306091.873913 gid=9 type=conn_close rip=2604:5500:c382:f300:: rport=57106 transport=tcp reason=normal cfd=29

Unix:
$ nc -U /tmp/sock

ts=1629307760.919189 gid=2 type=conn_new rip=unix rport=0 transport=local cfd=28
ts=1629307761.993160 gid=3 type=conn_close rip=unix rport=0 transport=local reason=normal cfd=28

After this change:
IPv4:
$ nc -4 <IP_ADDRESS> 11211

ts=1629307902.903407 gid=2 type=conn_new rip=<IP_ADDRESS> rport=40904 transport=tcp cfd=29
ts=1629307903.917884 gid=3 type=conn_close rip=<IP_ADDRESS> rport=40904 transport=tcp reason=normal cfd=29

IPv6:
$ nc -6 2604:5500:c382:f300:5cf1:bb25:1001:5964 11211

ts=1629307917.645603 gid=4 type=conn_new rip=2604:5500:c382:f300:5cf1:bb25:1001:5964 rport=57448 transport=tcp cfd=29
ts=1629307919.58216 gid=5 type=conn_close rip=2604:5500:c382:f300:5cf1:bb25:1001:5964 rport=57448 transport=tcp reason=normal cfd=29

Unix:
ts=1629307944.742783 gid=2 type=conn_new rip=unix rport=0 transport=local cfd=28
ts=1629307945.634618 gid=3 type=conn_close rip=unix rport=0 transport=local reason=normal cfd=28

thanks!

Looks fine. I'll get this merged up before next release.
Btw, I probably should've pinged you since you've been in the logging code; any complaints around https://github.com/memcached/memcached/pull/811 ? Still not completely happy with the code but I hope it's a step in the right direction.

Thanks! I did actually make a brief pass through #811 ~last week, it looks pretty good to me. Nice to remove some of the redundant boilerplate, I don't have any major concerns.


 This design never considers how to use a new scheme that have different keys compared to original one, in which case the BaseVehicle would never ever change its reward scheme and therefore raise an error since the keys of the reward scheme is mismatched.
I am going to refactor the, at least, cost / reward scheme processing!

Introducing an extra verify config function after generating the config!


Is it possible to train PICK-pytorch to detect table and all its line-items?

Hi wenwenyu, thank you for an amazing code.
I have been experimenting around with the code, and found out the training dataset can be adjusted for extractions of different informations.
However, there's one thing I am stuck on, and that is training the model to detect tables and its contents as well.
I want the customized model to be able to not only detect the header data, but also list all the table line-item.
From what I understand of the code, it seems possible to train it to detect table contents as well, but I don't know how I should set the training data's entities/labels, especially for tables with more than one line-items.
Any help or tip would be greatly apppreciated, thanks :)

it would be better to check a model for detecting table shape, and then you can parse the content and arrange it, I think the PICK model would be more efficient when you want to extract unstructured data and not tabular ones.


Repo sync for protected CLA branch

The pull request is created from main638071524927525746sync_temp to main to fix git push error for protected CLA branch

Learn Build status updates of commit 1da913b:
:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the learn.microsoft.com contributor guidesPost your question in the Learn support channel


DPL-802 Write up an SOP for the Bioscan tag plate generation process

User story
As R&D and PSD we would like to document the processes for creating new versions of Bioscan tag layout templates, and for creating instance sets of tag plates. This document would be used by both sides as a reference when new templates and plates are needed.
Who are the primary contacts for this story
Emma B
Scott T
Andrew S
Eduardo MR
Abdullah Y
Who is the nominated tester for UAT
Scott T for R&D and Eduardo MR for PSD
Acceptance criteria
To be considered successful the solution must allow:

[ ] document created and shared, with all parties agreeing on content
[ ] document lists steps required from all parties involved to create:

[ ] a new version of the tag layout templates e.g. when tags are changed by R&D
[ ] a set of tag plate instances from a specific version of the tag plate templates for use in the Lab

Dependencies
This story is blocked by the following dependencies:
None
References
This story has a non-blocking relationship with:
None
Additional context
This story came out of UAT discussion meeting on 27th June 2023.
Aim is to clearly define steps required so that nothing is missed.

Link to draft google doc:
https://docs.google.com/document/d/1EK0dYo_wufDAKDIOdKSIkc51onsBrUO96pHDyJj_DVw/edit?usp=sharing

Will transfer doc into Confluence once approved.


Minor API change

The neural net I'm using torch2jax for uses torch.autograd to compute gradients internally. To get torch2jax working, I had to delete the line with torch.no_grad() in api.py (2 occurrences) and in gradients.py (1 occurrence).
Would it be possible to make the use of with torch.no_grad() optional at the user-level? I suppose the only reason to use it here is for speedup during output shape inference. Maybe this could be something like uses_gradients_internally with default value False.

Great catch! This is a bug, the no_grad option shouldn't be there at all. I've just updated it on the multi-gpu branch!

Thanks a lot for the quick response, closing this now.

This is incorporated now!


provide instructions on how to setup ADS TOKEN

[moved from https://github.com/SIMPLE-AstroDB/SIMPLE-db/issues/152]
For the ingest_publication function, an ADS TOKEN environment variable should be set. We should provide instructions on how to do that.
make account on ADS and get the token from ADS: https://ui.adsabs.harvard.edu/help/api/
Then add it via an environment variable in your .bashrc or whatever init file you have.
export ADS_TOKEN="<your token>"

Question: is this something we'd want in the README, or in the readthedocs documentation? if the latter, we should chat about how we want to structure the documentation in general.

I think this goes in the RTD.
If we need to talk about structuring the documentation, maybe open a new issue and ask some specific questions?

Done. https://astrodb-utils.readthedocs.io/en/latest/pages/ingesting_publications.html#set-up-ads-token


Request to resend: IL received five files around 12:00-1:30 PM of 8/17/2024 that missing PID-3-1

IL received five files (100 messages) from "MEDEX MEDICAL SUPPLI&14D2252539&CLIA" around 12:00-1:30 PM of 8/17/2024 that missing PID-3-1.  Therefore, these messages need to be resent with the PID-3-1 filled.

Done!


doc: rpminspect

Depends on #2

@LecrisUT And thanks for sending this!

@lachmanfrantisek this PR should be fine to be merged. The documented interface is the intended one and we can deal with the formatting when we link it with packit.dev documentation


Pocket Items Lost on Death, Ignoring Configuration

There is a equipment.Pockets configuration option in the lostondeath.json file.
All other configuration types appear to work fine.
How to reproduce:

Modify the equipment.Pockets configuration option in lostondeath.json to false.
Enter raid with items in pockets.
Die

Expectation
Items remain in pockets.
Reality
All items are removed from pockets.

Originally written by Refringe

Set Pockets configuration option in lostondeath.json to false.
Die.
Tested on this fresh profile:
FIR Dogtag despawned
FIR Weapon Mod (ME Cylinder 12ga muzzle) lost FIR status
NON FIR Grenade remain after death
NON FIR Meds remain after death
Tested on main profile:
FIR Rubles stack decreased in value (assume FIR part despawned)
FIR KEY despawned on death
FIR Grenade lost FIR status
FIR Ammo (Boxed) Lost FIR

Attachments

server-2024-03-13.log (143 KiB)
LogOutput.log (55 KiB)
2024.03.13_16-14-27_<IP_ADDRESS>.28965 traces.log (83 KiB)

Originally written by itsMickle


Slide Master, vertical alignment of SlideNumber element

let slideNumber = {
	color: FOOTER_FONT_COLOR,
	x: WIDE_SLIDE_WIDTH - SLIDE_NUMBER_WIDTH,
	y: SLIDE_FOOTER_TOP,
	w: SLIDE_NUMBER_WIDTH,
	h: SLIDE_NUMBER_HEIGHT,
	fontFace: SLIDE_FONT_FACE,
	fontSize: BODY_FONT_SIZE,
	valign: "bottom",
	align: "right",
};

I have the above for the slideNumber in my slide master definition. All the attributes work except for the valign attribute. The slide number is always "top" aligned. Cannot get it to align t the middle.
What am I missing? or could this be a bug in the slideNumber element?

@kramsram - thanks for reporting this.
The valign prop was not working. It'll be fixed in the next release.


Adding namespace declaration in Grafana PersistentVolumeClaim

When using the Helm chart with a user specific namespace and Grafana persistency
enabled, the generated PersistentVolumeClaim for Grafana was missing a namespace,
leading in the Grafana pod to be stuck in the Pending state.
Linked to #11303

@jfaixo Do you mind rebasing to try to fix the failed required test?

@jfaixo please fix the failing tests

New changes are detected. LGTM label has been removed.

@jfaixo: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e_pilotv2_auth_sds.sh
214d0400b9ec661385429250328d48d4cf1474d2
link
/test istio_auth_sds_e2e

prow/istio-pilot-multicluster-e2e.sh
214d0400b9ec661385429250328d48d4cf1474d2
link
/test istio-pilot-multicluster-e2e

prow/istio-integ-k8s-tests.sh
ce6d883682cbb165912d0b2fbc83e661edbb2f99
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@jfaixo: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
ce6d883682cbb165912d0b2fbc83e661edbb2f99
link
/test istio-integ-k8s-tests

prow/istio-pilot-multicluster-e2e.sh
ce6d883682cbb165912d0b2fbc83e661edbb2f99
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@linsun: changing LGTM is restricted to assignees, and assigning you to the PR failed.

In response to this:

/lgtm

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: jfaixo, linsun, ymesika
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

install/OWNERS [linsun,ymesika]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@jfaixo can you get the circle CI test pass?

CircleCI tests passed.
@wenchenglu can you please merge this into 1.1, per the tracking meeting earlier?


Unable to Connect to Plex - Hostname error

Hey folks,
Wasn't able to find anything similar to this issue - trying to configure plex intergration on my unraid server.
This is the log output after trying to test the connection to plex in htpc-manager:

2018-04-17 20:22:56
modules.plex
ERROR
Unable to contact Plex via https://<IP_ADDRESS>:32400 error hostname '<IP_ADDRESS>' doesn't match '*.04d70e2e89884751b75bae5446e4acde.plex.direct'

<IP_ADDRESS> is my plex/unraid local IP address (actual IP of unraid, Plex is running in host mode).
I've also attached the summary docker config for both.
Any ideas?

Did you ever figure out the issue? I'm having the same problem.
Thanks.

Just had this problem myself.
The 'issue' is that you've enabled SSL. The SSL certificate used is issued to hostname *.04d70e2e89884751b75bae5446e4acde.plex.direct, however you're attempting to validate it against the hostname <IP_ADDRESS>
Simply connect without SSL to solve this

Hi, I am having this same issue.
"Unable to contact Plex via http://<IP_ADDRESS>:32400 error No JSON object could be decoded"
AND
"Failed to get authtoken from plex 401 Client Error: Unauthorized"
The HTPC manager is running on my new Pi. My PMS server is on my Windows machine at the IP and port listed above. Any suggestions? Thank you


how can i get web client's ip address?

how can i get web client's ip address? thanks

duplicate #1053

https://github.com/ndonkoHenri/Flet-Samples/tree/master/IP Revealer


test: Run quality checks on all (and only) repo directories

Add quality coverage to tests directory (and fix lint issues there)
Prevent virtualenv from being checked, if it's in the working directory

Also:

Add issue link to github.in entry
Fix typo in check name (check_build_badge)
Rename Travis fixture defs to match their registered names (easier to check imports that way)

Description:
Describe in a couple of sentences what this PR adds
Merge checklist:

[x] Changelog record added
[x] Documentation updated (not only docstrings)
[x] Commits are squashed

Pushed up a couple changes. Let me know what you think.


Remove redefinitions of typedef in c.

The release.linux build fails with enabled ALL_IN_ONE option.

I got confused by the title and the comment of this PR when I took a look at the patch. All I saw was that static asserts were extended with an additional message-like argument. I'm still not sure why that's needed. What kind of typedefs get redefined?
Other than that, the patch seems to be a fair step-by-step rewrite of all static asserts, that's good. (If that's really needed.)
Finally, please make sure that all touched files have the copyright year(s) updated in their headers. (At least some of them contain 2015.)

The generic direction is good, but I also agree with Akos that some names need to be a bit better.

Thanks for the comments, I've updated the patch with the suggested names.

Patch looks much better.

Thanks, I've updated, and I've also removed the duplicated static asserts too.

Patch looks good, thanks. But still, could you please provide a description of the error it fixes? At least for future reference.

@akiss77 : The JERRY_STATIC_ASSERT defines a variable, where the name contains the LINE of the assert. So, in case of all-in-one build, it's possible that there are at least two variables which have the same name (redefinition), because an assert occurs in the same line in another file.

@robertsipka thanks, LGTM (FWIW)

LGTM

LGTM


Upgrade guide doesn't talk about the changes needing to configure kestrel.

https://docs.microsoft.com/en-us/aspnet/core/migration/22-to-30?view=aspnetcore-2.2&tabs=visual-studio#hostbuilder-replaces-webhostbuilder
            .ConfigureKestrel(c => {
                c.AddServerHeader = false;
                //c.AllowSynchronousIO = false; // TODO: Investigate issue with JSON Serialization.
            })

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 76f39eec-9dd1-6167-56ec-47b26e0b5063
Version Independent ID: f9f2b0f3-01eb-cf58-8767-194fe0fc0e44
Content: Migrate from ASP.NET Core 2.2 to 3.0 Preview
Content Source: aspnetcore/migration/22-to-30.md
Product: aspnet-core
Technology: aspnetcore-migration
GitHub Login: @tdykstra
Microsoft Alias: tdykstra

This doesn't cover other web things you might be doing like health checks outside of a startup class.

It also doesn't cover the following:
app.Use((context, func) => context.Response.WriteAsync($"Running Job: {jobOptions.JobName}"));
Error | CS7036 | There is no argument given that corresponds to the required formal parameter 'configure' of 'EndpointRoutingApplicationBuilderExtensions.UseEndpoints(IApplicationBuilder, Action<IEndpointRouteBuilder>)' | Exceptionless.Web | C:\Code\Exceptionless\src\Exceptionless.Web\Startup.cs | 197 | Active

I'll take care of this one right after I hear back from engineering on the Kestrel-to-3.0 updates. https://github.com/aspnet/AspNetCore.Docs/pull/13775

Note to self: https://github.com/aspnet/AspNetCore.Docs/pull/14190#issuecomment-529011785


Issue #2786 Amend SVG library to use XmlWriter instead of XmlTextWriter.

Reference Issue
Enhancement #693
What does this implement/fix? Explain your changes.
Generalize XML output to use an XmlWriter instead of the more specific XmlTextWriter. In particular, my use case is that I want to use my own custom XmlWriter to send output as part of an Asp.Net Blazor application.
The changes to the various WriteXXX(...) methods are trivial. However, the XmlWriter class performs some stricter checks than the XmlTextWriter and so I had to change the handling of namespaces. The existing code specifies namespace simply by writing an "xmlns" attribute. However, with XmlWriter, this causes an exception to be thrown (similar to this question on StackOverflow )
Note also the Microsoft documentation for XmlTextWriter:

Starting with the .NET Framework 2.0, we recommend that you use the XmlWriter class instead.

Starting with the .NET Framework 2.0, we recommend that you create XmlWriter instances by using the XmlWriter.Create method and the XmlWriterSettings class to take advantage of new functionality.

See also conformance options based on ConformanceLevel
Any other comments?
The changes impacted SvgAttributeAttribute.cs such that:

the SVG namespace (http://www.w3.org/2000/svg) is no longer included in the list of namespaces that would be written into the root SvgFragment element from this list. Instead, all SvgElements are explicitly using this namespace as per SvgElement.cs
the 'NamespaceAndName' method is no longer needed. Instead, XmlWriter.WriteAttributeString is called with the namespace and prefix to handle this. (again, in SvgElement.cs)

Since the SvgNamespace is needed more widely across SvgElement and SvgAttributeAttribute, I extracted it into its own static class.
All unit tests pass.

Looks good to me. @H1Gdev, @wieslawsoltes - can one of you have another look please?

@mrbean-bremen
I will check this PR.

What is "Issue #2786" ?

Please separate PRs for each issue.

Oops - tidying up my branches. Well separate into smaller changes and resubmit.


replace custom context with custom queryClient

context
in v4, we introduced the possibility to pass a custom context to all react-query hooks. This allowed for proper isolation when using MicroFrontends. See the migration guide for examples and details.
However, context is a react only feature. All that context does is give us access to the query client. We could achieve the same isolation by allowing to pass in a custom query client instead of a custom context that then serves the query client.
vue-query does have this api already, so it makes sense to streamline it in react as well.
see also:

https://github.com/TanStack/query/discussions/4360

proposal

replace custom context param on react hooks with custom queryClient param

Blocked by https://github.com/TanStack/query/issues/4694


feat: add window effects api

What kind of change does this PR introduce?

[ ] Bugfix
[x] Feature
[ ] Docs
[ ] New Binding issue #___
[ ] Code style update
[ ] Refactor
[ ] Build-related changes
[ ] Other, please describe:

Does this PR introduce a breaking change?

[ ] Yes, and the changes were approved in issue #___
[x] No

Checklist

[x] When resolving issues, they are referenced in the PR's title (e.g fix: remove a typo, closes #___, #___)
[x] A change file is added if any packages will require a version bump due to this PR per the instructions in the readme.
[x] I have added a convincing reason for adding this feature, if necessary

Other information

I'm not sure why but I can't apply any of the effects on the API example (tried both macOS and Windows).

Ahh I forgot we removed transparency by default on the example. Let me check.

Ah I forgot we also need to document this on the config. And maybe it's best to put a reference on the decorations/fullscreen issue until it gets fixed, otherwise people will think the effects aren't working.


**About your pre-training model questions

When you use the pre-training model you provided for evaluation, F1 can reach 70. Your pre-training model uses bgr three-channel when reading images with opencv during training, so we also need to use bgr mode for reading Images during testing , and subtract three mean values.
As shown above, does the pre-training model parameters you provided are obtained by training 10,000 steps? The author mentioned in the original text that the training iterated 60K step!!!

@hawjnsafdjlk, the model is trained for 90K steps (I use 8 as batch size while original paper uses 12) and finetuned using another 10K steps. That is the reason why the checkpoint "shows" that the training steps are 10K.


RSM3415.cif no path in node cluster location

But this is not surprising as it has a floating component (mofchecker correctly flags it)


[10.prompt-validations] Unify structure of 'readme.md'

Proposed Changes
Unify the structure of readme.md across the differente languages in sample 10.prompt-validations.
Testing
The behaviour of the code was not modified.

No longer needed


Server side data provider fix for getting data from response / fixes #1

According to angular documentation:
The then method takes two arguments a success and an error callback which will be called with a response object. The success and error methods take a single argument - a function that will be called when the request succeeds or fails respectively. The arguments passed into these functions are destructured representation of the response object passed into the then method.
That's why currently parsedResponse.data is undefined. Changing success to then fixes the issue.
Important change is in line 100. The rest is white-space fix.

According to comments (#1) this pull request is invalid. Please close.

Closing.


GH-38255: [Java] Implement Flight SQL Bulk Ingestion

Please look at #38255 for details on this functionality. Support for Go and C++ was added as part of #38385.
This pull request is to add the required support for Java.

GitHub Issue: #38255

hi @lidavidm, any update please?

Sorry @eramitmittal, I will try to get to it tomorrow

It appears this fails the actual integration tests, though

Thanks looking into the integration failures. They passed on my local machine.

@lidavidm any hint on how to proceed?
Only Java 21+ runs are failing on Ubuntu complaining about memory leak in flightSqlIngestion test of IntegrationTest class.
However, I am not able to reproduce the problem.
I see a successful run when I manually run on Ubuntu with Java 21 in my local environment.
**

**
I have also tried to statically study the FlightSqlIngestionScenario code and I don't see how mem leak could be occurring.
Since the problem occurs only in CI, is there any possibility to run the tests in CI with DEBUG on for allocator?

That's not the error I'm concerned about, I'm more wondering why the integration tests fail:
 --------------
WARNING: Unknown module: org.apache.arrow.flight.core specified to --add-reads
WARNING: Unknown module: org.apache.arrow.memory.core specified to --add-opens
SLF4J(W): No SLF4J providers were found.
SLF4J(W): Defaulting to no-operation (NOP) logger implementation
SLF4J(W): See https://www.slf4j.org/codes.html#noProviders for further details.
Exception in thread "main" java.lang.AssertionError: Expected:
false
but got:
null
	at org.apache.arrow.flight.integration.tests.IntegrationAssertions.assertEquals(IntegrationAssertions.java:54)
	at org.apache.arrow.flight.integration.tests.FlightSqlScenario.lambda$validateMetadataRetrieval$0(FlightSqlScenario.java:172)
	at org.apache.arrow.flight.integration.tests.FlightSqlScenario.validate(FlightSqlScenario.java:227)
	at org.apache.arrow.flight.integration.tests.FlightSqlScenario.validateMetadataRetrieval(FlightSqlScenario.java:161)
	at org.apache.arrow.flight.integration.tests.FlightSqlScenario.client(FlightSqlScenario.java:76)
	at org.apache.arrow.flight.integration.tests.IntegrationTestClient.run(IntegrationTestClient.java:96)
	at org.apache.arrow.flight.integration.tests.IntegrationTestClient.main(IntegrationTestClient.java:66)

--------------

As for the memory leak, @vibhatha since we disabled the debug logging, how can it be reenabled here?

@lidavidm the integration tests should be fixed now with new pushed commit.

As for the memory leak, @vibhatha since we disabled the debug logging, how can it be reenabled here?

I need to take a look @lidavidm, I haven't enabled it.

Integration tests are passing now.
2 jobs are failing.
Java / AMD64 Ubuntu 22.04 Java JDK 17 Maven 3.9.6 (pull_request):
Failure is related to memory leak which I am unable to reproduce in my local environment. To diagnose the problem I need a CI run with allocator memory Debug turned on. Any suggestion on how to proceed?
Java JNI / AMD64 manylinux2014 Java JNI (pull_request):
These failure are strange given that I didn't touch Gandiva or Dataset projects:

Are we sure these are not some intermittent issues?

The failures were introduced by https://github.com/apache/arrow/pull/43503 .
The dataset failure is tracked by https://github.com/apache/arrow/issues/43506 .
The Gandiva failure isn't tracked yet. We need to use old LLVM for now. See also:

https://github.com/apache/arrow/pull/43503#issuecomment-2264551357
https://github.com/apache/arrow/pull/43503#issuecomment-2264965841

@kou thanks.
So, I just have to worry about mem leak failure.
Memory was leaked by query. Memory leaked: (2)
Allocator(ROOT) 0/2/32804/9223372036854775807 (res/actual/peak/limit)

	at org.apache.arrow.memory.BaseAllocator.close(BaseAllocator.java:503)
	at org.apache.arrow.memory.RootAllocator.close(RootAllocator.java:27)
	at org.apache.arrow.flight.integration.tests.IntegrationTest.testScenario(IntegrationTest.java:116)
	at org.apache.arrow.flight.integration.tests.IntegrationTest.flightSqlIngestion(IntegrationTest.java:85)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)```

Strangely sometimes it fails with the above error for JDK 17 and sometimes for JDK 21. It seems some kind of race condition. Let me try crossbow and see if it fails. Also, locally I will try to have multiple runs using same docker images as is being used here and try to reproduce the failure.

@github-actions crossbow submit java-jars

@github-actions crossbow submit -g java

Thanks. Integration tests pass with crossbow.

The failures we see in crossbow are all what @kou highlighted.
I ran the tests close to 10 times on my local machine using the maven docker image with JDK21 and they passed each time.
Only way now is an option to turn on allocator debug for CI.

@vibhatha can you show how to do that?

Sure @lidavidm I can take a look.

I am looking into this, I will post the steps once it is done.

hi @vibhatha any leads?

@eramitmittal unfortunately, I couldn't get the logs working in my initial attempt after adding the flags. I was testing it in another PR but couldn't try again, but the way to do is mentioned here: https://arrow.apache.org/docs/java/memory.html#debugging-memory-leaks-allocation

@vibhatha can you get it working locally? I think the debug mechanisms need to work...

Yes locally when I use the flags I can see the logs.

Let's try to get it working in CI then so we can debug this.

Let's try to get it working in CI then so we can debug this.

I will create an issue and work on this.

Issue created https://github.com/apache/arrow/issues/43843

@eramitmittal I have a draft PR: https://github.com/apache/arrow/pull/43880
This way you should be able to enable the logs. I checked and it works fine. Just make the changes shown in the pom.xml

@vibhatha done. Please approve CI run

@vibhatha done. Please approve CI run

@lidavidm please approve the CI run.

Sure, let's try it and see.

nope.... same old story. Interestingly same integration test pass every single time when run as part of conda integration tests.
Now I am at loss on how to proceed.
by the way is this expected (javac 11 getting used while the label says jdk 21)?

We could perhaps try to narrow down the source by using child allocators in certain contexts (e.g. separate server and client allocators), and/or modifying the debug code to only record a single traceback (where the buffer was originally allocated) and modifying the error to print this traceback out on failure too (the current debug code has very high overhead due to recording tracebacks nearly any time you do something with a buffer's refcount)

Also it seems 2 bytes leaked so possibly we could guess which buffer it was from that

@lidavidm I have added an allocationListener to the tests to keep track of allocation and release. In case of failure due to IllegalStateException I am then printing the allocation/release trail. Hopefully this should be able to provide information on where the mem leak occurs. Can you please approve the CI?

Done. It seems it's either the server or the client's handling of putResult that's the issue? (Maybe sometimes we aren't draining the stream on the client properly?)

I see client side allocating and releasing PutResult fine.
allocate: 2:
	org.apache.arrow.flight.integration.tests.TestBufferAllocationListener.onAllocation(TestBufferAllocationListener.java:39)
	org.apache.arrow.memory.BaseAllocator.buffer(BaseAllocator.java:337)
	org.apache.arrow.memory.BaseAllocator.buffer(BaseAllocator.java:297)
	org.apache.arrow.flight.PutResult.fromProtocol(PutResult.java:82)
	org.apache.arrow.flight.FlightClient$SetStreamObserver.onNext(FlightClient.java:488)
	org.apache.arrow.flight.FlightClient$SetStreamObserver.onNext(FlightClient.java:476)

release: 2:
	org.apache.arrow.flight.integration.tests.TestBufferAllocationListener.onRelease(TestBufferAllocationListener.java:43)
	org.apache.arrow.memory.AllocationManager.release(AllocationManager.java:171)
	org.apache.arrow.memory.BufferLedger.decrement(BufferLedger.java:153)
	org.apache.arrow.memory.BufferLedger.release(BufferLedger.java:123)
	org.apache.arrow.memory.BufferLedger.release(BufferLedger.java:104)
	org.apache.arrow.memory.ArrowBuf.close(ArrowBuf.java:1032)
	org.apache.arrow.flight.PutResult.close(PutResult.java:97)
	org.apache.arrow.flight.sql.FlightSqlClient.executeIngest(FlightSqlClient.java:328)

Problem is at server side:
//FlightSqlScenarioProducer.java
  private Runnable acceptPutReturnConstant(StreamListener<PutResult> ackStream, long value) {
    return () -> {
      final FlightSql.DoPutUpdateResult build =
          FlightSql.DoPutUpdateResult.newBuilder().setRecordCount(value).build();

      try (final ArrowBuf buffer = allocator.buffer(build.getSerializedSize())) {
        buffer.writeBytes(build.toByteArray());
        ackStream.onNext(PutResult.metadata(buffer));
        ackStream.onCompleted();
      }
    };
  }

I see release corresponding to ackStream.onCompleted()
release: 32:
	org.apache.arrow.flight.integration.tests.TestBufferAllocationListener.onRelease(TestBufferAllocationListener.java:43)
        ...........
	org.apache.arrow.flight.StreamPipe.onCompleted(StreamPipe.java:99)
org.apache.arrow.flight.integration.tests.FlightSqlScenarioProducer.lambda$acceptPutReturnConstant$0(FlightSqlScenarioProducer.java:526)
	org.apache.arrow.flight.FlightService.lambda$doPutCustom$0(FlightService.java:244)

..but no trace of buffer getting closed due to try-with-resources. It seems as if executor thread got completed before calling the finally!!!

  /** Request that the server shut down. */
  public void shutdown() {
    server.shutdown();
    if (grpcExecutor != null) {
      grpcExecutor.shutdown();
    }
  }

ExecutorService is being shutdown without waiting for pending tasks to finish.

@eramitmittal I guess then in this case we need to make sure we gracefully release all resources.

@vibhatha in general FlightServer.close() and shutdown methods need a review to account for pending tasks in ExecutorService as well. Other way obviously is that I can introduce a short delay in the IntegrationTest.testScenario method to allow ExecutorService threads to finish. But that will not solve the problem getting highlighted in FlightServer. I think graceful shutdown of FlightServer can be handled as a separate issue and for now I can just make the tests pass. What do you think?

Pushed a wait for ExecutorService to terminate in IntegrationTest.testScenario method. @lidavidm please approve the CI

failure in last run is not related to my changes:

All tests passed in last run except: AMD64 manylinux2014 Java JNI
Failure in that are not related to my changes:

Yes, this is irrelevant.

So nice to see the integration tests passing consistently now :). Hope to see this merged soon. Thanks

Argh, I didn't notice that this used a closed GitHub issue.


Support exclude specific routes from fetching session on prerendered pages

Hello! I was wondering if it is possible to implement a feature that excludes specific routes from session fetching on the client side when prerendering those pages.
Current behavior
When prerendering pages, every time a person accesses or reloads ANY page of my Nuxt app using this module, a GET request to /api/_auth/session will be made to fetch the session on mounted.
Feature request
My case would be:

Let's say I have a web app with prerendered page routes such as /about, /terms, /privacy, etc.
A /login page that externally redirects to SSR-enabled routes inside /app once authenticated.

I'd want the session to be fetched only for the /login page and page routes inside /app.
For the prerendered pages outside the application side, I'd like not to fetch the session while accessing them because it would be useless and waste requests count always calling /api/_auth/session every page access because it won't make use of that fetched data since the login auth will externally redirect to /app and will be SSR fetched there.
The approach I'm thinking of is adding a runtime config that accepts an array of specific paths that would work only for prerendered pages. This would be used to disable fetching.
Then in the file below return early from the mounted hook if the route path is included in the array, preventing session fetching.
https://github.com/Atinux/nuxt-auth-utils/blob/c8b02d0b84a53ab4dd41f1808d9365d1c52c8366/src/runtime/app/plugins/session.client.ts#L1-L9
Let me know what you think! 😊 or if there is already an alternative to achieve this.

TLDR;
Do you have a lot of pages which do not need authentication? I would suggest that only in that case you invest efforts. If it is just about simple homepage, terms page, etc. I would suggest to not waste your valuable development time on this.
The idea behind this is session is fetched when app is first loaded onto browser.
So, while your request is correct, I would still argue if it is feasible to add the complexity. Because a lot of the code expects session to be there. If user visits a whitelisted page and then navigates to authenticated page, session will be required.
Everything summed up, I would not recommend doing that. Our application also has the same case as yours and we decided to just accept it to be part of process and take that small hit (gaining a lot of simplicity and less potential bugs)

If user visits a whitelisted page and then navigates to authenticated page, session will be required.

I understand but in my case my app doesn't have any router navigation to an authenticated page outside the app routes.

Everything summed up, I would not recommend doing that. Our application also has the same case as yours and we decided to just accept it to be part of process and take that small hit (gaining a lot of simplicity and less potential bugs)

Perfect! I understand, thank you for the response and for your time reading my request ❤ the issue can be close as not planned 👍

Closing it but wanted to thank you both on brainstorming and giving more hints to other users reading this.
I agree with @amandesai01 as fetching the session anyway once loading the page avoid any complex logic to be added and this API call is not blocking anything too.


Added some functionality to footer section

Changed some features such as color and fonts in footer.
PR Description 📜
Added some new styling to footer section
Fixes #2762

Mark the task you have completed ✅

[ ✅] I follow CONTRIBUTING GUIDELINE & CODE OF CONDUCT of this project.
[ ✅] I have performed a self-review of my own code or work.
[✅ ] I have commented my code, particularly in hard-to-understand areas.
[ ✅] My changes generates no new warnings.
[✅ ] I have followed proper naming convention showed in CONTRIBUTING GUIDELINE
[✅ ] I have added screenshot for website preview in assets/images
[x] I have added entries for my game in main README.md
[ ] I have added README.md in my folder
[ ] I have added working video of the game in README.md (optional)
[ ] I have specified the respective issue number for which I have requested the new game.

Add your screenshots(Optional) 📸

Thank you soo much for contributing to our repository 💗

Thank you @tusquake ,for creating the PR and contributing to our GameZone 💗
Review team will review the PR and will reach out to you soon! 😇
Make sure that you have marked all the tasks that you are done with ✅.
Thank you for your patience! 😀

Thank You for letting me contibute @kunjgit bro❤

Please merge my pull request

/assign

i have already made the changes

Thank you  @tusquake , for your valuable time and contribution in our GameZone 💗.
It’s our GameZone, so Let’s build this GameZone altogether !!🤝
Hoping to see you soon with another PR again 😇
Wishing you all the best for your journey into Open Source🚀


calculator evaluates addition

Coverage increased (+32.1%) to 57.143% when pulling 77b2e992b45e42ef4034c217c7566c80490f8070 on addition into 3f785f1e34a24a5b98cbb1f0875c9af5603bff08 on master.


Add error reason for test failing in growl notifications

Hi I use mocha extensively throughout my node application and have it watching my directory to ensure that code is working as expected. It would be great if my growl notifications could display the reason for a failed test as well. If this is not a priority could u refer me to where I can perhaps learn how to implement this myself? Thanks.

A Growl reporter could be pretty sweet. Or some kind of hook.

I think it already does Growl but I'd argue it should be removed from core and available as a plugin. Hoping to have some solid time to work on the plugin API in the coming weeks. But, yes, a truncated failure report would be nice.

Issue has been inactive for years, closing.


JavaScript

JavaScript
User Story
As a boot camp student
I want the prework notes to be structured on a webpage
so that I can easily find and read the information.
Acceptance Criteria
GIVEN a Prework Study Guide website
WHEN I visit the website in my browser
THEN I can see four topics I learned  listed along with a suggestion on what I should study first

Complete


socketUrl when hosting a local server (via Docker)

Hi, it's me again,
So I was trying to collect some data using an experiment which submits the data to a server hosted locally via Docker (all working fine). While I am able to submit and retrieve data using directLink, the data submission is not working for my Prolific participants. I haven't been able to find anything in the documentation regarding the socketUrl in case of a local server, which I think might be my problem. I'm assuming that if I have serverUrl: 'http://localhost:4000/' my socketUrl would be socketUrl: 'http://localhost:4000/socket'. Is that correct? If not, any ideas what my issue might be? On the Prolific side it's all set up properly, with direct submission and participants being redirected back to Prolific directly.
Thank you!

Hi :)
Which version of magpie-base are you using?

Unless you use socket functionality you shouldn't need to set a socketUrl and if the socket URL was wrong, you'd already have trouble in directLink mode as well.

I've made the jump to magpie v3 at some point last year (finally, yay). Funnily directLink submission has worked when I was using socketUrl: 'http://localhost:4000/socket'. Should I just remove that field altogether then if I'm not using it?

What makes you think that socketUrl is the problem with prolific submissions?

To be fair, I was only guessing it had to do with that. Here's the config
file >
https://github.com/vini-macuch-silva/naturalness-rating/blob/master/src/magpie.config.js
< I'm not sure what else could be wrong, given that the directLink
submission works.
On Fri, Jan 26, 2024 at 1:09 PM Marcel Klehr @.***>
wrote:

What makes you think that socketUrl is the problem with prolific
submissions?
—
Reply to this email directly, view it on GitHub
https://github.com/magpie-ea/magpie-v3-docs/issues/4#issuecomment-1911973571,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AESJ5KTIRX7YM5ASSGVRUM3YQOMINAVCNFSM6AAAAABCKK3HO2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJRHE3TGNJXGE
.
You are receiving this because you authored the thread.Message ID:
@.***>

When people onprolific access your experiment, what do they enter? Setting the serverUrl to anything with localhost is probably not going to work with prolific, because the prolific people need to access the server. localhost always means "the local computer", so for people coming from prolific, their browser will try to look for the magpie backend on their computer.

You're right, I was sending them directly to my local server, which makes
sense that they should not be able to access. I somehow thought as long as
the Docker image was running the server would be accessible. I guess I have
to actually host the magpie backend on an online server somewhere. That
should solve my problem, duh!
On Fri, Jan 26, 2024 at 4:15 PM Marcel Klehr @.***>
wrote:

When people on prolific access your experiment, what do they enter?
Setting the serverUrl to anything with localhost is probably not going to
work with prolific, because the prolific people need to access the server.
localhost always means "the local computer", so for people coming from
prolific, their browser will try to look for the magpie backend on their
computer.
—
Reply to this email directly, view it on GitHub
https://github.com/magpie-ea/magpie-v3-docs/issues/4#issuecomment-1912231616,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AESJ5KQPJ7WALY66GA2MWBDYQPCCHAVCNFSM6AAAAABCKK3HO2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJSGIZTCNRRGY
.
You are receiving this because you authored the thread.Message ID:
@.***>

Hi @vini-macuch-silva 👋 , I think we need to update the documentation. In the meanwhile the README at https://github.com/magpie-ea/magpie-backend/ should document how to deploy on Gigalixir or Fly.io, two alternatives as Heroku is now no longer free. Could you give it a try and see if it works? Thanks 🙏 .

Hi @Xiang Ji @.***> Thanks for that, I remember you sending me
something like this a while back, probably the issue back then was that I
was still using magpie 2.0 (or I just never came back to the issue). I'll
have a look and report back.
On Fri, Jan 26, 2024 at 4:59 PM Xiang Ji @.***> wrote:

Hi @vini-macuch-silva https://github.com/vini-macuch-silva 👋 , I think
we need to update the documentation. In the meanwhile the README at
https://github.com/magpie-ea/magpie-backend/ should document how to
deploy on Gigalixir or Fly.io, two alternatives as Heroku is now no longer
free. Could you give it a try and see if it works? Thanks 🙏 .
—
Reply to this email directly, view it on GitHub
https://github.com/magpie-ea/magpie-v3-docs/issues/4#issuecomment-1912297418,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AESJ5KWXMFUD4TVNBBTMXHLYQPHGXAVCNFSM6AAAAABCKK3HO2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJSGI4TONBRHA
.
You are receiving this because you were mentioned.Message ID:
@.***>

OK, so I've tried the gigalixir installation and I'm getting stuck at the
last step, when doing the git push -u gigalixir master. Am I correct in
just following the steps in a blank folder with a git repo in it (with a
remote on github, ofc)?
On Fri, Jan 26, 2024 at 5:02 PM Vinicius Macuch @.***>
wrote:

Hi @Xiang Ji @.***> Thanks for that, I remember you sending me
something like this a while back, probably the issue back then was that I
was still using magpie 2.0 (or I just never came back to the issue). I'll
have a look and report back.
On Fri, Jan 26, 2024 at 4:59 PM Xiang Ji @.***> wrote:

Hi @vini-macuch-silva https://github.com/vini-macuch-silva 👋 , I
think we need to update the documentation. In the meanwhile the README at
https://github.com/magpie-ea/magpie-backend/ should document how to
deploy on Gigalixir or Fly.io, two alternatives as Heroku is now no longer
free. Could you give it a try and see if it works? Thanks 🙏 .
—
Reply to this email directly, view it on GitHub
https://github.com/magpie-ea/magpie-v3-docs/issues/4#issuecomment-1912297418,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AESJ5KWXMFUD4TVNBBTMXHLYQPHGXAVCNFSM6AAAAABCKK3HO2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJSGI4TONBRHA
.
You are receiving this because you were mentioned.Message ID:
@.***>

Right, so it's working as it should, data being submitted perfectly!
On Wed, Jan 31, 2024 at 8:08 PM Vinicius Macuch @.***>
wrote:

Never mind, I'm so stupid, ofc I need the backend files in the folder XD.
Problem solved, server up and running as it should! I'll report later once
I manage to get the data in from Prolific. Thanks for your help!
On Wed, Jan 31, 2024 at 7:20 PM Vinicius Macuch @.***>
wrote:

OK, so I've tried the gigalixir installation and I'm getting stuck at the
last step, when doing the git push -u gigalixir master. Am I correct in
just following the steps in a blank folder with a git repo in it (with a
remote on github, ofc)?
On Fri, Jan 26, 2024 at 5:02 PM Vinicius Macuch @.***>
wrote:

Hi @Xiang Ji @.***> Thanks for that, I remember you sending me
something like this a while back, probably the issue back then was that I
was still using magpie 2.0 (or I just never came back to the issue). I'll
have a look and report back.
On Fri, Jan 26, 2024 at 4:59 PM Xiang Ji @.***>
wrote:

Hi @vini-macuch-silva https://github.com/vini-macuch-silva 👋 , I
think we need to update the documentation. In the meanwhile the README at
https://github.com/magpie-ea/magpie-backend/ should document how to
deploy on Gigalixir or Fly.io, two alternatives as Heroku is now no longer
free. Could you give it a try and see if it works? Thanks 🙏 .
—
Reply to this email directly, view it on GitHub
https://github.com/magpie-ea/magpie-v3-docs/issues/4#issuecomment-1912297418,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AESJ5KWXMFUD4TVNBBTMXHLYQPHGXAVCNFSM6AAAAABCKK3HO2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJSGI4TONBRHA
.
You are receiving this because you were mentioned.Message ID:
@.***>


All cannot be chose from variable

While trying to report for all items or stores. Unable to choose All.

For now we can check all stores rather than all so i think we will live with that for now.


Auto "end run" when newgame and some physics lagging even when near host

what does auto end run mean lol, physics lagging is expected

Sry for my stupidity. All new run play the same seed till I click on "end run" button

alway turn on "end run" when new-game.

It was expected behaviour pre 0.30, something broke it seems.

seed not changing on new game? i cant replicate


STARTTLS w NNTP often doesn't work

Hello,
I am trying to get into a project I started testssl.sh NNTP support via STARTTLS flying.
I was using  OpenSSL 1.1.1 (the initially released version) just to get the STARTTLS handshake tested.
However it seems when looking at https://github.com/openssl/openssl/commit/8f85aa6bd3737547634bbbe1e141f728674fb594 from @robert-scheck openssl is trying to get the CAPABILITIES returned which in the cases I tested doesn't work at all.
Some realistic examples (CAPABILITIES,STARTTLS and ^] is my keyboard input):
prompt% telnet <IP_ADDRESS> 119
Trying <IP_ADDRESS>...
Connected to <IP_ADDRESS>.
Escape character is '^]'.
200 NNTP server ready
CAPABILITIES
500 command not recognized: CAPABILITIES
HELP
100 Avaliable commands:
ARTICLE
AUTHINFO
BODY
GROUP
HEAD
HELP
IHAVE
LAST
LIST
NEWGROUPS
NEWNEWS
NEXT
POST
QUIT
SLAVE
STAT
XHDR
.
STARTTLS
382 Continue with TLS negotiation
^]

prompt% telnet <IP_ADDRESS> 119
Trying <IP_ADDRESS>...
Connected to <IP_ADDRESS>.
Escape character is '^]'.
200 Kerio Connect 9.2.7 patch 3 NNTP server ready
CAPABILITIES
500 command not recognized: CAPABILITIES
STARTTLS
382 Continue with TLS negotiation
^]
telnet> Connection closed.

prompt% telnet <IP_ADDRESS> 119
Trying <IP_ADDRESS>...
Connected to <IP_ADDRESS>.
Escape character is '^]'.
200 CCProxy NNTP Service
CAPABILITIES
200 CCProxy NNTP Service
STARTTLS
Connection closed by foreign host.
prompt% telnet <IP_ADDRESS> 119
Trying <IP_ADDRESS>...
Connected to <IP_ADDRESS>.
Escape character is '^]'.
200 CCProxy NNTP Service
STARTTLS
200 CCProxy NNTP Service
^]
telnet>

prompt% telnet <IP_ADDRESS> 119
Trying <IP_ADDRESS>...
Connected to <IP_ADDRESS>.
Escape character is '^]'.
200 NNTP Service 6.0.3790.3959 Version: 6.0.3790.3959 Posting Allowed
CAPABILITIES
500 Command Not Recognized
STARTTLS
500 Command Not Recognized
^]
telnet> Connection closed.

This one works as expected and only here during my little research openssl returns  Didn't find STARTTLS in server response, trying anyway...\nSTARTTLS failed: 500 What?
telnet <IP_ADDRESS> 119
Trying <IP_ADDRESS>...
Connected to <IP_ADDRESS>.
Escape character is '^]'.
200 usenet-fr.net InterNetNews NNRP server INN 2.5.3 (20111012 snapshot) ready (posting ok)
CAPABILITIES
101 Capability list:
VERSION 2
IMPLEMENTATION INN 2.5.3 (20111012 snapshot)
AUTHINFO USER
HDR
LIST ACTIVE ACTIVE.TIMES COUNTS DISTRIB.PATS DISTRIBUTIONS HEADERS MODERATORS MOTD NEWSGROUPS OVERVIEW.FMT SUBSCRIPTIONS
OVER
POST
READER
.
STARTTLS
500 What?
^]
telnet>

This one works too (openssl: Didn't find STARTTLS in server response, trying anyway...\nSTARTTLS failed: 480 Authentication Required*
prompt% telnet <IP_ADDRESS> 119
Trying <IP_ADDRESS>...
Connected to <IP_ADDRESS>.
Escape character is '^]'.
200 Welcome to Usenet (fx06.am4)
CAPABILITIES
101 Capabilities list:
VERSION 1
AUTHINFO USER PASS
.
STARTTLS
480 Authentication Required*
^]
telnet> Connection closed.

In other cases openssl just hung after CONNECTED(00000005).
RFC 8143 [1], section 3.4, states news clients SHOULD attempt to negotiate TLS even if the server does not advertise the  STARTTLS capability label in response to the CAPABILITIES command  (Section 2.1 of [RFC4642]).
So probably either it would make sense not to wait whether STARTTLS shows up after  BIO_printf(fbio, "CAPABILITIES\r\n"); or maybe just to send STARTTLS instead and check whether there's a 382 returned for proceeding or anything else to stop with it.
I just picked a few server from shodan. This are probably not a good statistics but I checked a few common ones
Kerio Connect nntpd            4,546
Microsoft NNTP Service         2,815
CCProxy NNTP proxy             1,073
InterNetNews (INN)               463
Citadel (UX) messaging server    123

You might want to have a broader coverage as a few I checked didn't provide STARTTLS at all.
Cheers,Dirk
[1] Using Transport Layer Security (TLS) with Network News Transfer Protocol (NNTP)

while (mbuf_len > 1 && mbuf[0] != '.') ? Maybe while (mbuf_len == 0) is more appropriate to avoid the hanger.

Any news when this is being fixed.
I did a check on ~100 random hosts and the current status of NNTP support via STARTTLS is pretty much: not working. :-(

Hmmm. Support for the CAPABILITIES command seems to be required by RFC3977. Those servers that don't respond to is properly seem to be non-compliant....that's annoying! :-)
Probably we should check that the response code starts with 101, and ignore it otherwise.
We should also probably check that the STARTTLS response code is 382 and fail otherwise.
PRs welcome.

https://tools.ietf.org/html/rfc8143 :
3.4.  Prevention of SSL Stripping:

   In order to help prevent SSL Stripping attacks (Section 2.1 of
   [RFC7457]), NNTP implementations and deployments MUST follow the
   recommendations provided in Section 3.2 of RFC 7525 [BCP195].
   Notably, in case implicit TLS is not used, news clients SHOULD
   attempt to negotiate TLS even if the server does not advertise the
   STARTTLS capability label in response to the CAPABILITIES command
   (Section 2.1 of [RFC4642]).

We should also probably check that the STARTTLS response code is 382 and fail otherwise.

Oh - it seems we do do that in the latest version of the code.
@drwetter I'm not sure of the relevance of your quote. AFAICT the code does (attempt to) do that. It does check the CAPABILITIES response but then regardless of whether STARTTLS is explicitly listed or not, it tries to do it anyway. The only reason it checks CAPABILITIES at all is to print a helpful warning if it isn't explicitly there. Unfortunately that code seems to have a bug. It doesn't check for a 101 response code before it starts to parse the list of capabilities. This means if it gets something other than 101 it blocks trying to read the next line after the response code. But that input never comes because the server is waiting for the client to send it another command (hence the hang you experience).

@drwetter I'm not sure of the relevance of your quote

I was just referring to your previous answer, not to the code. :)
So it should send CAPABILITIES, look for an 101 or STARTTLS, maybe issue a warning and then just send STARTTLS and look for the 382?
--
Sent from my mobile. Excuse my brevity&typos+the phone's autocorrection

It should send CAPABILITIES and look for a 101 response code. If it gets 101 then look for STARTTLS as it does now. If it doesn't get 101 then don't attempt to parse the CAPABILITIES list (because we're not getting one). This is the bug as I see it at the moment - we attempt to parse CAPABILITIES whether or not we've had 101.
The rest of the behaviour seems correct AFAICT, i.e. if we've not seen STARTTLS in CAPABILITIES then we issue a warning. Regardless of whether or not we've seen STARTTLS in the CAPABILITIES list we try it anyway and check for a 382 response.

Just a clarification, should the client still attempt to negotiate TLS even after not receiving a 101 response code when requesting CAPABILITIES, or should it consider it a sign that the server is non-compliant and close the connection?

It should attempt to negotiate TLS no matter what CAPABILITIES returns. We only check CAPABILITIES to decide whether to print the warning or not.

I am happy to check this against a larger number of NNTP servers.


Make storage initializers required, just like local variables

Closes https://github.com/FuelLabs/sway/issues/2238

Make storage initializers required
Update all the tests and examples
Improve the docs for storage variables

Looks good to me. To summarize, if I understand correctly:

Update parser to remove optionality around initializer
remove any optionality in the type checker and constant evaluation of initializers
add initializers to all examples which were missing them

No new logic seems to be added, so this seems safe -- just removing the option stuff.
Nice PR, very thorough with all the examples and sway fmt stuff. 💪🏻

That's a good summary. As for the formatter stuff, sway-fmt-v2 seems to be dropping initializers actually so I filed in an issue here: https://github.com/FuelLabs/sway/issues/2279


Main reconcile logic and conditions

More unit test for stage transitions will be added in subsequent PRs.

/unhold

/hold

@eranco74 Thanks for the comments. I forgot to put the PR on hold as we are still refining the stage transition. Your comments are still valid though. So I responded.

/unhold

/approve


Refactor

Now support single file and folder structure IO
export/import are very mirrored for simple future refactoring
Remove JSON, now just use .jl (which retains structure and can be optionally included)

bors r+


[ISSUE #8596]  Remove redundant mvn test and log check steps from unit test workflow

Which Issue(s) This PR Fixes

Fixes #8596
Brief Description
Removed the redundant mvn test step and the separate JVM crash log check from the CI workflow. These changes simplify the process and improve efficiency.
How Did You Test This Change?
The test workflow

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 47.20%. Comparing base (00a05a5) to head (847d7f2).

Additional details and impacted files
@@              Coverage Diff              @@
##             develop    #8597      +/-   ##
=============================================
- Coverage      47.26%   47.20%   -0.06%
+ Complexity     11469    11448      -21
=============================================
  Files           1277     1277
  Lines          89342    89342
  Branches       11480    11480
=============================================
- Hits           42224    42174      -50
- Misses         41920    41961      +41
- Partials        5198     5207       +9

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Flaky tests are expected to be fixed and this is crucial. Opt pipeline is fine, IMO, but the prior one makes more sense.

Thank you for your guidance and feedback. I've reverted the changes to keep the macOS tests in the pipeline and will focus on stabilizing them to ensure consistent performance across all platforms.


could you add instructions for local builds?

hello, i would like to help contribute to this project, however, i cannot get the project to build from a clean git checkout. it seems like maven is expecting some environmental properties to be set. would you mind adding some documentation just to help developers get a local build/unit test setup working?
here are the errors i am getting from maven:
[ERROR] Child module /Users/michael/oss/arangodb-spark-datasource/arangodb-spark-datasource-${spark.compat.version} of /Users/michael/oss/arangodb-spark-datasource/pom.xml does not exist @
[ERROR] 'dependencies.dependency.artifactId' for com.arangodb:jackson-dataformat-velocypack${jackson.vpack.variant}:jar with value 'jackson-dataformat-velocypack${jackson.vpack.variant}' does not match a valid id pattern. @ line 165, column 25
[ERROR] 'dependencies.dependency.version' for com.arangodb:jackson-dataformat-velocypack${jackson.vpack.variant}:jar must be a valid version but is '${jackson.vpack.version}'. @ line 166, column 22
[ERROR] 'dependencies.dependency.version' for org.scala-lang:scala-library:jar must be a valid version but is '${scala.version}'. @ line 185, column 22
[ERROR] 'dependencies.dependency.artifactId' for org.apache.spark:spark-core_${scala.compat.version}:jar with value 'spark-core_${scala.compat.version}' does not match a valid id pattern. @ line 190, column 25
[ERROR] 'dependencies.dependency.version' for org.apache.spark:spark-core_${scala.compat.version}:jar must be a valid version but is '${spark.version}'. @ line 191, column 22
[ERROR] 'dependencies.dependency.artifactId' for org.apache.spark:spark-sql_${scala.compat.version}:jar with value 'spark-sql_${scala.compat.version}' does not match a valid id pattern. @ line 196, column 25
[ERROR] 'dependencies.dependency.version' for org.apache.spark:spark-sql_${scala.compat.version}:jar must be a valid version but is '${spark.version}'. @ line 197, column 22
[ERROR] 'dependencies.dependency.artifactId' for com.sksamuel.scapegoat:scalac-scapegoat-plugin_${scala.version}:jar with value 'scalac-scapegoat-plugin_${scala.version}' does not match a valid id pattern. @ line 202, column 25```

Here can see how to build, start the test environment and test: https://github.com/arangodb/arangodb-spark-datasource/blob/main/.github/workflows/test.yml#L16-L84
For more clarity I will also update https://github.com/arangodb/arangodb-spark-datasource/blob/main/dev-README.md accordingly.

@rashtao thank you very much! sorry i didn't see all the other README files. once i get my local build running i will close this issue.


PlatformIO compatibility

Hi,
I have been trying to integrate your repository with PlatformIO teensy 3.6, but it was not successful.
Any references for that ?

I have nothing to do with SdFat on PlatformIO.  Someone setup SdFat on PlatformIO but there are problems with versions.
See this.


Fix config_read_file error handling

There was a bug in the code and in the config file, we were not
using the right syntax.
Closes #43

@DanielAviv Please take a look of the syntax error we had, you might need to do some changes.


Could not resolve all files for configuration ':app:debugCompileClasspath'.

I am getting this error when I try to run app.
Execution failed for task ':app:dataBindingMergeDependencyArtifactsDebug'.

Could not resolve all files for configuration ':app:debugCompileClasspath'.
Could not find com.github.dkzwm:SmoothRefreshLayout:core.
Required by:
project :app

Possible solution:

Declare repository providing the artifact, see the documentation at https://docs.gradle.org/current/userguide/declaring_repositories.html

Have you configured the following code in the build.gradle file in the root directory?
allprojects {
    repositories {
        google()
        maven { url 'https://jitpack.io' }
    }
}

If you have already configured it, please try to clear the cache or restart the IDE, hope it can help you！

Yes, I have configured everything correctly. I just cleared and rebult IDE, also restarted my notebook. Same error occurs. Do you have any idea how to solve it?

em.. I don’t know what the problem is, I can’t help you temporarily. I will test it tomorrow, hope I can find the reason tomorrow.

There should be a bug in the JitPack repository, here you can use the dependency of JCenter first while JCenter has not been closed.
    implementation 'me.dkzwm.widget.srl:core:<IP_ADDRESS>.androidx'
    implementation 'me.dkzwm.widget.srl:ext-util:<IP_ADDRESS>.androidx'
    implementation 'me.dkzwm.widget.srl:ext-material:<IP_ADDRESS>.androidx'
    implementation 'me.dkzwm.widget.srl:ext-horizontal:<IP_ADDRESS>.androidx'
    implementation 'me.dkzwm.widget.srl:ext-classics:<IP_ADDRESS>.androidx'
    implementation 'me.dkzwm.widget.srl:ext-two-level:<IP_ADDRESS>.androidx'
    implementation 'me.dkzwm.widget.srl:ext-dynamic-rebound:<IP_ADDRESS>.androidx'

When I tried to use all 7 implementations with androidx, it gave me another error saying 'Execution failed for task ':app:dataBindingMergeDependencyArtifactsDebug'.
But all I need was core library. When I used implementation  with androidx version (`me.dkzwm.widget.srl:core:<IP_ADDRESS>.androidx') it worked perfectly.


Very slow installation

Steps to reproduce:

clone repository
time nix-shell --run "hydra-check  python310Packages.faiss"

...
these 183 paths will be fetched (418.20 MiB download, 2159.73 MiB unpacked)
...
these 77 derivations will be built:
...
real    3m38,434s

Compare it with the installation time of the previous version
nix-collect-garbage
git checkout 8bd45f8cef8b9ac27c99513ec260e92705db681f
time nix-shell --run "hydra-check  python310Packages.faiss"

real    0m2,014s

I do nix-collect-garbage pretty regularly so its painfully slow to use this tool every time I need it.
Its good that I remembered that it was fast when I used it first time .
So any other poor soul, who just wants to quickly run 200-line python script, use the previous version: 8bd45f8cef8b9ac27c99513ec260e92705db681f.

Hi, i am unsure how using an older version changes your download time, maybe your nixpkgs channel does not fit your system channel? Another possibility could be that accidentally the dependencies of the old version of hydra-check are cached by one of your other active derivations. Check out https://nixos.wiki/wiki/Storage_optimization#Look_for_result_symlinks
I checked the diff between 1.3.4 and 1.3.0 but actually 1.3.4 contains fewer dependencies: docopt and types-docoptwere removed.
Apart from that, hydra-check has requests,colorama and beautifulsoup as dependencies, which is really not that much.
I am closing this issue for now, not sure how else i could help here.

I am closing this issue for now, not sure how else i could help here.

I am fine with it. I created this issue for discoverability.

he old version of hydra-check are cached by one of your other active derivations

I tried in clean environment:

docker run -ti nixos/nix
nix-channel --update
clone latest version git clone https://github.com/nix-community/hydra-check.git
time nix-shell --run "hydra-check  python310Packages.faiss"

...
these 37 derivations will be built:
...
these 218 paths will be fetched (197.79 MiB download, 916.08 MiB unpacked):
...
real    1m45.961s

nix-collect-garbage
git checkout 8bd45f8cef8b9ac27c99513ec260e92705db681f
time nix-shell --run "hydra-check  python310Packages.faiss"

these 105 paths will be fetched (117.38 MiB download, 499.12 MiB unpacked):
...
real    0m12.477s

I need to say that your recent changes helped a lot.
Checkout revision that was latest when I opened this issue.
8. nix-collect-garbage
9. git checkout 876324f114983429a6aaae4af7fd87e2eabf5eee
10.  time nix-shell --run "hydra-check  python310Packages.faiss"
...
these 77 derivations will be built:
...
these 226 paths will be fetched (480.58 MiB download, 2349.08 MiB unpacked):
...
real    4m58.780s

My personal takeaway from this, is not to use poetry2nix in my personal projects.


Copyediting Pass: Form Page Error Block & Toughened Rescheduling Language

This PR updates copy in the error message block on the form page.

This PR updates the language around rescheduling and the impact on current appointments:

 Rewrite error block text for Form page

 Strengthen language around loss of original appointment [XS]

Some of the tests are failing (they are looking for specific text in error messages), but I will update those.


Triton Http proxy server

[x] Basic aiohttp server for setting and getting Triton parameters, with GET and POST
[x] RemoteTriton driver. Should be a drop in replacement for the current driver to use the proxy.
[x] Websockets exposing Json representation of the data.
[ ] Testing locally on real Triton computer
[ ] Benchmarking. Requires Local testing
[ ] Finish reading of Parameter names from regdatabase on local Triton.

I think this is now ready to be tested locally on a Triton computer. The websockets implementation is done. However as noted in the comments the aiohttp websockets library is now well suited for this kind of operation due to the design of its write coroutine.

@jenshnielsen  yes  that  web socket looks  really nasty. The loop wait time should also be configurable  .
Let's try this on fridge and see, then we can review this properly.

As per discussion  IRL, it's better to have the http server to also be ws client rather than exposing a web socket endpoint ?

First version of that pushed. Seems to work ok, gracefully handles ws-server going offline and coming back. The websockets library is way better than the aiohttp websockets support

@jenshnielsen probably not pushed but yes it is indeed much better!

pushed now. Not sure what happened yesterday thought I pushed

Looks good, need some polishing around but nice!
We may want to consider to add a endpoint to stop/start/restart the web socket client ?

Benchmarked with @MerlinSmiles on T12
Getting
Existing driver
%%time
triton.PT1()
CPU times: user 752 µs, sys: 1.08 ms, total: 1.83 ms
Wall time: 3.09 ms

Http server proxy
%%time
triton.T13()
CPU times: user 3.92 ms, sys: 916 µs, total: 4.83 ms
Wall time: 11.6 ms

Setting
Local
%%time
triton.pid_rate(0.001)
CPU times: user 1.04 ms, sys: 1.14 ms, total: 2.18 ms
Wall time: 281 ms

Http server proxy
%%time
triton.pid_rate(0.002)
CPU times: user 3.34 ms, sys: 1.29 ms, total: 4.63 ms
Wall time: 316 ms

So it is quite a bit slower getting values via a http get but the effect on setting them seems to be much smaller

Running benchmark again. T10 while websocket connection is sending every second
%%time
triton.T13()
CPU times: user 4.07 ms, sys: 1.26 ms, total: 5.33 ms
Wall time: 6.18 ms

Repeated several times seems to be consistent around 6 ms
Faster than above (due to wired network connection ?)

Repeating the same benchmark while another client is getting a result every 1 second seems to give similar results
%%time
triton.T13()
CPU times: user 3.84 ms, sys: 1.21 ms, total: 5.04 ms
Wall time: 5.83 ms

Increasing the polling to every 0.1 seconds does not seem to be an issue either.
%%time
triton.T13()
CPU times: user 3.32 ms, sys: 1.49 ms, total: 4.81 ms
Wall time: 5.09 ms

could you do those tests continously to see for how long another client blocks a request?

Running one client continuously with no throttle and getting from another one I see
%%time
triton.T13()
CPU times: user 4.51 ms, sys: 2.41 ms, total: 6.92 ms
Wall time: 8.72 ms

Getting the results locally is still faster (so the current speedup is likely due to wired network)
%%time
triton.T1()
CPU times: user 615 µs, sys: 815 µs, total: 1.43 ms
Wall time: 1.49 ms

@jenshnielsen let's merge this asap but not before #519 is fixed 🤐

lol sleep deprivation, it's closed already./


Highlight {roxygen2} comments in R code

When writing R code it would be great to have {roxygen2} comments highlighted similarly to how it's done in RStudio. This isn't part of R's official syntax, but as most R users, and particularly package developers, now use roxygen and RStudio, I think it would make sense to include it in Carbon. Here's an example (using the theme Cobalt):

Thanks for the hard work on Carbon, it's a lovely tool!

Hey @wurli this change would have to be made in the Codemirror R mode package.
Please open an issue there, and consider fixing the issue upstream 🙂


Added gfx906 kernel binary to the HIPFB file for hipModule test.

@kpyzhov Can't this be auto-generated by BUILD_CMD as done for HIP/HCC?

Hi Rahul,
I believe the binary can be auto-generated for HIP as well. I'm working on modifying the HIP test makefile now. Thanks!


Workflow migration feature in version 4.1 - is SPO site source supported?

I am so sorry that I am re-opening issue #3830 regarding workflow migration feature.
Can you please help clarifying if the migration source can be SharePoint Online?  Please review the 2 screenshots I provided in the attachment.   Our use case is that we have already migrated all SharePoint sites from on-prem to SharePoint online already but has some  2013 type of workflow still thee in SPO that we'd like to see SPMT can be used to migrate workflows in SPO sites to Power Automate.
Screenshots.docx

Any news?  @get-itips or @scanum ?

Hello @KuotingChiu
How are you? AFAIK SPMT supports only On-Premises sources
https://learn.microsoft.com/en-us/sharepointmigration/what-is-supported-spmt

Thanks!  That's seems to be the case the tool cannot use the URL of SPO as the source.   Too bad it is not supported as we have sunset our on-prem SharePoint farms already.

Not sure if a product design change request/idea can be submitted