Problems with local build instructions on Mac (and perhaps other 'nix platforms)

It turns out that the instructions for doing local builds ( at least on a Mac) don't work. The steps I've gone through so far to make progress (with links to some of the relevant roadblocks) are:

Install command line tools (xcode-select --install) (may not be necessary depending on how recent your last OS update was).
Update gem itself (see this in-progress update to the jupyter-book instructions, which includes a link to a statement from the bundler folks and an answer on stackoverflow). THIS IS NOT MAC-SPECIFIC
Install the conda compilers for mac: conda install clangxx_osx-64 (see https://stackoverflow.com/questions/47314899/how-to-avoid-clang-no-such-file-or-directory-error). My hunch is that something like this is needed on linux too.
On 10.14/Mojave, make the standard headers show up in the standard places because apple just had to move them.

Once that stuff is done, make install proceeds much farther, until it gets to building eventmachine, which uses openssl, and it can't find the headers for that:
./project.h:119:10: fatal error: 'openssl/ssl.h' file not found
#include <openssl/ssl.h>
         ^~~~~~~~~~~~~~~

That led me to this anaconda issue about building gems. I tried the monkey patch at https://github.com/ContinuumIO/anaconda-issues/issues/9863#issuecomment-460512146 but it didn't do the trick for me.
Will continue trying for a little bit longer and report back if I get something to work.

Success, sort of....
In addition to the monkey patch I linked to above, I had to add this to CPPFLAGS in rbconfig.rb:
-I/Users/mattcraig/anaconda3/anaconda/envs/ccd-guide/include

I think there are a couple of other places I could have stuck that.
Let me know if you want me to write this up into install instructions.

holy moly, thanks for all of this information. @emdupre you may be interested in this too in thinking about whether we should try to use something that's less of a hassle like hugo instead of jekyll

another quick question for you: it seemed like the ruby gems header problem wasn't present when using Ruby installed with conda-forge...did you double-check that this is the version of ruby you're using?

I was using the version from defaults, can try with the conda-forge version in a fresh environment if you want:
~/a/a/e/c/l/r/2/x86_64-darwin14 $ conda list ruby                                                                                (ccd-guide)
# packages in environment at /Users/mattcraig/anaconda3/anaconda/envs/ccd-guide:
#
# Name                    Version                   Build  Channel
ruby                      2.5.1                h74228e1_0

@mwcraig if conda-forge doesn't work, could you try with brew since you're on a mac? We will figure this out one way or another :-)

@choldgraf -- with just a touch of modification things seem to work with the conda-forge ruby. The twist is that the compilers either need to be installed before ruby or the ruby version needs to be pinned because the version on main is a more recent version than the one on conda-forge.
So this should work:
conda install -c conda-forge clangxx_osx-64
conda install -c conda-forge ruby

OR
conda install -c conda-forge ruby=1.4.5 clangxx_osx-64

It may still be necessary to do the header thing on OSX...I'm not sure because I don't know how to undo the header installation and didn't want to risk breaking things by deleting stuff from /usr/include 😀

Thanks for the tip, @mwcraig ! It sounds like you've gotten it all working now ?
We're starting to add Docker installation instructions in #127 to hopefully avoid some of these installation headaches. We'd really appreciate your feedback there, if you're interested !
Either way, thanks again for keeping us updated -- happy to hear it seems to be working :sparkles:

Yep, it is all working now.
You should add installation of the conda-forge compilers to the install instructions for mac/linux because both platforms will need them.

@mwcraig that's a good point - would you mind opening an issue about it so that we don't lose that point? Would also love to hear your thoughts over in #83 if you've got any :-)

To wrap this up, these steps worked for me in a clean environment on a system on which I had upgraded to mojave fairly recently (so the headers are not in the standard places); it ends being simpler than what I originally reported. The first two steps will be necessary on mac and linux. The list step should be needed only on OSX 10.14 or later:
# Line below should change for linux...
conda install clangxx_osx-64
conda install -c conda-forge ruby

# OSX 10.14+ only:
open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

Can open a PR to the instructions for this Wednesday am....

@mwcraig please do! improvements here are most-welcome :-)


Unable to extract uploader id

Checklist

[ ] I'm reporting a broken site support issue
[x ] I've verified that I'm running youtube-dl version 2021.12.17
[x ] I've checked that all provided URLs are alive and playable in a browser
[x ] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar bug reports including closed ones
[x ] I've read bugs section in FAQ

Verbose log

PASTE VERBOSE LOG HERE

$ youtube-dl --verbose https://www.youtube.com/watch?v=4S94TIDv9CA
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', 'https://www.youtube.com/watch?v=4S94TIDv9CA']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Python version 3.10.6 (CPython) - Linux-5.19.0-45-generic-x86_64-with-glibc2.35
[debug] exe versions: ffmpeg 5.1.2, ffprobe 5.1.2, rtmpdump 2.4
[debug] Proxy map: {}
[youtube] 4S94TIDv9CA: Downloading webpage
ERROR: Unable to extract uploader id; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/youtube_dl/YoutubeDL.py", line 815, in wrapper
return func(self, *args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/youtube_dl/YoutubeDL.py", line 836, in __extract_info
ie_result = ie.extract(url)
File "/usr/local/lib/python3.10/dist-packages/youtube_dl/extractor/common.py", line 534, in extract
ie_result = self._real_extract(url)
File "/usr/local/lib/python3.10/dist-packages/youtube_dl/extractor/youtube.py", line 1794, in _real_extract
'uploader_id': self._search_regex(r'/(?:channel|user)/([^/?&#]+)', owner_profile_url, 'uploader id') if owner_profile_url else None,
File "/usr/local/lib/python3.10/dist-packages/youtube_dl/extractor/common.py", line 1012, in _search_regex
raise RegexNotFoundError('Unable to extract %s' % _name)
youtube_dl.utils.RegexNotFoundError: Unable to extract uploader id; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Description

WRITE DESCRIPTION HERE
Seems to be the same problem listed earlier. I assume YouTube keeps changing up its format to cause this.

Same issue here.

Same here

[ ] I'm reporting a broken site support issue

You are, but shouldn't be: see below.

[x] I've searched the bugtracker for similar bug reports including closed ones

Learn how by reading #30839 (again?).

[x] I've read bugs section in FAQ

Apply this.

... YouTube keeps changing up its format to cause this.

If anything, you keep not using an updated version of the program. This hasn't changed since February.

Thanks!
Why don't you put it in the "known bugs" part of the FAQ? I didn't see the discussion of yt-dlp anywhere, and I tried updating 3 different ways but kept getting the 2021.12.17 version.
-yours
Rob

From: dirkf @.***>
Sent: Friday, June 23, 2023 2:02 PM
To: ytdl-org/youtube-dl
Cc: Robert Sheldon; Author
Subject: [Ext] - Re: [ytdl-org/youtube-dl] Unable to extract uploader id (Issue #32347)
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
[ ] I'm reporting a broken site support issue
You are, but shouldn't be: see below.
[x] I've searched the bugtracker for similar bug reports including closed ones
Learn how by reading #30839https://usg02.safelinks.protection.office365.us/?url=https%3A%2F%2Fgithub.com%2Fytdl-org%2Fyoutube-dl%2Fissues%2F30839&data=05|01||e07619b3d91449cf69df08db741c684b|b765b7c18d554c69b7336f16240cd8cb|0|0|638231437595422807|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=I1jsKKSH8OpT8Ysj%2Fnhvvx5%2BM%2FLYPCovDO2coLm8Kcc%3D&reserved=0 (again?).
[x] I've read bugs section in FAQ
Apply this.
... YouTube keeps changing up its format to cause this.
If anything, you keep not using an updated version of the program. This hasn't changed since February.
—
Reply to this email directly, view it on GitHubhttps://usg02.safelinks.protection.office365.us/?url=https%3A%2F%2Fgithub.com%2Fytdl-org%2Fyoutube-dl%2Fissues%2F32347%23issuecomment-1604737093&data=05|01||e07619b3d91449cf69df08db741c684b|b765b7c18d554c69b7336f16240cd8cb|0|0|638231437595427793|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=QySUAOQVcMWbvewNC6miWgUwSn84A7cOJ%2FOCTnlYuCg%3D&reserved=0, or unsubscribehttps://usg02.safelinks.protection.office365.us/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAG442BMYMUGEFILJU5LHRE3XMXR4XANCNFSM6AAAAAAZR34OOU&data=05|01||e07619b3d91449cf69df08db741c684b|b765b7c18d554c69b7336f16240cd8cb|0|0|638231437595432765|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=C3BQqYANvfm0QjAZ%2FfpMftMV1rJ5kAvVyoZQGTyAC%2Bo%3D&reserved=0.
You are receiving this because you authored the thread.Message ID: @.***>
This e-mail including any attachments may contain information that is privileged, confidential, and/or otherwise protected from disclosure to anyone other than its intended recipient(s). Any dissemination or use of this e-mail or its attachments by persons other than the intended recipient(s) is prohibited. If you received this message in error, please contact the sender immediately. Please then delete the e-mail and any attachments in its entirety.

Did you actually read the pinned issue (known issues, link to updating advice)?


Could not load file or assembly 'TechTalk.SpecFlow'

I'm getting the following compilation error message:
C:\Builds\1\1\src\MySolution\packages\SpecFlow.2.0.0\tools\TechTalk.SpecFlow.targets(47,5): error : Could not load file or assembly 'TechTalk.SpecFlow, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=0778194805d6db41' or one of its dependencies. The system cannot find the file specified. [C:\Builds\1\1\src\MySolution\MySpecFlowProject\MySpecFlowProject.csproj]

This issue only happens randomly, using MSBuild 14.0 (on a Visual Studio 2015 solution\project), and SpecFlow 2.0, even after removing the project reference to TechTalk.SpecFlow.Generator as indicated here.
MySpecFlowProject referenced NuGet packages (packages.config):
<?xml version="1.0" encoding="utf-8"?>
<packages>
  <package id="SpecBind" version="<IP_ADDRESS>2" targetFramework="net45" />
  <package id="SpecBind.CodedUI" version="<IP_ADDRESS>2" targetFramework="net45" />
  <package id="SpecBindGeneratorPlugin.SpecFlow" version="<IP_ADDRESS>2" targetFramework="net45" />
  <package id="SpecFlow" version="2.0.0" targetFramework="net45" />
  <package id="SpecFlow.CustomPlugin" version="2.0.0" targetFramework="net45" />
  <package id="SpecFlow.MsTest" version="2.0.0" targetFramework="net45" />
</packages>

MySpecFlowProject project references:
    <Reference Include="SpecBind, Version=<IP_ADDRESS>2, Culture=neutral, processorArchitecture=MSIL">
      <HintPath>..\packages\SpecBind.<IP_ADDRESS>2\lib\net45\SpecBind.dll</HintPath>
      <Private>True</Private>
    </Reference>
    <Reference Include="SpecBind.CodedUI, Version=<IP_ADDRESS>2, Culture=neutral, processorArchitecture=MSIL">
      <HintPath>..\packages\SpecBind.CodedUI.<IP_ADDRESS>2\lib\net45\SpecBind.CodedUI.dll</HintPath>
      <Private>True</Private>
    </Reference>
    <Reference Include="TechTalk.SpecFlow, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=0778194805d6db41, processorArchitecture=MSIL">
      <HintPath>..\packages\SpecFlow.2.0.0\lib\net45\TechTalk.SpecFlow.dll</HintPath>
      <Private>True</Private>
    </Reference>
    <Reference Include="TechTalk.SpecFlow.Parser, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=0778194805d6db41, processorArchitecture=MSIL">
      <HintPath>..\packages\SpecFlow.CustomPlugin.2.0.0\lib\net45\TechTalk.SpecFlow.Parser.dll</HintPath>
      <Private>True</Private>
    </Reference>
    <Reference Include="TechTalk.SpecFlow.Utils, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=0778194805d6db41, processorArchitecture=MSIL">
      <HintPath>..\packages\SpecFlow.CustomPlugin.2.0.0\lib\net45\TechTalk.SpecFlow.Utils.dll</HintPath>
      <Private>True</Private>
    </Reference>

MySpecFlowProject.csproj includes TechTalk.SpecFlow.targets as follows:
<Import Project="..\packages\SpecFlow.2.0.0\tools\TechTalk.SpecFlow.targets" />

MySpecFlowProject's App.config:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <configSections>
    <section name="specFlow" type="TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler, TechTalk.SpecFlow" />
  </configSections>
  <specFlow>
    <generator allowDebugGeneratedFiles="true" />
    <stepAssemblies>
      <stepAssembly assembly="MyOtherStepAssembly" />
    </stepAssemblies>
    <plugins>
      <add name="SpecBindGeneratorPlugin" path="..\packages\SpecBindGeneratorPlugin.SpecFlow.<IP_ADDRESS>2\tools" type="Generator" />
    </plugins>
    <unitTestProvider name="MsTest" />
  </specFlow>
  <runtime>
    <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
      <dependentAssembly>
        <assemblyIdentity name="TechTalk.SpecFlow" publicKeyToken="0778194805d6db41" culture="neutral" />
        <bindingRedirect oldVersion="<IP_ADDRESS>-<IP_ADDRESS>" newVersion="<IP_ADDRESS>" />
      </dependentAssembly>
    </assemblyBinding>
  </runtime>
</configuration>

The build directory is always cleaned before a build.
This only started happening when I added a NuGet reference to SpecBind.CodedUI.
SpecBind.CodedUI, Version=<IP_ADDRESS>2 references TechTalk.SpecFlow, Version=<IP_ADDRESS>.
There are two feature files in the project and their Custom Tool property is set to SpecFlowSingleFileGenerator.
Builds and rebuilds work fine in Visual Studio 2015, just not during automated / continuous integration (CI) builds.
I'm wondering if there's a way to explicitly indicate the path of where to look for TechTalk.SpecFlow.dll or something similar?
Thank you.

What fixed it for me was to remove https://ci.appveyor.com/nuget/specflow-ci from NuGet.config and also update the version of NuGet.exe I was using.

That makes sense, just wondering, why were you using the appveyor fend instead of NuGet.org?

I was testing with a newer version of SpecFlow to see if it had fixed some of the issues I was having.
For example, https://github.com/techtalk/SpecFlow/issues/482


Observed Generation Not Set to -1, Causing Incorrect Readiness Status

The observed generation is not set to -1 in the Terraform object's status, causing Kubernetes kstatus to consider it ready even when it is still applying.
When a Terraform resource is created, its status does not have the observed generation set to -1. This results in Kubernetes kstatus considering the Terraform resource as ready, even though it is still in the process of applying changes.

Code reference in GitRepository types: https://github.com/fluxcd/source-controller/blob/main/api/v1/gitrepository_types.go#L270
Code reference in Terraform types: https://github.com/weaveworks/tf-controller/blob/main/api/v1alpha2/terraform_types.go#L402
Example usage in Kustomization Controller: https://github.com/fluxcd/kustomize-controller/blob/main/internal/controller/kustomization_controller.go#L172

Proposed ACs:

The observed generation status of a Terraform resource is set to -1 upon resource creation.
The Flux runtime package applies an observedGeneration on each condition each time it observes that condition for

that generation.


Crash when removing headphone jack

Describe the bug
I simply removed my headphone and plugged my external speaker and this crash happened.

Full log: https://gist.github.com/ppamorim/5c769bd361a85b1ee232d8f6713ea539
Steps to Reproduce
Not sure, happened randomly when unplugging the jack.
Expected behaviour
Do not crash when removing the jack.
Setup information:

Audio device used for playback: Sony WH1000MX3
Audio transmission interface: 3.5mm Jack
macOS Version:10.15.4
eqMac Version: 0.2.0
eqMac UI Version: 1.0.0

Did you see #227 ?
This is a duplicate

No I didn't, the title is not easy to understand.


Logs list all cameras as same name

I have 4 IP cameras with different names working with this plugin (except live streams on AW https://github.com/KhaosT/homebridge-camera-ffmpeg/issues/77)
All 4 cameras are listed by the last camera name in my config.json file.
May  6 05:44:06 qnap-hb homebridge[6000]: [5/6/2017, 5:44:06 AM] Stairwell is running on port 34633.
May  6 05:44:06 qnap-hb homebridge[6000]: [5/6/2017, 5:44:06 AM] Stairwell is running on port 42610.
May  6 05:44:06 qnap-hb homebridge[6000]: [5/6/2017, 5:44:06 AM] Stairwell is running on port 40429.
May  6 05:44:06 qnap-hb homebridge[6000]: [5/6/2017, 5:44:06 AM] Stairwell is running on port 45871.

Same for me

me too


Rename 'examples' to 'templates'

The term "examples" is a term I would associate with complete and runnable manifests. However there are many placeholders inside them currently, and this confused me initially. I was not expecting to have to change them to get things running.
I think it would make a lot more sense to call them "templates".

cc @Bradamant3 for your thoughts

We've removed the templated static YAML and replaced with the velero install command. What's left are truly examples. Closing this issue.


[lint] remove usage of passing internal eslintrc via command line

and replacing them with eslint.config.mjs so that both command line eslint and
the vscode eslint extension can use the same config.

since vite-plugin-browser-test-map doesn't depend on eslint-plugin-azure-sdk, add linter dev dependencies and a flat config to it.

add an internal flat config to our eslint plugin. use it in added flat config for test utility packages

The failure in js - eslint-plugin - ci is unrelated and appeared earlier.

/check-enforcer override


Fixed broken link

Fixed #4148

Thanks!

@ericholscher actually the href attribute is still incorrect, now there are two http://

Ha, I see what you meant now. Will fix.


content/ja/docs/tasks/debug-application-cluster/get-shell-running-container.md is outdated

This is a Feature Request

What would you like to be added
Update content/ja/docs/tasks/debug-application-cluster/get-shell-running-container.md to follow the en updates.
Why is this needed
content/ja/docs/tasks/debug-application-cluster/get-shell-running-container.md is outdated.
Comments
/language ja
/assign
diff
https://gist.github.com/oke-py/c39af55156d03e6fc0e9bac169b2b99d

/close


TLS issue with Docker healthcheck

When running Zipkin with TLS enabled as a docker-compose service (using the following env var: - JAVA_OPTS=-Darmeria.ssl.key-store=keystore.p12 -Darmeria.ssl.key-store-password=password -Darmeria.ssl.enabled=true -Darmeria.ports[0].port=9411 -Darmeria.ports[0].protocols[0]=https), the server starts up and the UI works with https, but zipkin logs the Caused by: javax.net.ssl.SSLHandshakeException: error:1000009c:SSL routines:OPENSSL_internal:HTTP_REQUEST exception stacktrace every 5 seconds and the log size grows quickly.
It seems there's a docker healthcheck which only targets the http endpoint:
https://github.com/openzipkin/zipkin/blob/master/docker/Dockerfile#L70
https://github.com/openzipkin/zipkin/blob/master/build-bin/docker/docker-healthcheck#L46
Expected behaviour:
The healthcheck endpoint should be dynamic, based on which protocol is enabled (http/https).

We can only support things that are top-level config via ENV variables, even things passed through spring config might work.
In other words, trying to detect this config and handle it in the health check won't likely work. I would recommend you overwrite the docker-healthcheck on the image via a new image, volume mount or otherwise. K8s won't read it anyway.

ps anyone who has a workaround to share, please comment here!


plans for pyschedule

This is useful project but its development seems to have stalled. I hope this project can continue to improve. Does anyone know the answers to the following:

Is there some road map?
Some list of feature requests thought to be most useful for users?
Is there alternative Python project that is more active?

It might be that this project is done and is complete for its scope. And then people needing faster solutions or more complex models directly use integer programming, constraint solvers, SMT solvers, meta-heuristics, etc themselves.
I write this as I am surfing github and gitlab looking for some meaningful project to contribute to.

Hi Douglas,
I have this library very useful as it is.
We have managed to use it to do all our scheduling in a school I was
working at.
I am sure there are always ways to improve but it is pretty decent as is...
Just my 2 cents.
Le mar. 31 déc. 2019 à 00:17, Douglas Potter<EMAIL_ADDRESS>a
écrit :

This is useful project but its development seems to have stalled. I hope
this project can continue to improve. Does anyone know the answers to the
following:

Is there some road map?
Some list of feature requests thought to be most useful for users?
Is there alternative Python project that is more active?

It might be that this project is done and is complete for its scope. And
then people needing faster solutions or more complex models directly use
integer programming, constraint solvers, SMT solvers, meta-heuristics, etc
themselves.
I write this as I am surfing github and gitlab looking for some meaningful
project to contribute to.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/timnon/pyschedule/issues/90?email_source=notifications&email_token=ABXH2IB6MCYTL4R4C4QPJPLQ3JQPDA5CNFSM4KBQHVS2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IDM74CA,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABXH2IDGTZG7O47KP6EEVNDQ3JQPDANCNFSM4KBQHVSQ
.

Hi, there has indeed been little progress during the last year on this project, mostly due to timing contraints. If anybody wants to contribute or even take over, please step forward!
The initial intention of this project was to build a simple frontend to multiple backends (different contraint programming solvers, integer programming formulations, heuristics). This turned out to be quite cumbersome, since the different backends have very different strengths and weaknesses, and to support all of these is just too much work. In the end, it turned out that using a time-indexed integer-programming formulation with lots of redundancy-removals covers a lot of small to medium-sized use-cases very well, and offers maximum flexibility. So over time, everything was build around this formulation.

@timnon I sympathize that this is a huge task. I think the choice "time-indexed integer-programming formulation" is a good one.
I don't know if I will contribute but I think it would help others understand if:

) the README.md was updated by changing this phrase "FINAL CAUTION: pyschedule is under active development, there might be non-backward-compatible changes." to reflect the current state and show that contributions are welcomed.

2.) some list of areas known to need the most improvement. (This is something deeper than a list of the current open issues and would require knowledge of the code base and history of issues.)
Finally, do you know of any organizations that could share their use cases?
Again excellent work.

Hi guys, I noticed that the project seems to be frozen since the last release. The last commit is from early 2019, more than 18 months ago. No problem anyway, I know that maintaining an open source project can be quite a tough work !
For my own needs, I created a fork at https://github.com/tpaviot/pyschedule, which includes, in an unordered list: continuous integration and testing at Azure, online tests at mybinder, pylint, python2 drop and py3k only support, draft for plotly plotter. I have no roadmap, I just add what I need when I need it. Please @timnon let me know if ever/when you come back to this repository, I may have some PR to submit.
At last, thank you for this amazing work.


Download zip not working on Safari 10.0

Known issue; I haven't found really easy workarounds yet.

still not working (Safari 10.0.3)

Download also fails using chrome v58 on android

this should work now on macOS


error on parsing the statement

try1: SELECT SaneProduct.pid, SaneProduct.category, allRatings.rating, allRatings.review_ct
FROM (SELECT * FROM saneRatings UNION SELECT * FROM Ratings1) allRatings,SaneProduct
WHERE allRatings.pid= SaneProduct.id and SaneProduct.category IN ('phone','TV') OR allRatings.rating > 4
saneRatings: a schema matching lens on Ratings2 with ratings1typed schema as target schema
ERROR: Unhandled Feature: Expression[Unknown]: SaneProduct.category IN ('phone', 'TV')
try2: SELECT SaneProduct.pid, SaneProduct.category, allRatings.rating, allRatings.review_ct
FROM (SELECT * FROM saneRatings UNION SELECT * FROM Ratings1) allRatings,SaneProduct
WHERE allRatings.pid= SaneProduct.id and SaneProduct.category = 'phone' OR allRatings.rating > 4
ERROR: Type Mismatch [Numeric]: found TString, but expected TFloat

A IN (x, y, ...) is SQL Syntactic sugar for: A = x OR A = y OR A = z
This should be a simple fix.  It only needs to be added to SQLToRA. convert(Expression)


Move analysis from .rmd to .R

Some analysis are still performed in the .Rmd file as written by Stats4Dev. I've started a migration to the 'R/analysis_climmob.R' file for clarity and to make easy future debugs.

Solved here https://github.com/agrobioinfoservices/ClimMob-analysis/commit/e6944e043c4ea4f979bfa0818547223ee299ceaf


gulp.watch not working properly on node v9.6.1

What were you expecting to happen?
Watch should always be triggered when file changes.
What actually happened?
Watch triggers only twice when file changes.
My procedure
1.Run gulp watch
2.edit build/js/index.js
3.check console message,
4.✅watch is triggered properly
5.edit build/js/index.js
6.check console message,
7.✅watch is triggered properly
8.edit build/js/index.js
9.check console message,
10.❌nothing happened
Any further editing for index.js will never trigger watch event again.
However,Editing on another file works,for example index2.js ,but after repeating 2 times, index2.js will not be able to trigger either.
What did I do
This only happens on node v9.6.1 in my case (just update today).
Then I tried switch to node/8.4.0 and nothing goes wrong any more,it works like a charm.
I also have tried plugin gulp-watch,same phenomenon was found.
My Code
gulp.task('watch', function() {
    gulp.watch(["src/**/*.css"],['concat']);
    gulp.watch(["src/xxx/html/**"], ['process-html']);
    gulp.watch(["build/**/*"], ['watch-build']);
});

What version of gulp are you using?
CLI version 3.9.1
Local version 3.9.1
What versions of npm and node are you using?
node v9.6.1

Could you be so kind and show us your 'watch-build' task?

Sure.
var b = browserify({
    entries : options.tscBuildPath + "/js/xxx.app.js",
    debug   : true,
    plugin  : [watchify]
});

gulp.task('browserify', function() {
    // console.log("start task:browserify");
    return b.bundle()
        .pipe(source(options.destPath + "/js/xxx.app.js"))
        .pipe(buffer())
        .pipe(gulp.dest('./'));
});

gulp.task('copy-build',function(){
    return gulp.src(
                [
                    options.tscBuildPath + "/js/xxx.common.js",
                    options.tscBuildPath + "/js/xxx.word.js"
                ]
            ).pipe(gulp.dest(options.destPath + "/js/"));
});

gulp.task('watch-build',function(done){
    console.log("start watch-build");
    gulpSequence('copy-build',"browserify")(done);
});

And,Having noticed my build task would have some influence to watch task somehow, I had tried doing nothing only console.log in copy-build.But it doesn't help, and stop doing console.log at exactly third time too.
The src looks like

gulp.task('watch-build',function(done){
    console.log("start watch-build");
    done();
});

Okay, I've created a little test setup. Maybe you can verify that it does not work on your machine / version :)
structure
|- package.json
|- gulpfile.js
|- dist/
   |- js/
      |- test.js
|- node_modules/

code
var gulp = require('gulp');

gulp.task('save-the-world', function() {
    console.log('Now saving the world');
});

gulp.task('watch', function(){
    gulp.watch('dist/**/*', ['save-the-world']);
});

Now to test it just change something in test.js. It worked for me - I hope to test the latest node version this afternoon.

em.. interesting.
This simple test does work on latest node version.
I have to figure out what's wrong in my version.

OK,I've tried my project again.I switched to latest node , and It works without changing anything .
So maybe ,It was the computer or OS that goes wrong.This issue should have nothing to do with node or gulp.
Thanks a lot , and I am sorry for posting this issue.


Derivative/extrapolation enhancement

Hi Caleb,
This pull request implements the same extrapolation behavior as in extrapolate_integral. Tests were added too. There is a minor change in plot_T_dependent_property to allow extrapolation during a the test_documentation_plots test (calculate_derivative allowed for extrapolation before, but does not anymore).
Thanks!

Really great work Yoel! I may change some of the internal details of plot_T_dependent_property still. Again, please let me know if you want this in a release.
Sincerely,
Caleb

Thanks! I did notice the tests ran slower due to using T_dependent_property_derivative in the plot_T_dependent_property method. I made some changes to reduce function calls there. I'm not sure what internal changes you are planning for plot_T_dependent_property, but I hope this helps!
I also added the algebraic solution for the integral extrapolation with some tests.
I'll go ahead and complete the merge.
Thanks again!


Q: Epochs processing order

Right our Epochs processing happens in two separate steps:

Detrending, baseline correction, offset application, decimation
Projection

Looking through the code, I think that sometimes (1) comes before (2), and other times, (2) comes before (1) (e.g., potentially depending on delayed projection. I think we need to pick and order and document it somewhere. I propose the order above is fine and we should use it. Is that okay @dengemann @agramfort?

I noticed this because I was trying to use delayed projection, and that caused me to look into how to decrease memory usage in epochs.average(), which caused me to look into how this is done... and it doesn't seem to be consistent.


Create camelCase.py

Type of change

[x] Added new project
[x] Bug fix
[x] New features
[ ] Enhanced documentation

Changes proposed in this pull request

Project name :

camelCase

Short description :

Converts cameCase to snake_case
Checklist

[x] The code is Python 3 compatible
[ ] I have added relevant comments in my file to make code understandable
[ ] I have added my project in a folder which includes all the necessary files

Spam


[v0.10] General regression on arm64

libuv 0.10.28 was building and passing all tests on arm64 debian autobuilders (last tried on 2015-02-25).
With the latest upload of 0.10.36, the testsuite is now failing in multiple places (~150 tests failing):

https://buildd.debian.org/status/fetch.php?pkg=libuv&arch=arm64&ver=0.10.36-1&stamp=1430172460
https://buildd.debian.org/status/fetch.php?pkg=libuv&arch=arm64&ver=0.10.36-1&stamp=1430050989

This regression seems limited to arm64, as other arch are built fine.
Can anybody try on a different aarch64 machine, to confirm the regression?

Hi @lucab can you try applying this patch? https://github.com/libuv/libuv/commit/1d8332f7e6afb43daa08d643166c9e0cbdfa51a2
It made it to v1.x, but wasn't backported to v0.10. If tests pass I'll apply it there as well.

I'll do as soon as I've access to the arm64 porterbox, thanks for the hint.

[% 100|+ 197|-   0|T   0|S   0]: Done.

All tests passing on arm64 with above patch. Please backport, I'll wait for it in the next release.

Landed the backport in 8f931a2.


Look into cacert issues with Forcepoint proxy service

Description
After the latest cacert file update it seems that Forcepoint proxy service was removed.
We need to look into it and see if we need to add the cert to cacert file or make it possible for users to add their specific certs to the cert chain manually (in case we decide that it's not ok to add this cert to our cacert file)

just linking the case for reference: case link 🔗

There is was no Forcepoint cert in the previous cacert file. The issue is somehow connected to some other cert that Forcepoint is using.
Until we figure this out the current workaround for the user is to manually copy the cert into the cacert file. Sadly they have to do it on after every update.


Esprima demo interface seems to be broken

Steps to reproduce

Go to https://esprima.org/demo/parse.html#
The cursor location seems to be incorrect while editing the live source code area.

@goelayu Any information on the web browser and operating system used to reproduce this issue?

@ariya
I made a gif to preview this issue:

The editor in the demo is github.com/Microsoft/monaco-editor, so likely it's related to that.
Can you try with the online Monaco demo? Does it behave differently with another browser?

The editor in the demo is github.com/Microsoft/monaco-editor, so likely it's related to that.
Can you try with the online Monaco demo? Does it behave differently with another browser?

Yes, I have tried again to make sure I only see the problem in the Esprima demo.
And I use Monaco Editor a lot in my work, maybe I can help you?
But I don't actually see the original code for the demo in this repository, could you point me to it?

And I use Monaco Editor a lot in my work, maybe I can help you?
But I don't actually see the original code for the demo in this repository, could you point me to it?

Check the gh-branches. Maybe we just need an updated Monaco editor, and if you can help with that, it'll be fantastic!


Flesh out project organization markdown

Re organize so we start with a description of the dataset and give some biological context

paper: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6667181/

remove introduction to chipseq content
Have them copy over a project directory instead of mkdir
Keep the text and bullets point related to data management as long as it doesn't overlap with pre-reading content


[Bug] Spark connection level engines didn't be closed after all job finished

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Search before asking

[X] I have searched in the issues and found no similar issues.

Describe the bug
Env is HDP3.1.4, Apache spark3.2.3, Kyuubi 1.7.0
Some ETL task didn't finish after all its sql statments finish. I'm using connection level spark engine, I found that the application on yarn didn't go to SUCCEEDED state until kyuubi.session.idle.timeout happen.
Especially，I found a task only have a create table if not exists statment ，the statment execute cost few millisecond, but the yarn application keep alive 2 hour.

please use ecaced83-15d4-4ac6-9095-9e57bfa9e353 as key to search the log.
Affects Version(s)
1.7.0
Kyuubi Server Log Output
cat kyuubi-bigtop-org.apache.kyuubi.server.KyuubiServer-worker-10-0-xxx-24.out | grep ecaced83-15d4-4ac6-9095-9e57bfa9e353
2023-03-24 01:33:04.942 INFO org.apache.kyuubi.operation.log.OperationLog: Creating operation log file /data/soft/kyuubi/apache-kyuubi-1.7.0-bin/work/server_operation_logs/ecaced83-15d4-4ac6-9095-9e57bfa9e353/4846034a-c3d8-42b2-9e2b-d44061b70b61
2023-03-24 01:33:04.942 INFO org.apache.kyuubi.session.KyuubiSessionManager: x's session with SessionHandle [ecaced83-15d4-4ac6-9095-9e57bfa9e353] is opened, current opening sessions 36
        --conf spark.kyuubi.ha.engine.ref.id=ecaced83-15d4-4ac6-9095-9e57bfa9e353 \
        --conf spark.kyuubi.ha.namespace=/kyuubi_1.7.0_CONNECTION_SPARK_SQL/x/ecaced83-15d4-4ac6-9095-9e57bfa9e353 \
        --conf spark.app.name=kyuubi_CONNECTION_SPARK_SQL_x_ecaced83-15d4-4ac6-9095-9e57bfa9e353 \
        --conf spark.yarn.tags=KYUUBI,ecaced83-15d4-4ac6-9095-9e57bfa9e353 \
2023-03-24 01:33:05.009 INFO org.apache.kyuubi.session.KyuubiSessionImpl: [x:10.0.x.x] SessionHandle [ecaced83-15d4-4ac6-9095-9e57bfa9e353] - Starting to wait the launch engine operation finished
2023-03-24 01:33:26.656 INFO org.apache.kyuubi.ha.client.zookeeper.ZookeeperDiscoveryClient: Get service instance:worker-10-0-161-23:46425 engine id:application_1677295542968_70353 and version:1.7.0 under /kyuubi_1.7.0_CONNECTION_SPARK_SQL/x/ecaced83-15d4-4ac6-9095-9e57bfa9e353
2023-03-24 01:33:26.945 INFO org.apache.kyuubi.session.KyuubiSessionImpl: [x:10.0.x.x] SessionHandle [ecaced83-15d4-4ac6-9095-9e57bfa9e353] - Connected to engine [worker-10-0-161-23:46425]/[application_1677295542968_70353] with SessionHandle [ecaced83-15d4-4ac6-9095-9e57bfa9e353]]
2023-03-24 01:33:26.948 INFO org.apache.kyuubi.session.KyuubiSessionImpl: [x:10.0.x.x] SessionHandle [ecaced83-15d4-4ac6-9095-9e57bfa9e353] - Engine has been launched, elapsed time: 21 s
2023-03-24 01:33:27.253 INFO org.apache.kyuubi.operation.log.OperationLog: Creating operation log file /data/soft/kyuubi/apache-kyuubi-1.7.0-bin/work/server_operation_logs/ecaced83-15d4-4ac6-9095-9e57bfa9e353/1baf73fb-53ec-4d60-85bf-7decf558bea4
2023-03-24 03:33:17.848 INFO org.apache.kyuubi.server.KyuubiTBinaryFrontendService: Received request of closing SessionHandle [ecaced83-15d4-4ac6-9095-9e57bfa9e353]
2023-03-24 03:33:17.848 INFO org.apache.kyuubi.session.KyuubiSessionManager: x's session with SessionHandle [ecaced83-15d4-4ac6-9095-9e57bfa9e353] is closed, current opening sessions 37
2023-03-24 03:33:20.116 INFO org.apache.kyuubi.server.KyuubiTBinaryFrontendService: Finished closing SessionHandle [ecaced83-15d4-4ac6-9095-9e57bfa9e353]

Kyuubi Engine Log Output
23/03/24 01:33:27 INFO operation.ExecuteStatement:
           Spark application name: kyuubi_CONNECTION_SPARK_SQL_x_ecaced83-15d4-4ac6-9095-9e57bfa9e353
                 application ID: application_1677295542968_70353
                 application web UI: http://master-10-0-x-x:8088/proxy/application_1677295542968_70353,http://master-10-0-x-x:8088/proxy/application_1677295542968_70353
                 master: yarn
                 deploy mode: cluster
                 version: 3.2.3
           Start time: 2023-03-24T01:33:11.627
           User: x
23/03/24 01:33:27 WARN conf.HiveConf: HiveConf of name hive.metastore.uri.selection does not exist
23/03/24 01:33:27 WARN conf.HiveConf: HiveConf of name hive.tez.cartesian-product.enabled does not exist
23/03/24 01:33:27 INFO sqlstd.SQLStdHiveAccessController: Created SQLStdHiveAccessController for session context : HiveAuthzSessionContext [sessionString=2e31a169-63e2-48d2-b061-a644e26b5330, clientType=HIVECLI]
23/03/24 01:33:27 WARN session.SessionState: METASTORE_FILTER_HOOK will be ignored, since hive.security.authorization.manager is set to instance of HiveAuthorizerFactory.
23/03/24 01:33:27 INFO metastore.HiveMetaStoreClient: Mestastore configuration metastore.filter.hook changed from org.apache.hadoop.hive.metastore.DefaultMetaStoreFilterHookImpl to org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook
23/03/24 01:33:27 INFO metastore.HiveMetaStoreClient: Closed a connection to metastore, current connections: 0
23/03/24 01:33:27 INFO metastore.HiveMetaStoreClient: Trying to connect to metastore with URI thrift://worker-10-0-161-20:9083
23/03/24 01:33:27 INFO metastore.HiveMetaStoreClient: Opened a connection to metastore, current connections: 1
23/03/24 01:33:27 INFO metastore.HiveMetaStoreClient: Connected to metastore.
23/03/24 01:33:27 INFO metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=x (auth:SIMPLE) retries=1 delay=5 lifetime=0
23/03/24 01:33:27 INFO metastore.HiveMetaStoreClient: Trying to connect to metastore with URI thrift://worker-10-0-161-20:9083
23/03/24 01:33:27 INFO metastore.HiveMetaStoreClient: Opened a connection to metastore, current connections: 2
23/03/24 01:33:27 INFO metastore.HiveMetaStoreClient: Connected to metastore.
23/03/24 01:33:27 INFO metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=x (auth:SIMPLE) retries=1 delay=5 lifetime=0
23/03/24 01:33:27 INFO operation.ExecuteStatement: Execute in full collect mode
23/03/24 01:33:28 INFO operation.ExecuteStatement: Processing x's query[7dcbddbb-da82-42e6-9c08-033acf2d2b58]: RUNNING_STATE -> FINISHED_STATE, time taken: 0.676 seconds
23/03/24 01:33:56 INFO cluster.YarnClusterSchedulerBackend: Requesting to kill executor(s) 1
23/03/24 01:33:56 INFO cluster.YarnClusterSchedulerBackend: Actual list of executor(s) to be killed is 1
23/03/24 01:33:56 INFO yarn.ApplicationMaster$AMEndpoint: Driver requested to kill executor(s) 1.
23/03/24 01:33:56 INFO yarn.YarnAllocator: Resource profile 0 doesn't exist, adding it
23/03/24 01:33:56 INFO spark.ExecutorAllocationManager: Executors 1 removed due to idle timeout.
23/03/24 01:33:56 INFO cluster.YarnSchedulerBackend$YarnDriverEndpoint: Disabling executor 1.
23/03/24 01:33:56 INFO scheduler.DAGScheduler: Executor lost: 1 (epoch 0)
23/03/24 01:33:56 INFO cluster.YarnClusterScheduler: Executor 1 on worker-10-0-161-22 killed by driver.
23/03/24 01:33:56 INFO dynalloc.ExecutorMonitor: Executor 1 is removed. Remove reason statistics: (gracefully decommissioned: 0, decommision unfinished: 0, driver killed: 1, unexpectedly exited: 0).
23/03/24 01:33:56 INFO storage.BlockManagerMasterEndpoint: Trying to remove executor 1 from BlockManagerMaster.
23/03/24 01:33:56 INFO storage.BlockManagerMasterEndpoint: Removing block manager BlockManagerId(1, worker-10-0-161-22, 35597, None)
23/03/24 01:33:56 INFO storage.BlockManagerMaster: Removed 1 successfully in removeExecutor
23/03/24 01:37:13 INFO storage.BlockManagerInfo: Removed broadcast_0_piece0 on worker-10-0-161-23:46596 in memory (size: 3.5 KiB, free: 2004.6 MiB)
23/03/24 03:33:20 INFO spark.SparkTBinaryFrontendService: Received request of closing SessionHandle [ecaced83-15d4-4ac6-9095-9e57bfa9e353]
23/03/24 03:33:20 INFO session.SparkSQLSessionManager: x's session with SessionHandle [ecaced83-15d4-4ac6-9095-9e57bfa9e353] is closed, current opening sessions 0
23/03/24 03:33:20 INFO session.SparkSQLSessionManager: Session stopped due to shared level is Connection.
23/03/24 03:33:20 INFO spark.SparkSQLEngine: Service: [SparkTBinaryFrontend] is stopping.
23/03/24 03:33:20 INFO spark.SparkTBinaryFrontendService: Service: [EngineServiceDiscovery] is stopping.
23/03/24 03:33:20 WARN zookeeper.ZookeeperDiscoveryClient: This Kyuubi instance worker-10-0-161-23:46425 is now de-registered from ZooKeeper. The server will be shut down after the last client session completes.
23/03/24 03:33:20 INFO client.EngineServiceDiscovery: Clean up discovery service due to this is connection share level.
23/03/24 03:33:20 INFO imps.CuratorFrameworkImpl: backgroundOperationsLoop exiting
23/03/24 03:33:20 INFO zookeeper.ZooKeeper: Session: 0x38377a2b5e9d1df closed
23/03/24 03:33:20 INFO zookeeper.ClientCnxn: EventThread shut down for session: 0x38377a2b5e9d1df
23/03/24 03:33:20 INFO client.EngineServiceDiscovery: Service[EngineServiceDiscovery] is stopped.
23/03/24 03:33:20 INFO spark.SparkTBinaryFrontendService: Service[SparkTBinaryFrontend] is stopped.
23/03/24 03:33:20 INFO spark.SparkTBinaryFrontendService: SparkTBinaryFrontend has stopped
23/03/24 03:33:20 INFO spark.SparkSQLEngine: Service: [SparkSQLBackendService] is stopping.
23/03/24 03:33:20 INFO spark.SparkSQLBackendService: Service: [SparkSQLSessionManager] is stopping.
23/03/24 03:33:20 INFO session.SparkSQLSessionManager: Service: [SparkSQLOperationManager] is stopping.
23/03/24 03:33:20 INFO operation.SparkSQLOperationManager: Service[SparkSQLOperationManager] is stopped.
23/03/24 03:33:20 INFO session.SparkSQLSessionManager: Service[SparkSQLSessionManager] is stopped.
23/03/24 03:33:20 INFO spark.SparkSQLBackendService: Service[SparkSQLBackendService] is stopped.
23/03/24 03:33:20 INFO spark.SparkSQLEngine: Service[SparkSQLEngine] is stopped.
23/03/24 03:33:20 INFO spark.SparkTBinaryFrontendService: Finished closing SessionHandle [ecaced83-15d4-4ac6-9095-9e57bfa9e353]

Kyuubi Server Configurations
kyuubi.ha.addresses   xxx
kyuubi.authentication  LDAP
kyuubi.authentication.ldap.base.dn ou=People,dc=xx,dc=com
kyuubi.authentication.ldap.url  ldap://xxx:xxx/
kyuubi.authentication.ldap.guidKey cn
kyuubi.session.engine.initialize.timeout PT10M
kyuubi.session.idle.timeout  PT2H

Kyuubi Engine Configurations
No response
Additional context
No response
Are you willing to submit PR?

[ ] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix.
[ ] No. I cannot submit a PR at this time.

I've tried both spark'beeline and kyuubi's beeline against v1.7.0, but can not reproduce your issue. You may check whether your client-side has something blocking your connection to be closed

didn't go to SUCCEEDED state until kyuubi.session.idle.timeout happen

Typically, it is because the client does not close the session(JDBC connection) properly.

Thank you for your attention. Actually the ETL task script is very simple , and I can not reproduce this issue too.
ETL task script is :
/data/soft/spark3/bin/beeline -n xxx -p xxx -u  'jdbc:hive2://worker-10-0-xxx:2181,worker-10-0-xxx:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=kyuubi;?kyuubi.engine.share.level=connection;' -f '/data/scripts/datax/hdp-doc/scripts/gdwd/01.ods/ods_key_customer_touch_create.sql'
and the the sql file has only one sql :
create table if not exists ods.ods__touch
(
    ...
    update_time         timestamp   comment '更新时间',
    etltime             timestamp   comment 'etl时间'
)
stored as orc;

And from the DolphinScheduler's task history I can see the task's script not be changed for a long time , also the sql file submit to kyuubi not be changed for a long time by seach git history .

from the picture above , the colum '工作流实例' has value like 'ods_key_customer_touch-2-20230324010001985' which 'ods_key_customer_touch' is process name and '2' is  process version. This process runs once a day using version 2 and has been running for a long time.

It may need the jstack information of the beeline process or the one who calls it during such a long time pause

I have discovered another situation. The etl task also use spark's beeline to submit a sql file to kyuubi with connection level. This Sql file have three sql , The first sql is also a statment like create table if not exists ... stored as orc; After this statment  finished , the task was stuck  until kyuubi.operation.idle.timeout happen.

the task was stuck until kyuubi.operation.idle.timeout happen.

add beeline out put :

I've upgrade to kyuubi 1.7.0 about two weeks, These kind of issue happen first time. And I found the log file : ${KYUUBI_HOME}/logs/kyuubi-${kyuubi user}-org.apache.kyuubi.server.KyuubiServer-${kyuubi server host name}.out is much larger than other versions（two weeks, each server generated approximately 3GB), is this will cause some performance issue？

@lordk911 There is a known issue which may produce unnecessary logs https://github.com/apache/kyuubi/pull/4527

I've upgrade to kyuubi 1.7.0 about two weeks,

May I ask which version you used before? There may also an upgrade for log4j2 from log4j1.

I've upgrade to kyuubi 1.7.0 about two weeks,

May I ask which version you used before? There may also an upgrade for log4j2 from log4j1.

1.4.1 -> 1.6.1 -> 1.7.0

I almost make sure, all this two kind of issue happened only on one kyuubi server(I have 3 kyuubi server). And Operation OperationHandle [1ff4d7ea-dfa5-4be8-b375-9bf8fdfb3067] is timed-out and will be closed this kind of issue be the most part.

After stopped the kyuubi server which met these kind of issue, everything goes well.
Close this issue, because it's hard to reproduce, And I can't provide any more clues at the moment.

@lordk911
@yaooqinn
We are also getting the exact issue in our prod deployment. We also see this happening in 1 of the kyuubi server only out of 4. On checking the report.json metric file inside that server we get_operation_status having mean of 2600 millisecond.  have attached metric report for this.
report-4.json
basically this operation was taking loads of time and seems to get stuck a lot. This could be something related to high user load i guess. We have seen this a few times now. We investigate , but find only restarting as the solution mostly.
I was thinking of decreasing the kyuubi.operation.idle.timeout to a very small value maybe few minute. Will that have any wrong effects on my queries?  Some of our queries to run for hours.
Below is the kyuubi server stacktrace which we see has high cpu time and elapsed time. I could not co relate this guid with a string operation handle id, as i did not know how to easily convert and seemed like to much effort.
 "async-request-executor-TSessionHandle(sessionId:THandleIdentifier(guid:66 CD F8 DE F2 CC 44 72 86 96 8F E7 97 DC EB 00, secret:C2 EE 5B 97 3E A0 41 FC AC 16 9B D7 08 ED 8F 38)): Thread-54226" #54226 daemon prio=5 os_prio=0 cpu=290468489.96ms elapsed=707467.31s tid=0x00007f3fe8018000 nid=0x7c807 runnable  [0x00007f422b7fa000]    java.lang.Thread.State: RUNNABLE 	at<EMAIL_ADDRESS>Method) 	at<EMAIL_ADDRESS>Source) 	at<EMAIL_ADDRESS>Source) 	at<EMAIL_ADDRESS>Source) 	at<EMAIL_ADDRESS>Source) 	at<EMAIL_ADDRESS>Source) 	at<EMAIL_ADDRESS>Source) 	- locked <0x0000000083be8ad0> (a java.io.BufferedInputStream) 	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127) 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) 	at org.apache.thrift.transport.TSaslTransport.readLength(TSaslTransport.java:376) 	at org.apache.thrift.transport.TSaslTransport.readFrame(TSaslTransport.java:453) 	at org.apache.thrift.transport.TSaslTransport.read(TSaslTransport.java:435) 	at org.apache.thrift.transport.TSaslClientTransport.read(TSaslClientTransport.java:37) 	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86) 	at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429) 	at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318) 	at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219) 	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77) 	at org.apache.hive.service.rpc.thrift.TCLIService$Client.recv_GetOperationStatus(TCLIService.java:475) 	at org.apache.hive.service.rpc.thrift.TCLIService$Client.GetOperationStatus(TCLIService.java:462) 	at org.apache.kyuubi.client.KyuubiSyncThriftClient.$anonfun$getOperationStatus$1(KyuubiSyncThriftClient.scala:369) 	at org.apache.kyuubi.client.KyuubiSyncThriftClient$$Lambda$943/0x0000000840a79040.apply(Unknown Source) 	at org.apache.kyuubi.client.KyuubiSyncThriftClient.$anonfun$withLockAcquiredAsyncRequest$2(KyuubiSyncThriftClient.scala:142) 	at org.apache.kyuubi.client.KyuubiSyncThriftClient$$Lambda$938/0x0000000840a6f040.call(Unknown Source) 	at<EMAIL_ADDRESS>Source) 	at<EMAIL_ADDRESS>Source) 	at<EMAIL_ADDRESS>Source) 	at<EMAIL_ADDRESS>Source) 	at<EMAIL_ADDRESS>Source)


added Calculator in C

Calculator in C
Calculator in C with more functionalities like square-root, nth root, quadratic roots etc.
Fixes # (issue)
Type of change
Please delete options that are not relevant.

[x] New feature (non-breaking change which adds functionality)

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have made corresponding changes to the documentation
[x] My changes generates no new warnings
[x] New and existing unit tests pass locally with my changes

I was about to merge this PR but found this:
https://github.com/b-samarth/Beginners-Showcase/blob/main/C_calculator/Calculator.c
As I said in the guidelines too, Plagiarism not at all accepted. Pushing same code everywhere is not a  solution to get +1 on the counter.


Circular type imports wreck typechecking for unions.

Minimal example:
one.js
/* @flow */
import type { Two } from './two';
export type One = 1;

two.js
/* @flow */
import type { One } from './one';
export type Two = { one: One } | 2;
const x: Two = 3;

The expected behaviour is that the line with const x does not typecheck. However, both with 0.38.0 and 0.37.4, it does erroneously pass the typechecker.
Furthermore, the typechecking catches the error correctly, if any one of these changes are made:

one.js does not import two.js
type Two is defined without a union (i.e. = { one: One })
type Two does not use One (i.e. = { one: 1 } | 2)

As an aside, I could not find any information about circular type imports in the documentation or anywhere else. Are they actually allowed? (If not, then the expected behaviour would be an error about a circular import instead).
Thanks!

Thanks for making this minimal example.  I was just spending a long time trying to figure out what exact conditions were causing this problem in a very large codebase (had narrowed it down to circular imports but no further).
The problem is still present in 0.46, and actually even worse than you example makes out.  one.js doesn't have to import Two... any import statement against the file will silently break type checking.  This breaks:
one.js
/* @flow */
import {  } from './two';
export type One = 1;

two.js
/* @flow */
import type { One } from './one';
type Two = { one: One } | 2;
const x: Two = 3;

I was actually having this problem with intersections, not unions, but I've been unable to reproduce it from scratch.  It's along the same lines though - I can make it work again by breaking the circular chain, or writing the object type out in full without using an intersection.

As an aside, I could not find any information about circular type imports in the documentation or anywhere else. Are they actually allowed? (If not, then the expected behaviour would be an error about a circular import instead).

Any comment please?

@avikchaudhuri nice, thank you! Am I right in assuming circular imports should "just work" now, or are there any constraints?


#2850 - Accent characters getting translated during Institution file upload

Files uploaded to SIMS (buffer.toString)
As stated in the nodejs documentation when the encoding is set to ASCII the most significant bit will be unset (to ensure the 7bit characteristic from ASCII) causing the characters to be changed, as per the example below.

The same would not happen if latin1(ISO-8859-1) is used as stated by the documentation and tested during the file uploads.
The requirement for files integrations, via upload or SFTP, is to use a single-byte encoding, and since UTF-8 is multibyte, latin1(ISO-8859-1) seems to be the only way to proceed.
One E2E test was changed to ensure that the translation issue would be captured.
Files generated to the SFTP (Buffer.from)
While creating the files to be uploaded to the SFTP both ascii and latin1 resulted in the same bytes.

Char(áéí)
Encoding

UTF-8
195, 161, 195, 169, 195, 173

ASCII
225, 233, 237

Latin1(ISO8859-1)
225, 233, 237

Sample file generated with UTF8

Sample file generated with ASCII and Latin1(ISO8859-1) where the same result was produced

Sample file in latin1 using UTF8 characters outside ASCII extended

From the student profile below.

Closed due to a change in the approach that will be done in a new ticket. Details in the associated ticket comments (#2850).


Updated AccumuloITBase.java to enforce a default timeout

Updated default TimeoutSeconds() method to be 10 minutes

@MuazmaZ Were you able to run the ITs with this timeout? Did you notice any particularly problematic tests that might need to be tweaked as a result of this change?

@ctubbsii I didn't see any test, in particular, to be problematic

I'm attempting mvn verify with this change on my laptop now (with a timeout.factor). But I haven't run a verify on this laptop before, so I don't know whether this would typically succeed.

I would be happy to merge this in now, but I'm seeing other failures currently on the master branch, and would like to see those resolved first.

I can run the ITs for this branch.

I can run the ITs for this branch.

@keith-turner Can you clarify whether you mean:

You will run the ITs on this branch to see if this is okay to merge, or
You have run the ITs on this branch and you think it's okay to merge now.

@ctubbsii I have the ITs running now for this branch merged into master.  If it all is well I can merge.

Saw the following failures
[ERROR] Failures:
[ERROR]   UnorderedWorkAssignerReplicationIT.dataWasReplicatedToThePeer:288 Drain did not finish within 30 seconds
[ERROR] Errors:
[ERROR] org.apache.accumulo.test.MetaRecoveryIT.test(org.apache.accumulo.test.MetaRecoveryIT)
[ERROR]   Run 1: MetaRecoveryIT.test:97 » TestTimedOut test timed out after 240 seconds
[ERROR]   Run 2: MetaRecoveryIT.test »  Appears to be stuck in thread Time-limited test-SendThr...
[INFO]
[ERROR] org.apache.accumulo.test.functional.MetadataMaxFilesIT.test(org.apache.accumulo.test.functional.MetadataMaxFilesIT)
[ERROR]   Run 1: MetadataMaxFilesIT.test:80 » TestTimedOut test timed out after 300 seconds
[ERROR]   Run 2: MetadataMaxFilesIT.test »  Appears to be stuck in thread Time-limited test-Sen...
[INFO]
[ERROR] org.apache.accumulo.test.replication.UnorderedWorkAssignerReplicationIT.dataWasReplicatedToThePeerWithoutDrain(org.apache.accumulo.test.replication.UnorderedWorkAssignerReplicationIT)
[ERROR]   Run 1: UnorderedWorkAssignerReplicationIT.dataWasReplicatedToThePeerWithoutDrain:601 » TestTimedOut
[ERROR]   Run 2: UnorderedWorkAssignerReplicationIT.dataWasReplicatedToThePeerWithoutDrain »  A...
[INFO]
[INFO]
[ERROR] Tests run: 610, Failures: 1, Errors: 3, Skipped: 1

I am currently looking into these test failures.  Rerunning the failed test on this branch and in master.

I ran MetadataMaxFilesIT and MetaRecoveryIT a few times in this branch and the master branch.  For both branches the test would fail sometimes and succeed sometimes.  So those test seem to be a bit flaky even w/o this change.

@MuazmaZ  thanks for the contribution.  If you would like to add yourself as a contributor to the Accumulo people page please submit a PR.

Thanks @keith-turner @ctubbsii and @billierinaldi for resolving this. I also ran maven verify and got no errors.


Docs > API Reference add addOnWhichKey

Added the description of the component's addOnWhichKey property in the Docs > API Reference as it was not there.
I noticed a typo in my previous Pull Request, so I am submitting a new Pull Request to correct it.
Reference:
#48

Please, rewrite your PR just for the documentation here.
Nothing more.


feat(congregations): use pagination when navigating public talks

:tada: This PR is included in version 2.28.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


loading language files one by one

Is there any way of  language file on demand(not all languages files but only the language that the user chooses)? Because if I have 20 language files loading all will be a mess
For changing the language is there any simple function to call with the parameter the chosen language or I am obliged to initialize every time the language changes. If there is can it support loading file one by one following the choice of the user?
What is the use of name parameter in the initialization; name   | Partial name (or names) of files representing resource bundles (eg, ‘Messages’ or ['Msg1','Msg2']). Defaults to 'Messages' | Optional String or String[] |

I can answer your first question. You can add an languages.json file with the same path as the properties which mentioned in readme file.
In the languages.json, you can list all the language you support. However , if there is the language which you don't support , it will just get a default properties only as a response.

Language files are only loaded on demand. If you have a browser locale of EN_US the plugin will try messages.properties, then messages_en.properties, then messages_en_us.properties, layering the translations on as it goes. If the caller then switches locale, it will try the new locale. They're not all pulled straight away. Is this what you meant?


Fix QR code URL not working on the Google Cloud Storage buckets

What

Remove extra "/" character when building the QR code URL

How

When building the targetUrl, remove the extra "/" as it is unnecessary
Tested with on 3 different environments: localhost, map.mapsindoors.com and also storage.googleapis.com/mi-map-latest/index.html

@ammapspeople Yess, I'll send that in a DM to you.


Update howto-vm-sign-in-azure-ad-windows.md - managed identity under oid and not appid

Based on my tests, the access token contains the managed identity of the VM under oid and not appid as originally documented.

@cbezenco : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@SanDeo-MSFT can you please review this proposal is correct?

#sign-off


Increase the allowed size of Post.object_uri and Post.url from 500 ch…

…aracters to a more reasonable 2048. See https://stackoverflow.com/questions/417142/what-is-the-maximum-length-of-a-url-in-different-browsers.

I believe this technically needs a migration, even though PostgreSQL won't actually change anything?

Actually nevermind, I'll bundle the migration into the one I want to do for TimelineEvent.published


Change the filter 'nor' operator on the loopback 'neq' analogue

By documentation (https://docs.strongloop.com/display/public/LB/Where+filter), loopback doesn't have 'nor' operator.
I think, the 'nor' operator should be changed on 'neq'. It will more conform to loopback api

I admit that this may sound a bit ignorant, but could you help point out for me with a quick sample snippet, how can nor can be used with this connector right now?
And where do you see the code for it sitting? I don't remember writing it explicitly but its been several months and mind is all over the place.

I learn your code more carefully. It'll be cool if you just add 'neq' function support. I added a pull request to my issue. I think, it just only need some tests

I don't have an ES instance or a meaningful dataset that I can point it to for testing right now ... thinking ...


New pricing model pushes away indie hackers and project at the beginning of their life, bye! 👋🏻

You actually made it convenient to go through the burden of self deploying or go to TypeSense.
I really wonder how it is going to play out for you in the long term, sure thing indie hackers or small experiments will not approach meilisearch anymore, and when (and if) their product will scale, probably they won't be here.
Sure thing next month, or next 4 months will be great for you guys in terms of revenues, but then, 1 year from now? I genuinely wish you the best but I wouldn't bet money on this being a good long-term decision.
My use case is a social app with 8k DAU today and stable last 6 weeks with peaks of 10k/12k, I only use search in the signup phase when the user have to pick the school they are from. I am way below the 30€ per month at the moment, BUT in September I would have implemented user search via name, username, surname, etc. (at the moment is exact username match only, so I lookup my collections in Firebase) and the most natural thing would have been to do it via meilisearch, but as I will be switching now (since July and August i expect low traffic and does not make business sense to pay 30€ per month) then I'll also do user search with something else.
Wish you the best, was fun till it lasted!

Hi @mariopepe,
Thanks for reaching out and for your valuable feedback. And thank you for providing your use-case and additional context.
We appreciate the pricing change on Meilisearch Cloud isn't ideal for all users.
As we continue to invest in our infrastructure and feature improvements in our Cloud offering the pricing change was a necessity at this time.
Meilisearch remains proudly Open Source and free when self hosting if the Cloud offering isn't right for you at this time.

I'll close out this issue due to inactivity. If you'd like to continue the conversation further please feel free to re-open it. Thanks for the feedback.


Generator: Fixes for external calling

This adds some small changes/fixes for using lxd-generate from outside the lxd project.

When the -d or --database flag is passed, the keyword will now properly be used as a package receiver. The actual package path is imported via goimports.
The above flag will now no longer affect other generated helpers. The generator will assume that any necessary helper is defined within the package from which the generator was called. Instead, the -d flag will now only work on the globally defined prepared statement helpers: Stmt and RegisterStmt. It makes sense to have these be referenced since there should only be one collection of statements prepared when the database first opens.
A table name override flag has been added as well: Sometimes the generator just isn't too smart, or language is just too inconsistent to generate sensible table names. This lets us override the generated table name with one defined by us.

@tomponline nope, this should have no effect on LXD.

@stgraber That should fix the static analysis issues.

jenkins: test this please


ginkgo tests: 'port -l' is racy

test/e2e/port_test.go runs nginx but doesn't actually wait for it to start before running podman port -l.  Panic [1.258 seconds]
[+0335s] Podman port
[+0335s] /var/tmp/go/src/github.com/containers/libpod/test/e2e/port_test.go:15
[+0335s]   podman port  -l nginx [It]
[+0335s]   /var/tmp/go/src/github.com/containers/libpod/test/e2e/port_test.go:51
[+0335s]
[+0335s]   Test Panicked
[+0335s]   runtime error: index out of range
[+0335s]   /usr/lib/golang/src/runtime/panic.go:513
[+0335s]
[+0335s]   Full Stack Trace
[+0335s]   	/usr/lib/golang/src/runtime/panic.go:513 +0x1b9
[+0335s]   github.com/containers/libpod/test/e2e.glob..func47.5()
[+0335s]   	/var/tmp/go/src/github.com/containers/libpod/test/e2e/port_test.go:59 +0x746
[+0335s]   github.com/containers/libpod/vendor/github.com/onsi/ginkgo/internal/leafnodes.(*runner).runSync(0xc000f67ce0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
[+0335s]   	/var/tmp/go/src/github.com/containers/libpod/test/e2e/common_test.go:88 +0xe0
[+0335s]   testing.tRunner(0xc00013c900, 0x13be480)
[+0335s]   	/usr/lib/golang/src/testing/testing.go:827 +0xbf
[+0335s]   created by testing.(*T).Run
[+0335s]   	/usr/lib/golang/src/testing/testing.go:878 +0x35c
[+0335s]

This is obviously THE standard problem with run -d; the classic solution (run podman logs waiting for well-known string) doesn't work here because there's no log output. One possibility might be to repeat-loop the podman port -l until there's actual output. I'd love to submit a PR but I don't speak ginkgo.

This is basically what healthchecks are for, I think? Set one up and wait until it goes green?

Healthchecks?
I think this is a race in this specific test, though haven't looked at the code yet.

We can solve the race by setting up a healthcheck and looping until it goes green, which would indicate that it's ready to proceed

Though, honestly... Why do we need the container running? podman port just reads the configuration of the container.

Hell, why does it need to be nginx? We just need a podman create -t -i -p 80:80 alpine top or similar.

Some of the tests actually try to connect to the container.

All the other failures are in 'podman create'  Perhaps something bad broke there, and the 'podman port' panic's are just another symptom?

Why do we need the container running?

I don't think we do; but -P gets its ports from the container config, which means the image has to be pulled and read. Maybe the solution is to split it into podman create and then run?

Hm. Alright, this one's properly strange. It's not a run vs create issue (I'm still going to swap them over to using create for speed) - the container is definitely created, and that's all we need.
Instead, the issue seems to be that the container has no ports. This could be an issue with podman port or podman run with -P.

@cevich We're only seeing this flake on the system testing PR from @edsantiago right?

Alright, nevermind everything I said. It's as @edsantiago originally predicted - a race between podman run starting the container and podman port.
I assumed it wasn't because there is no technical reason we need the container to be running for port to work, but I ignored Docker compatability - Docker will only print the ports of running containers, and we duplicated that.

Thanks for finding the problem.
As for 'only seeing this', I'm going to go out on a limb and say I've only seen it on this specific CI run. All my CI runs fail, and all of them seem to be different failures, but I think I would've noticed the [Panic] if I'd seen it before.

What we really need here is a generic "wait for container to be ready" call in the Ginkgo machinery - something that uses some mechanism (events, maybe?) to ensure a container is running before proceeding.

s/in the Ginkgo machinery/in general for pretty much every situation where I've ever needed podman run -d/
hashtag maybeanideafor1.5?

Outside of the context of tests, we'd really recommend healthchecks (more complex, allows you to check that what's in the container is ready) or events (if you see a start event for the ID the container is running).

This is what I emailed you about last week with the idea of signals, which I agree is maybe more complex than needed.  The idea of a healthcheck could be one way to do it as well.  It would allow us to use podman function to verify readiness without having to use separate tooling.  The downside is that whatever the healthcheck uses (like curl) needs to be installed in the container.

fwiw, i self assigned this one and had planned to fix it Monday whenI return from PTO.  do you want/need it fixed faster?

I'll leave the faster thing up to @edsantiago
I really wish we had a better way of triggering stuff on events coming in... Might be worth grabbing Giuseppe and Dan and huddling about ways of doing that without killing performance.

(Writing the events thing because it's be super neat if we could easily hook into a single container and trigger things on specific events coming in - solves this easily)

No rush on my account. I'm thinking of spinning the CI roulette wheel again on my PR and seeing what comes up.


Vertical scrolling with a mouse is too fast, and it's not possible to configure the sensitivity

Related to https://github.com/atom/atom/issues/1666:

support/0cfc90ecceb611e39200c9d04f25774f
support/d6a39d02cc9811e386ac809b029b3fb9

When using the scroll wheel on my mouse, the editor jumps 8 lines at a time. Any way for me to change this configuration? I would prefer a smooth scroll one line at a time.

...

The scrolling is not smooth(it would be really nice if it was), and the sensitivity is not adjustable, for me (i have a g500 mouse, the mouse wheel has very little friction) and for me it scrolls to far many lines.

Some users are working around this by hacking on Atom, as described in this forum thread:

If anyone wants to hack this value to slow down scrolling, edit app/src/editor-view.js and find the mousewheel event listener (~line 877) and add a modifier into the delta assignment condition. For example, I edited line 890 to multiply it times .3
if (delta = (e.originalEvent.wheelDeltaY * .3)) {
Now the speed feels much more natural.

(I don't have a mouse so that I can verify any of this)

Where can I find this new option?

@soerendip - it's at Settings > Editor > Scroll Sensitivity (this issue is pretty old so the option isn't exactly as described above anymore).  If you have general questions about Atom in the future, there are a lot of options and resources:

User Documentation — The Atom Flight Manual
Developer Documentation — Atom API Documentation
FAQ — The Atom FAQ on Discuss
Message Board — Discuss, the official Atom and Electron message board
Chat — Join the Atom Slack team

On Discuss and in the Atom Slack team, there are a bunch of helpful community members that should be willing to point you in the right direction.


Add HOMEBREW_API_AUTO_UPDATE_SECS

This sets the default and allows customising how often we try to download files from the API.
This does not affect brew update as we want to always check every time on an explicit call.
@Bo98 @Rylan12 @carlocab Feel free to push directly to this PR and merge as desired. I will be unlikely to be able to make any more code changes here until Monday and would like to land this (or something like it) as soon as possible before then to allow it to bake.

Review period skipped due to critical label.

What is the importance of this? This just adds a concerning, vague print message during brew update

I'm in favour of nicer naming but HOMEBREW_TAP_REFRESH_SECS sounds like it applies to all taps, but it doesn't.

What is the importance of this? This just adds a concerning, vague print message during brew update

Are you referring to the "Downloading https://formulae.brew.sh/api/formula.json"?

I feel like our use of the word "API" for this stuff may be confusing and unclear what it means.

A bit too late now, I think, unless we're going to change this API naming everywhere.
Given it's downloading from a /api/ path: it seems appropriate.


Get-UiPathRobot : A task was canceled.

Getting below error when ran Get-UiPathRobot in powershell
Get-UiPathRobot : A task was canceled.
At line:1 char:1

Get-UiPathRobot | Format-Table

  + CategoryInfo          : NotSpecified: (:) [Get-UiPathRobot], TaskCanceledException
  + FullyQualifiedErrorId : System.Threading.Tasks.TaskCanceledException,UiPath.PowerShell.Cmdlets.GetRobot

This seems like a Orchestrator timeout on getting the robot list, not a library problem. If you have a large number of robots, add the -Paging argument to the cmdlet:
PS C:\>Get-UiPathRobot -Paging


Refactor DMA to make API more consistent across devices

tl;dr of this is that the peripheral driver is now called Dma, regardless of whether the underlying peripheral is PDMA or GDMA, and the constructor also now takes a (sometimes virtual) peripheral singleton in its constructor for all devices. The Dma struct is not longer split from peripherals.SYSTEM on devices with PDMA.

Code looks fine but needs a rebase

Doesn't the ESP32-S3 use PDMA for SDHOST(SDMMC)? So it needs to be able to create both types I think.

Doesn't the ESP32-S3 use PDMA for SDHOST(SDMMC)? So it needs to be able to create both types I think.

Thanks for pointing this out. I somehow missed that before - seems also SDIO on C6 comes with dedicated DMA. I am not really sure how we will handle those special cases - most probably those drivers need to implement things on their own especially because the structure of the DMA linked-list is quite different


Physiological derivatives BIDS proposal

Title
Physiological derivatives BIDS proposal
Leaders
Marie-Eve Picard
email<EMAIL_ADDRESS>Github Username: me-pic,
Discord Username: me_pic (Marie-Eve Picard (she/her)),
Collaborators
Mary Miedema, Github Username: m-miedema
Brainhack Global 2023 Event
BrainHack Vanderbilt
Project Description
Physiopy is a community dedicated to developing solutions for handling physiological data acquired in neuroimaging setups. Currently, physiopy supports and maintains phys2bids, a library that standardizes raw physiological data in accordance with the Brain Imaging Data Structure (BIDS). The community is also developing and managing other libraries like peakdet (processing and analysing), Phys2Denoise (producing physiological regressors for neuroimaging denoising), and PhysioQC (quality control). All those libraries produce a bunch of different derivatives. However, the derivatives structure and requirements for physiological data (i.e. cardiac, respiratory, blood pressure, etc.) are not yet covered by the BIDS specification. With this project, we are hoping to start a discussion around an eventual BIDS proposal on physiological derivatives.
This project does not require coding skills to join. If you are enthusiastic about physiological data, BIDS, both, or just a curious person, we’re happy to have you on board!
Link to project repository/sources
https://github.com/orgs/physiopy
Goals for Brainhack Global
Open discussion on how to BIDSify physiological data derivatives (i.e. preprocessed data, peaks, troughs, rvt, hrv, etc.)
Good first issues
Come and join the discussion!
https://github.com/orgs/physiopy/projects/3/
Communication channels
#bids_derivatives channel on https://discord.gg/GyeeVbYC
Skills

Physiological data: cardiac, respiratory, skin conductance etc. [all levels]
BIDS standards: intermediate/advanced
Data management: all levels
github: all levels
python: intermediate/advanced

Onboarding documentation
No response
What will participants learn?
Participants will learn about BIDS standards, and derivatives management.
Data to use
No response
Number of collaborators
1
Credit to collaborators
Contributors are welcome to participate in drafting the physiological derivatives BIDS Extension Proposal (BEP), and be listed among the authors.
Image

Type
data_management
Development status
0_concept_no_content
Topic
physiology
Tools
BIDS
Programming language
not_applicable
Modalities
other
Git skills
0_no_git_skills
Anything else?
No response
Things to do after the project is submitted and ready to review.

[ ] Add a comment below the main post of your issue saying: Hi @brainhackorg/project-monitors my project is ready!
[ ] Twitter-sized summary of your project pitch.

@me-pic
your project is displayed on the website

http://brainhack.org/global2023/projects/


release_calc_previous: Set minimum versions based on build-suggestions

We now have a mandate to ensure that z-stream upgrades are cumulative.
Overall it is also better for us to always offer and upgrade path that's
directly to the best version available. So rather than relying on a rolling
window of ten most recent z-streams and the two most recent previous minor
versions we'll use a static minimum versions and a block list sourced from OTA
managed content.
Prior to this change
$ ./doozer release:calc-previous --version 4.5.8
4.5.8,4.5.7,4.5.6,4.5.5,4.5.4,4.5.3,4.5.2,4.5.1,4.5.0-0.hotfix-2020-08-24-185832,4.4.20,4.4.19,4.4.18
After this change
$ ./doozer release:calc-previous --version 4.5.8
4.5.8,4.5.7,4.5.6,4.5.5,4.5.4,4.5.3,4.5.0-0.hotfix-2020-08-24-185832,4.4.20,4.4.19,4.4.18,4.4.17,4.4.16,4.4.15,4.4.14,4.4.13
Based on the following metadata
$ curl https://raw.githubusercontent.com/sdodson/cincinnati-graph-data/suggestions/build-suggestions/4.5.yaml
---
default:
  minor_floor: 4.4.13
  minor_block_list: []
  z_floor: 4.5.3
  z_block_list: []
s390x:
  minor_floor: 4.4.24
  minor_block_list: []
  z_floor: 4.5.3
  z_block_list: []

Depends on https://github.com/openshift/cincinnati-graph-data/pull/425

Codecov Report

Merging #277 into master will decrease coverage by 0.16%.
The diff coverage is 9.67%.

@@            Coverage Diff             @@
##           master     #277      +/-   ##
==========================================
- Coverage   27.42%   27.26%   -0.17%
==========================================
  Files          30       30
  Lines        6366     6433      +67
  Branches     1282     1301      +19
==========================================
+ Hits         1746     1754       +8
- Misses       4560     4617      +57
- Partials       60       62       +2

Impacted Files
Coverage Δ

doozerlib/cli/__main__.py
0.00% <0.00%> (ø)

doozerlib/distgit.py
32.61% <0.00%> (-0.03%)
:arrow_down:

doozerlib/image.py
13.12% <0.00%> (-0.16%)
:arrow_down:

doozerlib/metadata.py
25.54% <0.00%> (-0.12%)
:arrow_down:

doozerlib/olm/bundle.py
0.00% <0.00%> (ø)

doozerlib/repos.py
42.48% <10.00%> (-0.90%)
:arrow_down:

doozerlib/operator_metadata.py
79.06% <44.44%> (-1.69%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 303a9ea...1e243a0. Read the comment docs.

/cc @jupierce @sosiouxme

The concept seems fine.

I'll take a pass at cleaning this up tomorrow but I think we need to hold this until we get the release controller to initiate tests for all upgrades in stable imageStream images.

/wip cancel
/hold
This is no longer WIP but we should hold this until the release controller runs all the requisite tests.

/hold cancel

$ doozer release:calc-previous --version 4.5.32
4.5.31,4.5.30,4.5.29,4.5.28,4.5.27,4.5.25,4.5.24,4.5.23,4.5.22,4.5.21,4.5.20,4.5.19,4.5.18,4.5.17,4.5.16,4.5.15,4.5.14,4.5.13,4.5.12,4.5.11,4.5.10,4.5.9,4.5.8,4.5.7,4.5.6,4.5.5,4.5.4,4.5.3,4.4.33,4.4.32,4.4.31,4.4.30,4.4.29,4.4.28,4.4.27,4.4.26,4.4.25,4.4.24,4.4.23,4.4.22,4.4.21,4.4.20,4.4.19,4.4.18,4.4.17,4.4.16,4.4.15,4.4.14,4.4.13

what stands out to me, aside from the length of the list, is that there are no hotfixes included... just wanted to double check if that seems right.

The hotfixes only get added as upgrades if they have less than two upgrade edges, here's the results I get from master.
$ ./doozer release:calc-previous --version 4.5.32
4.5.31,4.5.30,4.5.28,4.5.27,4.5.24,4.5.23,4.5.22,4.5.21,4.5.20,4.5.19,4.5.18,4.4.33,4.4.32,4.4.31

That explains why I see no hotfixes, but not all the extra old entries... investigating

Oh, that's the intent. 4.4.13 is the minimum for minor version upgrades, and 4.5.3 is the minimum for z-stream upgrades per the data maintained by OTA.
https://github.com/openshift/cincinnati-graph-data/blob/master/build-suggestions/4.5.yaml

yeah, i misunderstood... it's working fine. thanks :)


Graphic for python not working

Does this issue occur when all extensions are disabled?: Yes/No

VS Code Version:
OS Version:

Steps to Reproduce:

yes


fix: fields fromstring not working

Please read contributing guidelines and remove this line.

Benchmark results
Metrics with a significant change:

proof_construction_time_poseidon_hash_ms (4): 45.0 (+32%)
avm_simulation_time_ms (Token:mint_public): 446 (+476%)
avm_simulation_time_ms (Token:assert_minter_and_mint): 36.2 (-75%)
avm_simulation_time_ms (Token:transfer_public): 47.2 (+63%)
avm_simulation_time_ms (Token:_increase_public_balance): 56.5 (+279%)
protocol_circuit_witness_generation_time_in_ms (private-kernel-reset-small): 1,093 (-18%)

Detailed results
All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call  to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.
This benchmark source data is available in JSON format on S3 here.
Proof generation
Each column represents the number of threads used in proof generation.

Metric
1 threads
4 threads
16 threads
32 threads
64 threads

proof_construction_time_sha256_ms
5,715 (-1%)
1,548 (-1%)
706 (-1%)
745 (-2%)
775 (+1%)

proof_construction_time_sha256_30_ms
11,742 (-1%)
3,141 (-1%)
1,412 (+1%)
1,448 (+2%)
1,465

proof_construction_time_sha256_100_ms
43,785
11,762
5,447 (+1%)
5,446 (+1%)
5,409 (+1%)

proof_construction_time_poseidon_hash_ms
78.0 (-1%)
:warning: 45.0 (+32%)
34.0
59.0 (+2%)
88.0 (+1%)

proof_construction_time_poseidon_hash_30_ms
1,514 (-1%)
414 (-1%)
199
228
270 (+2%)

proof_construction_time_poseidon_hash_100_ms
5,736 (-1%)
1,563 (-1%)
721
771 (+1%)
797

L2 block published to L1
Each column represents the number of txs on an L2 block published to L1.

Metric
4 txs
8 txs
16 txs

l1_rollup_calldata_size_in_bytes
1,412
1,412
1,412

l1_rollup_calldata_gas
9,476
9,472
9,440

l1_rollup_execution_gas
611,215
611,362
611,481

l2_block_processing_time_in_ms
752 (-1%)
1,410
2,695 (-1%)

l2_block_building_time_in_ms
19,116 (+1%)
36,969 (-1%)
74,083

l2_block_rollup_simulation_time_in_ms
19,115 (+1%)
36,969 (-1%)
74,083

l2_block_public_tx_process_time_in_ms
16,271 (+1%)
34,045 (-1%)
71,137

L2 chain processing
Each column represents the number of blocks on the L2 chain where each block has 8 txs.

Metric
3 blocks
5 blocks

node_history_sync_time_in_ms
7,013
9,884

node_database_size_in_bytes
12,427,344
16,212,048 (+1%)

pxe_database_size_in_bytes
16,254
26,813

Circuits stats
Stats on running time and I/O sizes collected for every kernel circuit run across all benchmarks.

Function
time_ms
bytecode_size_in_bytes

GasToken:_increase_public_balance
69.3 (-5%)
13,790

GasToken:set_portal
14.9
3,339

Token:constructor
91.6 (+1%)
23,692

FPC:constructor
61.8 (+1%)
13,592

GasToken:mint_public
49.6 (-2%)
10,158

Token:mint_public
:warning: 446 (+476%)
19,034

Token:assert_minter_and_mint
:warning: 36.2 (-75%)
12,925

AuthRegistry:set_authorized
28.1 (+1%)
7,812

FPC:prepare_fee
108
15,062

Token:transfer_public
:warning: 47.2 (+63%)
31,218

FPC:pay_refund
130 (+2%)
25,260

Benchmarking:increment_balance
1,958
15,267

Token:_increase_public_balance
:warning: 56.5 (+279%)
15,006

FPC:pay_refund_with_shielded_rebate
114 (-25%)
26,347

Public DB Access
Time to access various public DBs.

Function
time_ms

get-nullifier-index
0.155 (+1%)

Tree insertion stats
The duration to insert a fixed batch of leaves into each tree type.

Metric
1 leaves
16 leaves
64 leaves
128 leaves
256 leaves
512 leaves
1024 leaves

batch_insert_into_append_only_tree_16_depth_ms
10.4
16.6 (-1%)
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_16_depth_hash_count
16.8
31.7
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_16_depth_hash_ms
0.601
0.511 (-1%)
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_32_depth_ms
N/A
N/A
47.8 (-1%)
76.2 (+1%)
131 (-1%)
244
471 (-1%)

batch_insert_into_append_only_tree_32_depth_hash_count
N/A
N/A
95.9
159
287
543
1,055

batch_insert_into_append_only_tree_32_depth_hash_ms
N/A
N/A
0.488 (-1%)
0.468 (+1%)
0.451
0.443
0.440 (-1%)

batch_insert_into_indexed_tree_20_depth_ms
N/A
N/A
59.1 (-1%)
112
183 (-1%)
352
694 (-1%)

batch_insert_into_indexed_tree_20_depth_hash_count
N/A
N/A
109
207
355
691
1,363

batch_insert_into_indexed_tree_20_depth_hash_ms
N/A
N/A
0.500 (-1%)
0.500
0.485
0.476
0.476 (-1%)

batch_insert_into_indexed_tree_40_depth_ms
N/A
N/A
72.4
N/A
N/A
N/A
N/A

batch_insert_into_indexed_tree_40_depth_hash_count
N/A
N/A
133
N/A
N/A
N/A
N/A

batch_insert_into_indexed_tree_40_depth_hash_ms
N/A
N/A
0.516
N/A
N/A
N/A
N/A

Miscellaneous
Transaction sizes based on how many contract classes are registered in the tx.

Metric
0 registered classes
1 registered classes

tx_size_in_bytes
74,057
667,841

Transaction size based on fee payment method
| Metric |  |
| - |  |

fixes #7334


Android Studio unable to load resources from dependency

When running Roboelectric using JUnit configuration on Android Studio it fails to find resources (Color) defined within a dependency. The same issue is not true when using the CLI.
The library being used is https://github.com/AndroidAlliance/EdgeEffectOverride
Running on CLI (works)
DEBUG: Loading resources for ..** from /Users//****/*****/app/build/intermediates/res/myFlavorName/debug
Running on Android Studio (fails to find resource in library)
DEBUG: Loading resources for ..** from ./res...
Setting the Config.manifest would likely work, however there is no manifest within  /Users//****/*****/app/build/intermediates/res/myFlavorName/debug,
I confirmed the above would work by copying the res/myFlavorName/debug folder into a folder called res at the root of the project (so it wouldn't be overwritten) and then adding a manifest beside it. Then set the config to that manifest file. Essentially we need need away to define the manifest separately from the res folder and to have it dependent on the flavour.

Never mind, resourceDir was designed for this.
Although it doesn't help when testing multiple flavours, could extend TestRunner  to have more control of setting resourceDir, see http://stackoverflow.com/questions/21464384/how-do-i-specify-custom-res-directory-in-robolectric-2-2
Decipher what version is running by checking the default resourceDir or packageId and setting resourceDir to the correct intermediate directory based on that.


[HUDI-1779] Fail to bootstrap/upsert a table which contains timestamp column

Tips

Thank you very much for contributing to Apache Hudi.
Please review https://hudi.apache.org/contributing.html before opening a pull request.

What is the purpose of the pull request
current when hudi bootstrap a parquet file, or upsert into a parquet file which contains timestmap column, it will fail because these issues:

At bootstrap operation, if the origin parquet file was written by a spark application, then spark will default save timestamp as int96(see spark.sql.parquet.int96AsTimestamp), then bootstrap will fail, it’s because of Hudi can not read Int96 type now.(this issue can be solve by upgrade parquet to 1.12.0, and set parquet.avro.readInt96AsFixed=true, please check https://github.com/apache/parquet-mr/pull/831/files)

after bootstrap, doing upsert will fail because we use hoodie schema to read origin parquet file. The schema is not match because hoodie schema  treat timestamp as long and at origin file it’s Int96

after bootstrap, and partial update for a parquet file will fail, because we copy the old record and save by hoodie schema( we miss a convertFixedToLong operation like spark does)

Brief change log
(for example:)

Modify AnnotationLocation checkstyle rule in checkstyle.xml

Verify this pull request
add new UT, and also check by exists UTs
Committer checklist

[ ] Has a corresponding JIRA in PR title & commit

[ ] Commit message is descriptive of the change

[ ] CI is green

[ ] Necessary doc changes done or have another open PR

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

cc @nsivabalan could you help to take a look, thank you

@li36909 : can you fix the links in the description. guess its cuttoff.
parquet.avro.readInt96AsFixed=true, please check https://github https://github/.com/apache/parquet-mr/pull/831/files)

IIUC, this patch is not about failing a bootstrap or upsert w/ timestamp. We are adding support for timestamp column by upgrading parquet version. If yes, please do fix the title of the patch.
Also, looks like we are upgrading the parquet version in this patch. @vinothchandar @n3nash @bvaradar : thoughts. is there any considerations required on this end.

Hello Guys,
is it a bug on hudi 0.8.0?
I migrate all my workload to Hudi yesterday using hudi 0.8.0 and Im having problems with timestamp.

@nsivabalan thank you, I had change the link. let me check the UT fail first, then I will chagne the title

there are some UT fail cause by: https://github.com/apache/parquet-mr/pull/747/files
at this pr, parquet set requiredSchema first then do filter, and when we run count() at spark morRelation, the requiredSchma is empty, then the filter result is empty.

@li36909 Thanks for your contribution! Queued up for review.

@umehrot2 in case you have some time, please take a pass.

cc @n3nash , could you please chime in here.

@li36909 : also would be good to put up a separate patch for upgrading parquet version.

@xushiyan @alexeykudinkin : another patch thats awaiting parquet upgrade.

With Spark 3, parquet version 1.12..2 is being used. Is this issue still happening with that setup ?


feat: expose rebuild routes progress as a controller property

For some reason we were storing the progress but not exposing it before.

@zwave-js-bot automerge


Actions: Ability to list and delete caches

Summary
Dependency caching is one of the most effective ways to make jobs faster on GitHub Actions. However, there is no great way to clear caches or debug issues with them. Also, while we evict caches at a certain threshold limit, the cache performance can take a hit when the storage is near its quota. This prevents some customers from using the Actions cache.
In the last quarter we have enabled cache users to view and monitor their current cache storage usage within a Repository. We are now enhancing the Cache management experience for users to list and delete their cache entries within a Repository.
Intended Outcome

GitHub Actions enables customers to cache intermediate outputs and dependencies for their workflows. Caching is an effective way to make jobs faster and a management experience for cache usage would further promote its adoption.
Cache Management makes it easier to debug caching issues and provides greater control over cache usage by enabling users to view and delete existing caches.
As we expand caching to ES and AE, a self-serve way will all admins to troubleshoot and debug corrupted caches.

How will it work?
We will enable cache users to list and delete their cache entries across all the workflows in their repository using APIs and CLI.

this has shipped to dotcom! https://github.blog/changelog/2022-10-20-manage-caches-in-your-actions-workflows-from-web-interface
leaving open to track to GHES

This shipped with GHES 3.7<EMAIL_ADDRESS>


KEEPALIVE timing

How long should server wait between KEEPALIVEs initiated by the client before it assumes that the client is dead?
How long should the client wait for the server to echo KEEPALIVE back before it assumes the server is dead?

The spec mentions that a reasonable time period is 500 ms

A reasonable time period between KEEPALIVE frames SHOULD be 500ms.

Although, I feel this should be overridable/agreeable during Setup.
Handling the unexpected section specifies that the determination of when to assert a peer is dead is an application choice. We may want to define best practices but not enforce it on the protocol end. Another best practice could be to tolerate X number of missed keep alive frames as a GC pause can make a peer delay a heartbeat and hence create a connection churn.

The client knows what the period is but is period communicated to the sever so it to can take action if the client misses X keep alive frames.

Correct, that's why I said, it should be agreeable at the setup time.

Agree with @NiteshKant here.

Adding the keepalive interval and timeout to the SETUP would work. The server side would use those settings for that client.


Fails to import PGN file, but without any error

I've tried to import this pgn file with the library, but for I reason I was not able to find, it fails to be imported and without any error
[Event ""]
[Site ""]
[Date ""]
[Round ""]
[White ""]
[Black ""]
[Result ""]
[FEN "8/K7/8/8/8/1k6/1N1p4/8 w - - 0 1"]
[SetUp "1"]

*

[Event ""]
[Site ""]
[Date ""]
[Round ""]
[White ""]
[Black ""]
[Result ""]
[FEN "8/8/8/8/5N2/8/2p4K/2k5 w - - 0 1"]
[SetUp "1"]

*

[Event ""]
[Site ""]
[Date ""]
[Round ""]
[White ""]
[Black ""]
[Result ""]
[FEN "8/8/8/8/4N3/1p6/7K/1k6 b - - 0 1"]
[SetUp "1"]

*

[Event ""]
[Site ""]
[Date ""]
[Round ""]
[White ""]
[Black ""]
[Result ""]
[FEN "8/8/K7/4N3/8/7p/7k/8 w - - 0 1"]
[SetUp "1"]

*

[Event ""]
[Site ""]
[Date ""]
[Round ""]
[White ""]
[Black ""]
[Result ""]
[FEN "K7/8/8/5N2/8/3k3p/8/8 w - - 0 1"]
[SetUp "1"]

*

[Event ""]
[Site ""]
[Date ""]
[Round ""]
[White ""]
[Black ""]
[Result ""]
[FEN "8/2K5/8/8/4N3/8/2k4p/8 w - - 0 2"]
[SetUp "1"]

[Event ""]
[Site ""]
[Date ""]
[Round ""]
[White ""]
[Black ""]
[Result ""]
[FEN "8/2K5/8/8/4N3/3k4/7p/8 w - - 0 2"]
[SetUp "1"]

*

[Event ""]
[Site ""]
[Date ""]
[Round ""]
[White ""]
[Black ""]
[Result ""]
[FEN "8/2K5/8/8/3kN3/8/7p/8 w - - 0 2"]
[SetUp "1"]

*

[Event ""]
[Site ""]
[Date ""]
[Round ""]
[White ""]
[Black ""]
[Result ""]
[FEN "8/8/8/8/8/p7/2K1N3/k7 w - - 0 1"]
[SetUp "1"]

*

Regards
test.zip

Thanks for reporting this issue. There was not support for SetUp and FEN in 1.2. I've added it in 1.4.
Check it out!


Fix TaskHasName check for tasks with an empty name field

ansible-lint failed to detect tasks with an empty name field. This
commit fixes the issue and adds some tests for it.

@willthames do you have any thoughts about this PR? Does it need changes or is reviewing/merging just a time issue?
Let me know if I can do anything to help :slightly_smiling_face:


How to set/add custom diagnostics? [question]

How can I add/set custom diagnostic rules?

Custom diagnostics are difficult to implement because they require a deep understanding of the implementation within LSP.
It may be possible to implement it later by executing custom scripts, but it is not currently available.


Why “cipher_suites” in this  extensions.transport_sockets.tls.v3.TlsParameters just for TLSv1_2, how to set cipher_suites for TLSv1_3 ?

Title: *cipher_suites for TLSv1_3 *
Description:
cipher_suites
(repeated string) If specified, the TLS listener will only support the specified cipher list when negotiating TLS 1.0-1.2 (this setting has no effect when negotiating TLS 1.3).
So is there any way for us to setting the cipher_suites for TLSv1_3 right now ? or we need self-extension a new parameter ?
@ggreenway

From the BoringSSL docs:
// TLS 1.3 ciphers do not participate in this mechanism and instead have a
// built-in preference order. Functions to set cipher lists do not affect TLS
// 1.3, and functions to query the cipher list do not include TLS 1.3 ciphers.

I don't believe there's a way to configure boringssl to use a non-default set of cipher suites for TLSv1.3.

Correct, TLSv1.3 has mandated a set of ciphers as part of the protocol version. There is only one recommended cipher (CHACHA20-based one, non-FIPS compliant) that can possibly be adjusted.

ok, thanks very much . Seems we have to turn to using the openssl version Envoy for these limitation of BoringSSL.