only_if_cached + stale_if_error not working "offline" with more recent library dependencies

Something fishy is happening with the "offline mode" described in the docs.
I see it working as I understand it with more stable dependencies (CentOS 9), but then not working as expected with newer dependencies (Fedora 40). In both cases I'm using requests_cache 1.2.1.
I have a nice clean repo case at https://gitlab.com/castedo/not_only_if_cached.
Steps to repo
git clone https://gitlab.com/castedo/not_only_if_cached.git
cd not_only_if_cached
./test-with-centos9-offline.sh
./test-with-centos9-online.sh
./test-with-fedora40-offline.sh
./test-with-fedora40-online.sh

Got
requests.__version__='2.25.1'
requests_cache.__version__='1.2.1'
resp.status_code=200
requests.__version__='2.25.1'
requests_cache.__version__='1.2.1'
resp.status_code=200
requests.__version__='2.31.0'
requests_cache.__version__='1.2.1'
Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/urllib3/connection.py", line 174, in _new_conn
    conn = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/urllib3/util/connection.py", line 72, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/socket.py", line 976, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/urllib3/connectionpool.py", line 715, in urlopen
    httplib_response = self._make_request(
                       ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/urllib3/connectionpool.py", line 404, in _make_request
    self._validate_conn(conn)
  File "/usr/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1060, in _validate_conn
    conn.connect()
  File "/usr/lib/python3.12/site-packages/urllib3/connection.py", line 363, in connect
    self.sock = conn = self._new_conn()
                       ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/urllib3/connection.py", line 186, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f72e4ecb9b0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/urllib3/connectionpool.py", line 801, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/urllib3/util/retry.py", line 594, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /search/commits?q=hash%3A1ca488e4d3f33059a04656f855ba22ef5393298a (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f72e4ecb9b0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/mnt/./get_only_if_cached.py", line 25, in <module>
    resp = session.get(
           ^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/requests_cache/session.py", line 127, in get
    return self.request('GET', url, params=params, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/requests_cache/session.py", line 183, in request
    return super().request(method, url, *args, headers=headers, **kwargs)  # type: ignore
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/requests_cache/session.py", line 230, in send
    response = self._send_and_cache(request, actions, cached_response, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/requests_cache/session.py", line 254, in _send_and_cache
    response = super().send(request, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /search/commits?q=hash%3A1ca488e4d3f33059a04656f855ba22ef5393298a (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f72e4ecb9b0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))
requests.__version__='2.31.0'
requests_cache.__version__='1.2.1'
resp.status_code=200
 cache/d68ef4288b0d316a.json | 184 ++++++++++++++++++++++----------------------
 1 file changed, 92 insertions(+), 92 deletions(-)

The two tests on CentOS 9 worked as expected. The cached file did not change and there was no network usage.
But the third test, on Fedora 40, without a network, failed because the network was used.
And then on the fourth, when there is a network, the cached file got changed, as indicated by git diff.
Expected
For all cases to behave the same, no network access, and for the cached file to be left unchanged.
requests.__version__='2.25.1'
requests_cache.__version__='1.2.1'
resp.status_code=200
requests.__version__='2.25.1'
requests_cache.__version__='1.2.1'
resp.status_code=200
requests.__version__='2.25.1'
requests_cache.__version__='1.2.1'
resp.status_code=200
requests.__version__='2.25.1'
requests_cache.__version__='1.2.1'
resp.status_code=200

BTW, those test helper scripts runpodman but you can probably change or alias it to docker and it will still work. If you want to see repo cases on Ubuntu or Debian images let me know.

And the Containerfiles (Dockerfiles) for those two images can be found at:
https://gitlab.com/ellerman/nest/dev (Fedora 40 with tons of extra stuff)
and
https://gitlab.com/perm.pub/dock/hidos-dev/-/tree/main/centos-9 (CentOS 9 with some extra stuff)

Thanks for the thorough bug report. I can't immediately tell what the problem is, but it looks like you've given enough details to figure it out. I should be able to get to this later this week.
Meanwhile, if you or anyone else out there has time to help narrow down which specific dependency change(s) are causing this, the help would be appreciated!

Glad to help this awesome package. I discovered and started using it just a few days ago. I'm glad I didn't start re-inventing the wheel.
I've updated the test helper script to output more package dependency versions. Unfortunately, I have little clue what might causing this so I'm kind of stabbing in the dark for what packages to compare.
Here's  the output when I ran it on a gianomous Ubuntu 24.04 (Noble) container that I'm using.
requests.__version__='2.31.0'
requests_cache.__version__='1.2.1'
version('attrs')='23.2.0'
version('cattrs')='22.2.0'
version('urllib3')='2.0.7'
version('url_normalize')='1.4.3'
resp.status_code=200

So interesting that this doesn't seem to have to do with requests 2.31.0 because it's working fine on Ubuntu.
I'll test more more Fedora image variations this week and report back what happens.

I've got a great isolation of the issue. It has something to do with brotli. Installing the Python brotli Fedora package causes this problem. I'm going to try and confirm on some smaller and cleaner images and will post the details here (pointing to the repo).

@JWCook I think I've got I nice clean repo case saved for you at:
https://gitlab.com/castedo/not_only_if_cached/-/blob/03831502c6e452a93dd9794a723a56ee9a62efd7/README.txt
That README.txt is the starting point. Definitely seems like something weird, confusing, and not ideal is happening with some kind of brotli package interaction. I'll leave the rest for you. Let me know if you want anymore help.

I've found a quick fix:
session.get(
    ...
    headers = {
        ...
         "Accept-Encoding": "gzip, deflate",
        ...
    }
)

Forcing the "Accept-Encoding" setting in the headers I'm guessing prevents some downstream software from checking to see if brotli is available, then deciding the change the request from what is in the cache, and thus cause online request and not use the cache.
I'm not sure if this is something requests-cache should prevent from happening. Maybe this is something requests-cache should allow to happen. Nonetheless, it's very confusing to have cache behavior change only because a compression library becomes available.

@castedo Is the problem reproducible only for that url + query + headers? Or the request parameters can be simplified while still reproducing the problem? I am asking because that Github URL returns 422 response status code, not 200.

Step by step

requests-cache calls Session.request() method of the requests lirbary
then the traceback travels back to the requests-cache (Cachedsession.send() method), etc.
requests calls make_headers source
make_headers calls ACCEPT_ENCODING += ",br" source

Without brotli we have: 'accept-encoding=deflate, gzip'
With brotli we have: 'accept-encoding=br, deflate, gzip'.
A cached request is invalidated due to the different headers.
Since the cache request has been invalidated, the library is trying to make a new request, and, obviously, fails if you are offline.
Conclusion
I understand that this can be confusing, but a library cannot guarantee portability across all possible different environment (with/without Brotli, with other different dependencies or system packages, etc.).
The best we can do is mention in the README that installing/uninstalling Brotli may invalidate the cache.
@JWCook Does this make sense?

Yep, that makes sense. Seems like the only change worth making it a change to documentation. I can do a PR if that makes sense.
Two areas for the docs that come to mind are:

https://requests-cache.readthedocs.io/en/stable/user_guide/expiration.html#cache-only-requests
https://requests-cache.readthedocs.io/en/stable/user_guide/matching.html#selective-parameter-matching

For the 1. maybe it's worth adding a caveat that cached request identification is not solely dependent on the parameters to session.get, but also involves how CachedSession is configured and how requests/urllib3 modify/prepare the request headers after the session.get.
In the area of the documentation of 2. it might be worth mentioning that request parameters can sometimes be "forced to be fixed" by passing them, and an example would be the work around I mention above of always passing "Accept-Encoding": "gzip, deflate" so that one of those other layers does not tweak it.
Thoughts?

I can do a PR if that makes sense.

Two areas for the docs that come to mind are:

Yes, what you say sounds even better than I expected. A pull request with the above noted would be great.

@alessio-locatelli #1037 is a PR to resolve this issue.


fix(presenceUpdate): [OWL-25156117] Added logs and reorder logic for removing role when member goes offline

FYI, merging this to fix another bug on removing the Now Live role. cc: @alvinb


Allow download of invalid record

We should allow download of invalid record.
The use case is that someone starts with a draft, wants to save it and continue some other time.
It should be basically just not disabling the download button.

Whether the user whether he is downloading an invalid draft, or a valid submittable version should be clearly visually distinguished in the UI IMO

It is now distinguished by the validation result message, which is right above the download button.
We might emphasize it by also changing the download button color and text.


UI for enhancements to rancher settings

UI for https://github.com/rancher/rancher/issues/17488
I am adding this on behalf of @orangedeng because I am not sure if the changes introduced in https://github.com/rancher/rancher/pull/17208 going to require UI changes to be fully utilized.
In short, if a setting is set via an environment variable a rancher start-up time, the setting is NOT EDITABLE. This was implemented by removing the update link upon returning the setting in the API and erroring out if an update is required.
So, if the UI automatically respects the absence of the update link for settings, then no UI change should be needed

Version: Master 2.2 (2/4/19)
Confirmed fix.
Case1: run with env var CATTLE_UI_INDEX=local and ensured UI Index is NOT editable in the UI. API looks correct.
Case2: run without env var and ensure UI Index IS editable and can change to local and smoke test UI. API looks correct.
Local UI is v2.0.77 and so we are missing mc app functionality, etc. @loganhz is this expected when we run with local UI index? Or should I open a separate bug for this? I expected LATEST UI with master branch. Also I noticed that I can attempt to "Revert to default" which results in 405 error (as expected due to api) but the UI should either have a nicer error to explain what went wrong or we should remove revert to default. I think we have this option because in API "customized": false, so I presume I should open another separate bug for this as well?


R CMD check warning about S3 method

Running R CMD check generates the following warning:
checking S3 generic/method consistency ... WARNING
  plot2:
    function(x, y, ...)
  plot2.formula:
    function(formula, data, type, xlim, ylim, main, sub, xlab, ylab, ann,
             axes, frame.plot, asp, grid, palette, palette.args,
             legend.position, legend.args, pch, ...)

  See section ‘Generic functions and methods’ in the ‘Writing R
  Extensions’ manual.

The current dispatch calling looks right to me. Maybe it's a roxygen2 issue? Or do I have to explicitly allow all arguments in each method?

Batcall to S3 maestro @zeileis in case you can spot anything obvious that I'm missing ;-)

Disclaimer: I haven't actually tried the code, though, and I haven't looked at the documentation again.
Nevertheless: A couple of comments.

The convention for S3 generic/method consistency is that the method should have all the arguments that the generic has (including ...). It may have additional arguments, though.
However, the dispatch works correctly regardless of argument naming.
An exception to the S3 generic/method consistency rule was that the first argument of *.formula methods should be called formula. However, I think that the opinion has changed on this exception and that it shouldn't be made anymore.

So I would expect that the checks want you to name the first two arguments of plot2.formula "x" and "y". In addition, you may have "formula" and "data" arguments but they shouldn't be the first two.

What triggered the discussion about the inconsistency of naming the first argument of *.formula methods "formula" was this: https://bugs.R-project.org/show_bug.cgi?id=18299
The inconsistency lead to problems with piping in combination with the formula method of aggregate(). Hence, the first argument of aggregate.formula() was renamed from formula to x.
I had argued that the renaming of the first argument is good but that I would still support the formula argument for backward compatibility. Unfortunately, my proposed patch was not incorporated. In fact, no one from R Core even commented.

Without having looked at the details of the code, I think the following might work:

Generic: plot2(x, ...).
So the generic just has arguments x and ... that all methods must implement.
Default: plot2(x, y = NULL, by = NULL, data = NULL, [...], ...).
The default method then has the y argument and the other arguments.
Formula: plot2(x, data, type, [...], formula, ...).
The formula method has the first argument x for consistency but also provides the argument formula for convenience. Internally, you just need to catch that x and formula cannot both be specified simultaneously.

Just tested locally and it works. Thanks Achim!


Code Refactor - Make modifications for this project to build on Mac OS and Linux

Compiled ASAR for linux and OSX, modified tasks.json to run different commands per OS.

Addresses #15


RHT-1313: 6.2.2a_10 iPeople Implementation Initiate job-data-change

Introduced new call processes

LGTM


Vouching claims to support Markdown, but output does not

Vouching for https://mozillians.org/en-US/u/timothygu/ I wrote:
Timothy has made numerous contributions to web standards:
* https://github.com/whatwg/url/commits?author=TimothyGu
* https://github.com/whatwg/html/commits?author=TimothyGu

It seems lists are not supported, despite this indicating they are:

Use Markdown for bold, italic, lists and links.

Is anyone working on this? Can I contribute to this?


Screenshots and the steps

Hello,
I wanted to use your tutorial but the newest AAD Connect client has TOTALLY different options.
pls give link to the version of AADC you used or change steps and screenshots
Best regards
Emil

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 816f0138-d336-3905-9f4f-e60c04da9326
Version Independent ID: ce5166e4-5d15-ef1f-1090-576a69e601e8
Content: Configure hybrid Azure Active Directory join for managed domains
Content Source: articles/active-directory/devices/hybrid-azuread-join-managed-domains.md
Service: active-directory
Sub-service: devices
GitHub Login: @MicrosoftGuyJFlo
Microsoft Alias: joflore

Client version I'm using is <IP_ADDRESS> (from the link you delivered in tutorial)

@EmilSPL
Thanks for your feedback! We will investigate and update as appropriate.


fix(service/postgresql): remove prepared statement cache for the postgresql service

fix #3173

Because the tokio pg is global based connection.. this PR is meaningness..


Updated Pacejka Fy and Mz coefficients

Update coefficients for Fy and Mz Pacejka nonlinear tire model. Please note: the input angles must be in [deg]. This was necessary to have better data fitting quality

Looking good, here is the plot:

The camber angle term in the moment looks quite different than before, should we go with that?

The camber angle term in the moment looks quite different than before, should we go with that?

It is the best I can obtain so far. I would go with that. Then, tuning after tuning, we can have better results for different camber angles

Bad quality but here is a quick view of that run:
https://github.com/user-attachments/assets/f2855233-1b92-446d-9732-85fcb668726c
I think you have something to work with now!

https://github.com/user-attachments/assets/9f39eece-da43-4b52-aded-52376b9dd5fc


There are no releases under "Downloads"

Original report by Anonymous.

Somehow all of the releases in the Downloads folder got removed. I noticed this when running a migration on Bioconda, since the versioned source tarballs are no longer available :(

Propose to close this since this concerns bitbucket.


Added py.typed to project

Fixes #342. Added a py.typed marker file to enable typing, per PEP 561. Also included the file in setup.cfg.

coverage: 94.684% (-0.005%) from 94.689%
when pulling fa3de529890b49c2d212d360b9e9558ae65ad236 on manforowicz:master
into d232fc1b74d5ec458fca134afc85004308145a49 on OpenCyphal:master.


Fix: Cannot find code editor: VSCode; although installed on linux

VSCode's executable name is inconcistent and godot only searches for "code" when opening C# file with external editor set to Visual Studio Code.
When installing VSCode on linux from MS, the executable's name is visual-studio-code, but when installing from the repository, the executable's name it's either code or vscode.

I haven't tested this on windows yet.

Thanks!

There's a reason why  it's vscode in some and code in others. The binaries offered by Microsoft (VSCode, with trademarked logos and telemetry) are proprietary, while when compiled from source (just Code) are not.

it seems that installing vscode from https://code.headmelted.com sets the executable name to code-oss.

Indeed, I will add a way to specify a custom path. I'm fine with adding code-oss to the current list though.

Now that Godot 3.1 has been released, we don't plan to cherry-pick new features and enhancements to the 3.0 branch, unless there is very strong support in favor of it. Removing cherrypick label for 3.0.


Add Container Deployer Support for "component create" CLI command

Description
Similar to the existing support of the helm-ls and manifest deployer, the CLI must also support the container deployer. So similar to the other deployers, it should be possible to add a container-based deployment via the CLI to a component.
This is crucial for some of the Gardener-Components.
Done Criteria

[x] All Features implemented
[x] Code has been reviewed by other team members
[x] Unit Tests created for new code
[x] Integration Test Suite updated (if applicable)
[ ] Internal technical Documentation updated
[x] Enduser Documentation updated (if applicable)
[ ] Successful demonstration in Review

basic functionality implemented and documented
missing

integration tests
handling of image pull secrets
description/example for data in blueprint, blueprint and component descriptor reference in deploy item
adapt description of component deployed in landscaper project

Current State: See above comment

So all the missing work mentioned above is done ?

[ ] integration tests
[ ] handling of image pull secrets
[ ] description/example for data in blueprint, blueprint and component descriptor reference in deploy item
[ ] adapt description of container deployer in landscaper project

Sure. That's the reason why I marked all done criteria accordingly :-)

Remaining docu for the CLI & for the container deployer in the landscaper docu has been created and tested
Unit tests & Integration tests have been created


TagUI check radio buttons, elements with changing ID, modifying DOM directly etc..

Hello @kensoh i need some help. I have a radiogroup on some web page, i need to click a radio button option, but i cant do it with "click" or "select".. or maybe im doing it wrong.
theres the html code:

and a example of the radio button

i try with the document.getElementById("iCheck-do2pjq").checked = true
but doesnt work

@fez226 try dom step to run code on the webpage DOM, hope that helps!
dom document.getElementById("iCheck-do2pjq").checked = true

Hello @kensoh thanks for you quickly reply, but not lucky.

i try with the XPath but the same result, im doing something wrong?
dom document.getElementByXPath('//*[@id="iCheck-do2pjq"]').checked = true

i think i find a reason of the error, every time i run the tagui and open one chrome windows, the label of the radio button change

i finally have it, if anyone have a similar problem, you need to copy the xpath of the radiogroup:
in this case:
//*[@id="infoFormStep1"]/div[5]/div[1]/p/span
and add the /label[1]/input for the first option /label[2]/input for the second one...
so the next line will click the Yes Radio Button:
click //*[@id="infoFormStep1"]/div[5]/div[1]/p/span/label[1]/input

Cool detective work! Yes some webpages will have new IDs each time you visit, that makes it impossible to locate by ID. Your above method is an elegant solution to the problem.
Another way I saw your screenshot above there is a name attribute. In XPath you can try out using //*[@name = "useOpportunity"] followed by something to locate that radio group.

@kensoh i dont know that, we can change the @ID to @NAME or another attribute. But i have a question, both of the radio buttons option name "useOpportunity" (yes and the no option) so how i can selected?
select //*[@name="useOpportunity"] as Y ?? (doesnt work)
i copy both elements to more examples

You can many things with XPath, any attribute can be used with @attribute_name.
You can do something similar to what you did earlier by layering -
//*[@id="infoFormStep1"]/div[5]/div[1]/p/span/label[1]/input
Something like below, you have to experiment on the webpage -
click //*[@name="useOpportunity"]//label[1]/input

im trying, but doesn't work. Actually my first solution at the end doesn't work, because the html code change when finished loading. the Final html code is on my last screenshot.
I will continue investigating, but any recommendation is grateful

Some other ideas -

You can also consider adding a wait step to ensure webpage completes loading.

Your previous attempt at the web page using relative paths like below should be a good approach for websites with changing IDs.

//*[@id="infoFormStep1"]/div[5]/div[1]/p/span/label[1]/input

Lastly, you can consider using the computer vision automation method to interact with the web page radio button.

PS - I forgot to add that if the webpage is a public webpage, you can share with me your replication workflow script so that I can take a look.

i see your point, i think sikuli may dont work, because its a lot of radio buttons with Yes or No answers.
Regrettably the webpage is not public.
the next line doesnt work, because when the page is finally load, the html code change and there is no Label but ichecked
//*[@id="infoFormStep1"]/div[5]/div[1]/p/span/label[1]/input

Anyone of the following works for me for this website. Good luck!
http://icheck.fronteed.com/#skins
click //*[@class="iradio_minimal"]
wait
click (//*[@class="skin skin-minimal"]//*[@class="skin-section"]/ul[2]//div)[2]
wait
click (//*[contains(@class,"iradio_minimal")])[1]
wait

Below is a useful guide on using XPath from W3School -
https://www.w3schools.com/xml/xpath_intro.asp

Hello @kensoh i learned a lot with the w3schools and your example, finally i can click on the "yes" radio buttons with this line
click (//[@class="iradio_square-blue"]//[@name="useOpportunity"])
i try for the "no" answer but doesnt work with this:
click (//[@class="iradio_square-blue"]//[@name="useOpportunity"])[2]
i try with this one, but there i way to put 2 attributes for one click, something like
click (//[@class="iradio_square-blue"]//[@name="useOpportunity"] | [@value="N"])
see the image below to know more about the last line of code

update: for some reason this line of code click the "no" answer
click //*[@id="infoFormStep1"]/div[5]/div[1]/p/span

i finally have it, i test it with other radio button i check
//no click (//*[@id="infoFormStep1"]/div[5]/div[1]/p/span/label[2]) //si click (//*[@id="infoFormStep1"]/div[5]/div[1]/p/span/label[1])
its like my first soution but without the /input
anyway, i thank u kensoh for all the help you can give me.. and also the question its on table:
can i click one element with two attributes:
click (//*[@class="iradio_square-blue"]//*[@name="useOpportunity"] | [@value="N"])

yes xpath is very powerful - https://www.w3schools.com/xml/xpath_operators.asp
for example //*[@class='abc' and @name='xyz'] - see above link for more useful operators 👍

oooh excelent i get it now its inside the [] thanks a lot, i try it and works great... so in this issue we see: tagui automation with another xpath that ID, elements that changes, DOM, and operators like /, //, *, and, or... great thanks!

welcome @fez226 and have fun with xpath!
it is the most expressive and powerful way to locate web elements, more powerful than usual css selectors, your time invested to understand it will give great rewards in your automation work.


Where are the definitions of the edges?

Hi guys! Joern is such a powerful tool! Thanks for your work!
I find many types of edges in the output of joern, e.g., AST, CFG, PDG, and DDG.
And there are even more types in the format of graphson files, e.g., REF, AST, SOURCE_FILE, and CONTAINS.
These two classes of edges even are not in the same manner.
Where can I find the definitions of the edges?

You can find them in

https://cpg.joern.io/
https://github.com/ShiftLeftSecurity/codepropertygraph/blob/fdb95ad27e67ba1f642b191f5885a43fe8e2dc5a/schema/src/main/scala/io/shiftleft/codepropertygraph/schema/Ast.scala?_pjax=%23js-repo-pjax-container%2C div[itemtype%3D"http%3A%2F%2Fschema.org%2FSoftwareSourceCode"] main%2C [data-pjax-container]#L389

OK, Thanks!

I found too many DDG edges in CPG graphs?
What are the DDG edges? I can not find any specification in the two website.

For example, a simple code snipet:
`int add(int t)
{
return t + 1;
}
int main()
{
int a = add(2);
int b = a+1;
return 0;

}
`
The corresponding CPG is:

There are more than one DDG edge between two nodes?
Is this normal?

What are the DDG edges?

https://cpg.joern.io/#schema-ref-pdg

There are more than one DDG edge between two nodes?
Is this normal?

@fabsx00 Need some offical help here :)


ExternalXML in RAVEN Code Interface

Pull Request Description
What issue does this change request address? (Use "#" before the issue to link it, i.e., #42.)
Closes #593.
What are the significant changes in functionality due to this change request?
Generalizes the ExternalXML expansion tools (moving them from Simulation to xmlUtils) and expands these blocks in the RAVENparser within the RAVEN interface.
Changes the DataObjects block in the raven-runs-raven test to be placed externally by way of test, and adds testing functionality explicitly for the xmlUtils implementation of the external node expansion.

For Change Control Board: Change Request Review
The following review must be completed by an authorized member of the Change Control Board.

[x] 1. Review all computer code.
[x] 2. If any changes occur to the input syntax, there must be an accompanying change to the user manual and xsd schema. If the input syntax change deprecates existing input files, a conversion script needs to be added (see Conversion Scripts).
[x] 3. Make sure the Python code and commenting standards are respected (camelBack, etc.) - See on the wiki for details.
[x] 4. Automated Tests should pass, including run_tests, pylint, manual building and xsd tests. If there are changes to Simulation.py or JobHandler.py the qsub tests must pass.
[x] 5. If significant functionality is added, there must  be tests added to check this. Tests should cover all possible options.  Multiple short tests are preferred over one large test. If new development on the internal JobHandler parallel system is performed, a cluster test must be added setting, in  XML block, the node <internalParallel> to True.
[x] 6. If the change modifies or adds a requirement or a requirement based test case, the Change Control Board's Chair or designee also needs to approve the change.  The requirements and the requirements test shall be in sync.
[x] 7. The merge request must reference an issue.  If the issue is closed, the issue close checklist shall be done.
[x] 8. If an analytic test is changed/added is the the analytic documentation updated/added?

For Change Control Board: Change Request Review
The following review must be completed by an authorized member of the Change Control Board.

[x] 1. Review all computer code. done
[x] 2. If any changes occur to the input syntax, there must be an accompanying change to the user manual and xsd schema. If the input syntax change deprecates existing input files, a conversion script needs to be added (see Conversion Scripts). n/a
[x] 3. Make sure the Python code and commenting standards are respected (camelBack, etc.) - See on the wiki for details. ok
[x] 4. Automated Tests should pass, including run_tests, pylint, manual building and xsd tests. If there are changes to Simulation.py or JobHandler.py the qsub tests must pass. they pass
[x] 5. If significant functionality is added, there must  be tests added to check this. Tests should cover all possible options.  Multiple short tests are preferred over one large test. If new development on the internal JobHandler parallel system is performed, a cluster test must be added setting, in  XML block, the node <internalParallel> to True. done
[x] 6. If the change modifies or adds a requirement or a requirement based test case, the Change Control Board's Chair or designee also needs to approve the change.  The requirements and the requirements test shall be in sync. n/a
[x] 7. The merge request must reference an issue.  If the issue is closed, the issue close checklist shall be done. ok. issue referenced
[x] 8. If an analytic test is changed/added is the the analytic documentation updated/added? n/a


[TEVA-2145] Save job click event

This allows us to trigger click events to actions in controllers that require authentication.
Jira ticket URL
https://dfedigital.atlassian.net/browse/TEVA-2145
Changes in this PR

Add generic click event before action
Update saved job request spec

Review app deployed to https://teaching-vacancies-review-pr-3033.london.cloudapps.digital


Redis Support via Lambda

I want to trace the ElasticCache performance from my lambda function, could not find a capture reference for redis in the documentation. Is it supported ?

Sorry for the late response. The SDK currently doesn't officially support any Redis client. You can check this https://github.com/aws/aws-xray-sdk-node/issues/28 and add any further comment for your use case.
Depending on which client you are using you might still be able to capture the cache get latency by using general captureAsync or captureCallback functions.
If you need further help, please let us know the client and the function you want to capture so we can take a further look.

@willarmiros can this be re-opened as a Feature Request?

Sure. Related to: https://github.com/aws/aws-xray-sdk-python/issues/194

It would be great if this were added to the .NET Core SDK as well!

FWIW, the AWS managed Lambda layer for AWS Distro for OpenTelemetry has support for Node-Redis 2 and 3 and can publish traces to Xray:
https://aws-otel.github.io/docs/getting-started/lambda/lambda-js
https://github.com/open-telemetry/opentelemetry-lambda/blob/636df37/nodejs/packages/layer/package.json#L47


Bump/lru cache

Bump the LRUCache dependency to avoid transpilation problem when building inside of yarn workspaces.
close #24

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

9 of 9 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
661

Relevant Lines:
661

💛  - Coveralls

Thanks! Released as 2.5.1.


Unable to customize response object

I was going through your source code for the module when i came across
this file: https://github.com/squeaky-pl/japronto/blob/master/src/japronto/response/py.py
While the current commit comment for it claims its a "dead import", will there ever be a way to make a wrapper for the response object in the future? Ultimately, my goal is to change the way the response behaves and change the order in which the headers are returned.
Will there be anything like this in the future?
Hopefully something like this:
from Japronto import Response

class NewResponse(Response):
    """
    Custom Japronto Response object
    """

    def __init__(self, *args, **kwargs):
        Response.__init__(self, *args, **kwargs)
        #custom code here

    def render(self):
        #custom code here
        pass

The python file you are referring to isn't actually used,
also the dead import commit actually removed some dead import from the file as well as others.
The actual file used for the response object is here: https://github.com/squeaky-pl/japronto/blob/master/src/japronto/response/cresponse.c.
Which can be imported as follows:
from japronto.response.cresponse import Response

Although it is possible to subclass a class implemented in C it would require some work by @squeaky-pl or another contributor who finds this important.

A reference guide can be found here:
https://www.python.org/dev/peps/pep-0253/


Allow one redirect by default

We want to allow both http and https for requests, so configure agent to do so
This changes the request config to allow up to one redirect by default (in case the request url is shortened and needs to redirect). This can be customized by passing in a custom client config (see readme).

Doesn't this creates a new agent per request -- this will effectively create a new connection  right?

@flippmoke right 😬 decided to remove configuring an http/https agent. Testing with downstream libraries indicates we don't need it. Now this just updates the request config to allow for one redirect.


Memory map doesn't include TEB pages

The !address function in Windbg reconstruct the memory maps including the TEB pages. I can modify the code to add those in the map as well, but I am not sure how Windbg knows their sizes:
18`ed1df000       18`ed1e0000        0`00001000 MEM_PRIVATE MEM_COMMIT                                     PEB        [720]
18`ed1e0000       18`ed1e2000        0`00002000 MEM_PRIVATE MEM_COMMIT                                     TEB        [~0; 720.38fc]
18`ed1e2000       18`ed1e4000        0`00002000 MEM_PRIVATE MEM_COMMIT                                     TEB        [~1; 720.2560]
18`ed1e4000       18`ed1e6000        0`00002000 MEM_PRIVATE MEM_COMMIT                                     TEB        [~2; 720.3c14]
18`ed1e6000       18`ed1e8000        0`00002000 MEM_PRIVATE MEM_COMMIT                                     TEB        [~3; 720.37b0]

Figure how this works - there's probably a region in the mem map and the TEB start addresses which might be enough to do the same.

The teb base should be in the thread structure somewhere. Not sure how the sizes are calculated, but I just hardcode it to 0x2000

Makes sense - and yes I do have the TEB start addresses already; I'll need to check on both x86/x64 if hardcoding this works fine :) Thanks for sharing!
Cheers

I took a look at this tonight - so actually it's not as bad as I thought; I thought I was missing the regions in the map but it's here.

Here is what parser.exe shows w/ tests.zip/x86.ma.dmp:
ed8000           eea000            12000  MEM_COMMIT MEM_PRIVATE   00 00 01 04...

And WinDbg breaks down the regions and create artificial 'holes':
ed8000   ed9000     1000 MEM_PRIVATE MEM_COMMIT        <unknown>  [................]
ed9000   edc000     3000 MEM_PRIVATE MEM_COMMIT        PEB        [1d78]
edc000   ee0000     4000 MEM_PRIVATE MEM_COMMIT        TEB        [~0; 1d78.3674]
ee0000   ee4000     4000 MEM_PRIVATE MEM_COMMIT        TEB        [~1; 1d78.658]
ee4000   ee8000     4000 MEM_PRIVATE MEM_COMMIT        TEB        [~2; 1d78.4a14]
ee8000   eea000     2000 MEM_PRIVATE MEM_COMMIT        TEB        [~3; 1d78.1270]

It seems like it hardcodes 0x4'000 per TEB and for the last one there's just a remaining hole of 0x2'000 bytes.
Anyways, I am happy with the current behavior and am closing this down. Feel free to re-open if I'm missing something!
Cheers


Minimal API: TryParse

Generate TryParse (when possible) to support use in minimal APIs.

see #18


I have archives of old websites related to this.

One of the site's I have a archived is the old CrackPL Tools. This site has old packers and unpacker tools and tutorials.
The other site is www.mesa-sys.com which is an old crackme's website not sure if its useful to you,

Moreover, mesa-sys.com seems to be for sale. Could you tell more about the site you mentioned ?

The sites no longer exist.
Here is a google drive folder of them.
https://drive.google.com/drive/folders/15GxWRGZx3E-xdEXWNY6-EsiJ5TkaU2v2?usp=sharing
The sites were dedicated to reverse engineering. the crackme which was the mesa website was a site with a collection of small programs made by the community with protections to break.

PR ?... 😁

Sorry not sure what PR stands for in this instance.

I mean this is worth a Pull Request. Simply edit the README.md (respecting the contributing guidelines, of course) with your new ideas. You can reference the links to your own site as the original ones are broken.


cluster egress-selector mode may break kubectl port-forward

The cluster egress selector mode only allows connections to kubelets and service endpoints; it doesn't properly route connections to pods. Unfortunately, kubectl port-forward needs to connect to a pod via its cluster IP, and to do that via the agent tunnel needs to know which node the pod is on.
We'll need go watch pods instead of endpoints in order to properly route requests to the node that the pod is running on.
This is the root cause for the failures seen in

https://github.com/rancher/rke2/issues/3010

I suspect that this may also affect pod mode, preventing the the apiserver connecting to pods (and service endpoints) when the pod uses host network since the pod IP is the node's IP, not within the cluster CIDR.

It appears that kubectl port-forward, exec, and logs all go through the agent tunnel to the kubelet, so port-forward should not require access to a pod address. This may not be an issue after all, but I'll continue to investigate.

The issue with service endpoints on host network pods is real though, for example the rke2 ingress-nginx admission controller runs with host net (ref: https://github.com/rancher/rke2/issues/2206) and cannot be accessed in pod or cluster mode:
brandond@dev01:~$ kubectl get endpoints -n kube-system rke2-ingress-nginx-controller-admission
NAME                                      ENDPOINTS         AGE
rke2-ingress-nginx-controller-admission   <IP_ADDRESS>:8443   7m3s

This happens because we handle this as a connection to the kubelet, which is redirected to loopback address before being remote dialed, and then rejected by the agent for being on non-allowed port.

Validated in v1.22.10-rc2+k3s1, v1.23.7-rc2+k3s1, and v1.24.1-rc4+k3s1 this appears to be working now as expected. The better validations and information will happen in rke2 but I am able to deploy rancher using ingress-nginx instead of traefik in these.

Reopening for now. On sonobuoy tests I actually saw failures. Some are potentially flakes, but there are a lot of consistencies here between the versions and what previously failed using multus or calico on rke2. Also, setting egress-selector-mode: disabled resolves this and conformance tests fully pass. Run via sonobuoy run --kubernetes-version=v1.24.1 --mode=certified-conformance where the kubernetes-version value should match the version.
# v1.24.1
 [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should be able to deny pod and configmap creation [Conformance]
 [sig-cli] Kubectl client Update Demo should scale a replication controller  [Conformance]
 [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] patching/updating a mutating webhook should work [Conformance]
 [sig-api-machinery] CustomResourceConversionWebhook [Privileged:ClusterAdmin] should be able to convert a non homogeneous list of CRs [Conformance]
 [sig-network] DNS should provide /etc/hosts entries for the cluster [Conformance]
 [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] listing mutating webhooks should work [Conformance]
 [sig-network] DNS should resolve DNS of partial qualified names for services [LinuxOnly] [Conformance]
 [sig-api-machinery] CustomResourceConversionWebhook [Privileged:ClusterAdmin] should be able to convert from CR v1 to CR v2 [Conformance]
 [sig-network] DNS should provide DNS for services  [Conformance]
 [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] listing validating webhooks should work [Conformance]
 [sig-network] DNS should provide DNS for the cluster  [Conformance]
 [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should mutate custom resource with pruning [Conformance]
 [sig-apps] ReplicationController should serve a basic image on each replica with a public image  [Conformance]
 [sig-cli] Kubectl client Guestbook application should create and stop a working application  [Conformance]
 [sig-node] PreStop should call prestop when killing a pod  [Conformance]
 [sig-network] DNS should provide DNS for pods for Hostname [Conformance]
 [sig-apps] ReplicaSet should serve a basic image on each replica with a public image  [Conformance]
 [sig-cli] Kubectl client Update Demo should create and stop a replication controller  [Conformance]
 [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should be able to deny attaching pod [Conformance]
 [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should mutate custom resource with different stored version [Conformance]
 [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should deny crd creation [Conformance]
 [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should mutate configmap [Conformance]
 [sig-network] DNS should provide DNS for ExternalName services [Conformance]
 [sig-network] Proxy version v1 A set of valid responses are returned for both pod and service Proxy [Conformance]
 [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should mutate pod and apply defaults after mutation [Conformance]
 [sig-api-machinery] Aggregator Should be able to support the 1.17 Sample API Server using the current Aggregator [Conformance]
 [sig-scheduling] SchedulerPredicates [Serial] validates resource limits of pods that are allowed to run  [Conformance]

# v1.23.7
 [sig-network] DNS should provide DNS for the cluster  [Conformance]
 [sig-network] DNS should provide DNS for pods for Subdomain [Conformance]
 [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] patching/updating a mutating webhook should work [Conformance]
 [sig-network] DNS should provide DNS for ExternalName services [Conformance]
 [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] listing validating webhooks should work [Conformance]
 [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should honor timeout [Conformance]
 [sig-network] DNS should resolve DNS of partial qualified names for services [LinuxOnly] [Conformance]
 [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should be able to deny attaching pod [Conformance]
 [sig-network] DNS should provide DNS for pods for Hostname [LinuxOnly] [Conformance]
 [sig-cli] Kubectl client Update Demo should scale a replication controller  [Conformance]
 [sig-cli] Kubectl client Guestbook application should create and stop a working application  [Conformance]
 [sig-api-machinery] CustomResourceConversionWebhook [Privileged:ClusterAdmin] should be able to convert from CR v1 to CR v2 [Conformance]
 [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should mutate custom resource with different stored version [Conformance]
 [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should mutate pod and apply defaults after mutation [Conformance]
 [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should unconditionally reject operations on fail closed webhook [Conformance]
 [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should deny crd creation [Conformance]
 [sig-api-machinery] Aggregator Should be able to support the 1.17 Sample API Server using the current Aggregator [Conformance]

# v1.22.10
 [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should mutate pod and apply defaults after mutation [Conformance]
 [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] patching/updating a validating webhook should work [Conformance]
 [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should honor timeout [Conformance]
 [sig-apps] ReplicationController should serve a basic image on each replica with a public image  [Conformance]
 [sig-network] DNS should resolve DNS of partial qualified names for services [LinuxOnly] [Conformance]
 [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should mutate custom resource with different stored version [Conformance]
 [sig-cli] Kubectl client Update Demo should scale a replication controller  [Conformance]
 [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] listing mutating webhooks should work [Conformance]
 [sig-cli] Kubectl client Update Demo should create and stop a replication controller  [Conformance]
 [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should mutate configmap [Conformance]
 [sig-network] DNS should provide DNS for ExternalName services [Conformance]
 [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should be able to deny pod and configmap creation [Conformance]
 [sig-api-machinery] CustomResourceConversionWebhook [Privileged:ClusterAdmin] should be able to convert from CR v1 to CR v2 [Conformance]
 [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should unconditionally reject operations on fail closed webhook [Conformance]
 [sig-api-machinery] CustomResourceConversionWebhook [Privileged:ClusterAdmin] should be able to convert a non homogeneous list of CRs [Conformance]
 [sig-api-machinery] Aggregator Should be able to support the 1.17 Sample API Server using the current Aggregator [Conformance]
 [sig-network] Proxy version v1 should proxy through a service and a pod  [Conformance]
 [sig-network] DNS should provide DNS for pods for Hostname [LinuxOnly] [Conformance]
 [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should be able to deny custom resource creation, update and deletion [Conformance]
 [sig-cli] Kubectl client Guestbook application should create and stop a working application  [Conformance]
 [sig-api-machinery] AdmissionWebhook [Privileged:ClusterAdmin] should mutate custom resource [Conformance]

This appears to be working now when using VMs with dedicated CPU, so I'm going to close it out


process::Process::net_io_counters for macOS

It seems that it would be impossible (or at least excessively hard) to implement this for Windows (same goes for #125), so probably process::Process::net_io_counters should be demoted and moved into the Linux extension trait.


Java Client Driver for talking to DiceDB

Self Explanatory Title, I would like to contribute by building a Java based driver that can be plugged into Java applications to talk to a DiceDB Instance, similar to https://redis.io/docs/latest/develop/connect/clients/java/lettuce/ for Redis.

Hi @KamaleshProjects why can't the lettuce client simply connect to DiceDB instance instead of connecting to redis server instance?

Hi @shivamprasad99 ,
My bad, I just saw that DiceDB is a drop in replacement for Redis, then all the redis clients would be compatible. Thanks.


Make JavaSerializationAMQPMessageConverter compatible with Axon 2

PR for issue #358

Hey @renedewaele,
is there a specific reason you're changing the existing class? Why not create a new one that can deal with legacy event formats (and omits the aggregate type). For users that didn't come from Axon 2, I'd prefer to have Events in the original format (as much as possible) when reading from AMQP.

Hey @abuijze
As far as I can see this class only exists to deal with 'legacy' events from Axon 2 applications (it's even in a 'legacy' package). In the documentation it is also remarked that this is its purpose and the documentation recommends against using this for non-legacy events. Therefore, I think it's quite safe to say that the number of Axon 3 applications that use this class to deal with Axon 3 events is (close to) zero.

Thanks for the fast reply. And that obviously a perfectly valid reason... didn't even notice the "legacy" package.


Keep correct ordering of chunks when building a body value

Hi,
It seems that there is a subtle bug in body buildup function where chunks are accumulated.
Current handler which processes single incoming chunk calls multiple functions internally before it appends result to the body value (we have Ref.modify and its callback, void and multiple calls inside Stream.onDataString):
https://github.com/cprussin/purescript-httpure/blob/master/src/HTTPure/Body.purs#L87
It seems that sometimes javascript event loop interrupts chunk processing (probably on one of this internal function call frames) and runs handler for the next chunk before appending currently processed chunk. As a result it is possible to get body which has correct length but is a concatenation of chunks in a wrong order. We have noticed this behavior only for large chunks and on a nodejs instance behind nginx reverse proxy...
These are related changes which seem to fix this problem:
Body.purs
Body.js
I can provide clean PR if you suggest how would you like to fix this bug. I'm not sure if I able to provide any regression for it...

Interesting bug!  It seems like this should be fixable without resorting to FFI no?  If there's no fix outside FFI, then I believe that the FFI portion and purescript binding should be part of purescript-node-buffer instead since it sounds like a bug caused by the binding itself.  Other than that, the way you have it fixed makes sense, the only modification I would request is that you add a comment explaining the bug and why it's coded that way since we won't likely be able to add a unit test for this case.

For what it's worth, I've had it in the back of my mind to revisit this for some time.  I'd like to support streaming requests (#63) so likely we'll end up getting rid of this as being the default and expose it as a helper instead.  The fix will still need to go in the helper of course, so still worth fixing here.

@paluh any chance you've been able to put more thought into this one?  If there's any reason we can't get the FFI binding in -node-buffer, I'm happy to accept a PR to put one in httpure temporarily, at least so that httpure isn't buggy.

Hi @cprussin ,

any chance you've been able to put more thought into this one?

Yeah, I thought a bit about this during my offline weekend ;-)

If there's any reason we can't get the FFI binding in -node-buffer (..)

I'm not sure if I'm able to propose any serious and reasonable API change / improvement for -node-buffer at the moment...

I'm happy to accept a PR to put one in httpure temporarily, at least so that httpure isn't buggy.

I'm going to provide it today evening (around 20:00 - 22:00 09-24 CEST). Should I leaveaggregateChunk like it is in Body.purs or move it to some Utils/Node/Stream.purs  submodule?

Woot, thanks @paluh!  Much appreciated!

I'm not sure if I'm able to propose any serious and reasonable API change / improvement for -node-buffer at the moment...

At the very least, I think we should open the discussion in the form of a issue on -node-buffer.  We can defer to the folks most familiar with the -node-buffer package to help us come up with the correct solution, but I certainly think long term this problem should be solved by that package, not this one, so we should start the discussion.  I'd just prefer you to, since I haven't seen any such bug and so I can't provide details on repro cases or theories...

At the very least, I think we should open the discussion in the form of a issue on -node-buffer (...) so we should start the discussion. I'd just prefer you to, since I haven't seen any such bug and so I can't provide details on repro cases or theories...

Ok, I'm going to open an issue under -node-buffer and use our case from HTTPure.

No problem, it's all open source :) I'm happy to weigh in as I can, but tbh it doesn't really make sense to me how this could be happening (node shouldn't interrupt a synchronous call stack) and I haven't been able to repro so I'm not so sure how to advist


Implement modal to add user to admin group

Issue #:
https://github.com/bcgov/met-public/issues/718
Description of changes:

Create resource and service methods to add user to keycloak group
Add modal in the front-end to select user and role

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the met-public license (Apache 2.0).

Codecov Report

Merging #1084 (97602ba) into main (ecbd947) will increase coverage by 0.60%.
The diff coverage is 44.00%.

:exclamation: Current head 97602ba differs from pull request most recent head e97852f. Consider uploading reports for the commit e97852f to get more accurate results

@@            Coverage Diff             @@
##             main    #1084      +/-   ##
==========================================
+ Coverage   73.52%   74.13%   +0.60%
==========================================
  Files         227      227
  Lines        6530     6552      +22
  Branches      451      451
==========================================
+ Hits         4801     4857      +56
+ Misses       1659     1625      -34
  Partials       70       70

Flag
Coverage Δ

metapi
79.33% <36.36%> (+1.14%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

met-web/src/apiManager/endpoints/index.ts
100.00% <ø> (ø)

met-api/src/met_api/services/user_service.py
85.48% <14.28%> (-9.07%)
:arrow_down:

met-api/src/met_api/resources/user.py
76.92% <46.66%> (-12.83%)
:arrow_down:

met-web/src/apiManager/httpRequestHandler/index.ts
58.82% <100.00%> (ø)

met-web/src/services/userService/index.ts
33.33% <100.00%> (ø)

met-api/src/met_api/utils/enums.py
100.00% <0.00%> (ø)

met-api/src/met_api/utils/token_info.py
100.00% <0.00%> (ø)

... and 8 more


typography

started adding html for the typography section and added font to css body

I saved this as a Trish branch and created a draft pull request. I started to add the typography to html and added google font to the css. I am not sure this is how to do it!


Mistake in Convert_Helper__ParserPart__STRING_DATE___Dx.Helper__DetectPartPositions

https://github.com/ibprovider/Lcpi.EFCore.LcpiOleDb-v06.00/blob/f9a3ad907c72afe26f749b586552e9e3d9f3fa0b/Code/Provider/Source/Core/Engines/Dbms/Firebird/Common/Mirror/Value/Converts/Helper/Convert_Helper__ParserPart__STRING_DATE___D1.cs#L538
Must be
if(separators[0]=='-' && separators[1]=='-')

The equal problem in D3 code:
https://github.com/ibprovider/Lcpi.EFCore.LcpiOleDb-v06.00/blob/f9a3ad907c72afe26f749b586552e9e3d9f3fa0b/Code/Provider/Source/Core/Engines/Dbms/Firebird/Common/Mirror/Value/Converts/Helper/Convert_Helper__ParserPart__STRING_DATE___D3.cs#L516

Fixed


Set Source as Stream or byte[]

Hi,
Thanks for the library. Is there a way to set the source as Stream instead of uri?
Thanks

Hello.
Please see other issues https://github.com/vitalii-vov/Maui.PDFView/issues/8 to avoid creating duplicates.


refactor: refactor throwError operator to use factory function in unit tests and .ts file

…

Description:

As per docs, throwError accepts factory functions to emit an error and no longer able to accept an error directly. That's why i went ahead and made changes in all unit tests wherever throwError is getting used to accept factory function.

Related issue (if exists):
Noop

Hi @cartant, i am little confuse here. Can you please help me with what i understood is correct or not?
As per your suggestion,

changes which needs to be done are in 'dtslint/observables/throwError-spec.ts' file only, right?
'dtslint/observables/throwError-spec.ts' should have 2 versions of throwError and is should look like this :

`it('should accept any type and return never observable', () => {
const a = throwError(1); // $ExpectType Observable
const a1 = throwError(() => (1)); // $ExpectType Observable
const b = throwError('a'); // $ExpectType Observable
const b1 = throwError(() => ('a')); // $ExpectType Observable
const c = throwError({a: 1}); // $ExpectType Observable
const c1 = throwError(() => ({ a: 1 })); // $ExpectType Observable
const d = throwError(() => ({ a: 2 })); // $ExpectType Observable
});
it('should support scheduler', () => {
const a = throwError(1, animationFrameScheduler); // $ExpectType Observable
const a1 = throwError(() => (1, animationFrameScheduler)); // $ExpectType Observable
});`

should i make this change in all of the files?

Please correct me if i am wrong.

I'd create separate it functions to make it clearer:
it('should support an error value and a scheduler', () => {
  const a = throwError(1, animationFrameScheduler); // $ExpectType Observable<never>
});`

it('should support a factory and a scheduler', () => {
  const a = throwError(() => 1, animationFrameScheduler); // $ExpectType Observable<never>
});`

Got it. Thanks @cartant for quick support. I have committed this changes. Please take a look at it.


No error seen on the UI when cluster upgrade fails on an RKE2 cluster

Rancher Server Setup

Rancher version: 2.6head commit id: 11a7451
Installation option (Docker install/Helm Chart): docker install

Information about the Cluster

Kubernetes version: v1.21.9+rke2r1
Cluster Type (Local/Downstream): Linode node driver cluster - 1 etcd, 1 cp and 3 worker nodes. Container network - [multus-canal]

User Information

What is the role of the user logged in? (Admin/Cluster Owner/Cluster Member/Project Owner/Project Member/Custom) Standard user

Describe the bug
No error seen on the UI when cluster upgrade fails.
To Reproduce:

Deploy an RKE2 cluster.

When the cluster is Active, Edit Config --> Modify the Upgrade strategy to

Save changes made.

Edit cluster config again. Change the k8s version to v1.22.6+rke2r1 and save changes made.

The cluster goes into Provisioning state.

Worker node is stuck in Cordoned state

Error in rancher Logs:

2022/02/13 23:07:29 [ERROR] error syncing 'fleet-default/sowmya-ec2-1-bootstrap-template-b29nk-machine-plan': handler machine-drain: cannot delete Pods with local storage (use --delete-emptydir-data to override): kube-system/rke2-metrics-server-8574659c85-82sqv, requeuing

Expected Result:
There should be an error displayed on the UI why the cluster upgrade failed. User must not have to log into the server to see the error.

Given the difference in functionality between RKE1 and RKE2/K3s, it is very difficult to interpolate the exact reason a plan failed when updating.
I think this is functionality that we can add down the line, otherwise, maybe we just add a "plan has failed to apply N number of times, please check rancher-system-agent.service logs" or something to indicate that the plan is failing

On 2.6-head commit id: 024e6fa

Deployed an RKE2 cluster - 3 etcd, 2 cp and 3 worker nodes
Upgrade strategy is set on the cluster
Edit cluster --> Upgrade k8s version from 1.22 to 1.23
Error seen on the cluster -
Draining etcd node(s) sowmya-do3-pool1-6c6f5f7f4f-x66td: error draining machine sowmya-do3-pool1-6c6f5f7f4f-x66td: error when evicting pods/"calico-typha-6546b78f-qpg5l" -n "calico-system": global timeout reached: 30s
But the state of the cluster is Provisionining
And in another scenari, on the control plane error Draining control plane node(s) sowmya-test-pool2-56d9c84585-xx5hz: error draining machine sowmya-test-pool2-56d9c84585-xx5hz: cannot delete Pods with local storage (use --delete-emptydir-data to override): cattle-system/system-upgrade-controller-5b6457d644-n8qhx

Logging a separate bug for the status being shown as Provisioning. -


Random crash upon waking computer from sleep

Describe the bug
Sorry, not really sure what this bug is or how to reproduce. I woke my computer up from sleep and orbstack jumped and told me "something went wrong" and apparently there's been a crash and I had to restart orbstack 🤷 did not do anything special, just know that the computer had 9% battery when I woke it up
To Reproduce
No response
Expected behavior
No response
Diagnostic report (REQUIRED)
OrbStack info:
Version: 1.5.0
Commit: afc6a0a9a22761c33329835c75a3b42a4ceb29b8 (v1.5.0)
System info:
macOS: 14.3.1 (23D60)
CPU: arm64, 10 cores
CPU model: Apple M1 Max
Model: MacBookPro18,2
Memory: 32 GiB
Full report: https://orbstack.dev/_admin/diag/orbstack-diagreport_2024-03-25T08-07-44.919257Z.zip

Screenshots and additional context (optional)
No response

Happened again, this time not after waking up from sleep, just randomly
OrbStack info:
  Version: 1.5.0
  Commit: afc6a0a9a22761c33329835c75a3b42a4ceb29b8 (v1.5.0)

System info:
  macOS: 14.3.1 (23D60)
  CPU: arm64, 10 cores
  CPU model: Apple M1 Max
  Model: MacBookPro18,2
  Memory: 32 GiB

Diagnostic report failed: mkdir /Users/daniel/.orbstack/diag: no space left on device
<!-- (To review the report, check ~/.orbstack/diag) -->

Diagnostic report failed: mkdir /Users/daniel/.orbstack/diag: no space left on device

Your computer is completely out of disk space. Please free some space.
See #882


CSS Import Error on Nuxt3 generate

Bug report 🐞
Importing<EMAIL_ADDRESS>from nuxt3 and entering the nuxt generate command results in an error stating that ../../styles/common.css was not found as shown above.
Version & Environment
{
  "private": true,
  "scripts": {
    "build": "nuxt build",
    "dev": "nuxt dev",
    "generate": "nuxt generate",
    "preview": "nuxt preview"
  },
  "devDependencies": {
    "@nuxtjs/eslint-config-typescript": "^10.0.0",
    "@typescript-eslint/eslint-plugin": "^5.24.0",
    "@typescript-eslint/parser": "^5.24.0",
    "eslint": "^8.15.0",
    "eslint-plugin-vue": "^8.7.1",
    "nuxt": "3.0.0-rc.3",
    "typescript": "^4.6.4",
    "vite-plugin-eslint": "^1.6.0"
  },
  "dependencies": {
    "@css-render/vue3-ssr": "^0.15.10",
    "@pinia/nuxt": "^0.1.9",
    "@varlet/ui": "^1.27.8",
    "less": "^4.1.2",
    "less-loader": "^10.2.0",
    "pinia": "^2.0.14"
  }
}

package.json
import * as Varlet from '@varlet/ui';
import<EMAIL_ADDRESS>
export default defineNuxtPlugin((nuxtApp) => {
  nuxtApp.vueApp.use(Varlet);
});

plugins/varlet-ui.ts
Expectation
Actual results (or Errors)
 ERROR  Rollup error: Could not load raw:../../styles/common.css: ENOENT: no such file or directory, open '../../styles/common.css'                                                          nitro 14:11:48
 ERROR  Could not load raw:../../styles/common.css: ENOENT: no such file or directory, open '../../styles/common.css'

try in nuxt.config.ts
export default defineNuxtConfig({
  build: {
    transpile: ['@varlet/ui']
  }
})

@haoziqaq Awesome! That's worked for me. Thank you!

@haoziqaq Do you happen to have a document referenced to think of a solution?

This is a mistake in our documentation.
Our css files are exposed through js, so babel needs to be transpile, we will supplement this part of the documentation later
Thanks for your feedback!


release

Codecov Report

Merging #26 (b422b62) into master (3abb064) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##            master       #26   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           11        11
  Lines          491       526   +35
=========================================
+ Hits           491       526   +35

Impacted Files
Coverage Δ

maxheap/maxheap.go
100.00% <100.00%> (ø)

minheap/minheap.go
100.00% <100.00%> (ø)

priorityqueue/priorityqueue.go
100.00% <100.00%> (ø)

unionfind/unionfind.go
100.00% <100.00%> (ø)

utils/arraylist/arraylist.go
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3abb064...b422b62. Read the comment docs.


TeamCoco.com videos returning 404s and other errors

Experiencing errors when downloading videos from TeamCoco.com. Video 1 and video 2 I tested with, amongst others. Not sure if the issue is youtube-dl or something my end.
Full log of video 1:
[debug] System config: []
[debug] User config: ['-o', 'M:\\Videos\\%(title)s.%(ext)s', '--restrict-filenames', '--console-title']
[debug] Command-line args: ['http://teamcoco.com/video/full-episode-weds-3-4-conan-in-cuba', '--verbose']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2<IP_ADDRESS>
[debug] Python version 2.7.8 - Windows-8-6.2.9200
[debug] exe versions: ffmpeg N-66742-gf6777ce, ffprobe N-56999-g83fc6c8
[debug] Proxy map: {}
[Teamcoco] full-episode-weds-3-4-conan-in-cuba: Downloading webpage
[Teamcoco] 88035: Downloading embed page
ERROR: Unable to download webpage: HTTP Error 404: Not Found (caused by HTTPError()); please report this issue on https:
//yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl
 with the --verbose flag and include its complete output.
  File "youtube_dl\extractor\common.pyo", line 314, in _request_webpage
  File "youtube_dl\YoutubeDL.pyo", line 1674, in urlopen
  File "urllib2.pyo", line 410, in open
  File "urllib2.pyo", line 523, in http_response
  File "urllib2.pyo", line 448, in error
  File "urllib2.pyo", line 382, in _call_chain
File "urllib2.pyo", line 531, in http_error_default

And the second:
[debug] System config: []
[debug] User config: ['-o', 'M:\\Videos\\%(title)s.%(ext)s', '--restrict-filenames', '--console-title']
[debug] Command-line args: ['http://teamcoco.com/video/clueless-gamer-assassins-creed-unity', '--verbose']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2<IP_ADDRESS>
[debug] Python version 2.7.8 - Windows-8-6.2.9200
[debug] exe versions: ffmpeg N-66742-gf6777ce, ffprobe N-56999-g83fc6c8, rtmpdump 2.4
[debug] Proxy map: {}
[Teamcoco] clueless-gamer-assassins-creed-unity: Downloading webpage
[Teamcoco] 86049: Downloading embed page
Traceback (most recent call last):
  File "__main__.py", line 19, in <module>
  File "youtube_dl\__init__.pyo", line 403, in main
  File "youtube_dl\__init__.pyo", line 393, in _real_main
  File "youtube_dl\YoutubeDL.pyo", line 1447, in download
  File "youtube_dl\YoutubeDL.pyo", line 648, in extract_info
  File "youtube_dl\extractor\common.pyo", line 275, in extract
  File "youtube_dl\extractor\teamcoco.pyo", line 63, in _real_extract
TypeError: 'long' object has no attribute '__getitem__'

@Fironet, thanks for the report, however, in the future, please open separate issues for seemingly different problems.
The first issue is due to the fact that we use embed pages to extract information, and the video in question (88035) can no longer be embedded.
As for the second issue, I believe it has already been fixed, please upgrade to the latest version (see our FAQ, if you need help updating).

I wasn't sure if they were related tbh, and seeing 1000+ other issues didn't want to split them considering it was the same domain, but will do from now on.
Thanks, updating to the very latest version fixed the second video. It's a pity the first can't be downloaded. I'll keep the embedding limitation in mind next time it returns a 404.

Can you play the video on the browser? It doesn't work for me with Firefox or Chrome.

Not here on Chrome. I'm thinking it may be a regional restriction. although with some sites (such as CC.com) youtube-dl grabs it just fine.

If it's only available on the USA, we'll need someone to test if it can be played and downloaded from there.

It plays for me in browser. Get the following error:
[debug] System config: []
[debug] User config: ['-wit', '--write-srt', '--sub-lang', 'en', '--restrict-filenames', '-f', '18/43/34/h264-sd/flv/mp4-sd/mp4/rtmp-750/worst/best']
[debug] Command-line args: ['--verbose', 'http://teamcoco.com/video/full-episode-weds-3-4-conan-in-cuba']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2015.03.09
[debug] Python version 2.7.5+ - Linux-3.10-3-amd64-x86_64-with-debian-jessie-sid
[debug] exe versions: avconv 1.2.4, avprobe 1.2.4, ffmpeg 1.2.4, ffprobe 1.2.4, rtmpdump 2.4
[debug] Proxy map: {}
[Teamcoco] full-episode-weds-3-4-conan-in-cuba: Downloading webpage
ERROR: Unable to extract video id; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/home/damack/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 649, in extract_info
    ie_result = ie.extract(url)
  File "/home/damack/bin/youtube-dl/youtube_dl/extractor/common.py", line 275, in extract
    return self._real_extract(url)
  File "/home/damack/bin/youtube-dl/youtube_dl/extractor/teamcoco.py", line 50, in _real_extract
    self._VIDEO_ID_REGEXES, webpage, 'video id')
  File "/home/damack/bin/youtube-dl/youtube_dl/extractor/common.py", line 557, in _html_search_regex
    res = self._search_regex(pattern, string, name, default, fatal, flags, group)
  File "/home/damack/bin/youtube-dl/youtube_dl/extractor/common.py", line 547, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)

The first video is gone, and I've fixed Teamcoco for downloading the second video mentioned by @Fironet without errors.


By default set the event create price to euro, and allow zero

During event creation, set the price currently by default to EUR
Allow setting the price value to zero
If the price is set to zero on a non free currency, revert to setting the currency to free when sending to server

Hey Lucian, I'd like to work on this

I believe someone else has been working on it, I can't remember atm, I'll make a note to ask in the dev meeting


Merge GO:1902187 negative regulation of viral release from host cell and GO:0044790 negative regulation by host of viral release from host cell

Merge GO:1902187 negative regulation of viral release from host cell and
GO:0044790 negative regulation by host of viral release from host cell
*Merge and change label to 'suppression of viral release by host'
*change subclass to 'defense response to virus' (instead of GO:1902186 regulation of viral release from host cell)

move 'GO:0039587 suppression by virus of host tetherin activity' under  'mitigation of host defenses by virus'

To be consistent with other terms, we could consider changing the label to  'evasion of suppression of viral release by host via inhibition of host tetherin activity', but that is not very clear, so I will leave 'regulation of activity' for now.

Change label to symbiont-mediated-mediated suppression of host tetherin activity


Refactor screenshot tool to be standalone

We're calling the whole of the screenshot tooling "pixacto". It consists of code to:

Take screenshot from jasmine-based e2e tests
Store those screenshots in Firebase
Compare test results to "golds" and set PR status to pass/fail based on differences
Provide a control panel UI where anyone can view the image diffs and project owners can approve the changes.

Technical overview doc
Draft set-up guide

[ ] Move code into own repo (angular/pixacto)
[ ] Extract all screenshot related config to be read from a file rather than hard-coded (pixacto.config)

Firebase paths: untrusted results, trusted test results, goldens
GitHub info: repo name, etc. (see tools/screenshot-tests/functions/github.ts)

[ ] Move screenshot.ts from e2e/ and rename capture.ts
[ ] Refactor out gulp/tasks/screenshot.ts into a standalone module, something like compare.
[ ] Curate bits of how the whole thing works from the overview documentation for a README
[ ] Finalize the set-up guide based on @tinayuangao's initial draft guide (restricted to angular.io accounts). Some of the items include:

Adding the JWT as the in Travis
Creating the unauthenticated "untrusted" screenshot bucket in Firebase
Adding authorized users to approval process
Adding users to firebase permissions
Create functions/index.js and what that should contain

I think the ultimate structure for the project should look something like
src/
├── capture/        (capturing screenshots in e2e tests)
├── compare/        (comparing test result screeenshots to golds)
├── control-panel/  (UI for showing and approving diffs)
|   └── functions/
|       ├── config.json
|       └── index.js
├── firebase/       (adding event handlers, copying data)
├── github/         (setting github status)
└── jwt/            (verifying the json web token)

I have just come across that issue. Thanks for sharing the path going forward!
We have been facing the same need for Material-UI.
We came up with two independent solutions.

vrtest A JavaScript library for running visual regression tests on your components cross browser via selenium.
argos-ci a hosted service to automate visual regression testing.

Here is an example of build. I'm gonna follow that thread. I'm really curious to see the solution deployed 💅 .

Technical overview doc
Draft setup guide


provide an interface to query Emulator server status

Issue To Be Solved
It is a commonly encountered issue that there are intermittent failures for tests backed by flow-js-testing library, per described in https://github.com/onflow/flow-js-testing/issues/47.
It turns out the root cause is that there is not a reliable way for processes to check whether the emulator is fully started, hence tests could be run before HTTP/gRPC servers are ready, causing failures.
Suggest A Solution
Rather than relying on this logging line, there should be a reliable interface (i.e. maybe /status) provided by the emulator to reflect the status of the servers

There are actually a couple ways to do this already.
An Access API client can call the Ping gRPC method, which returns OK if the gRPC server is running.
I don't advise using this one outside of a hosted environment (e.g. kubernetes cluster), but the emulator also has an HTTP liveness endpoint exposed that should work with GET http://localhost:8080/live. It returns 200 if the server is running.

There are actually a couple ways to do this already.
An Access API client can call the Ping gRPC method, which returns OK if the gRPC server is running.
I don't advise using this one outside of a hosted environment (e.g. kubernetes cluster), but the emulator also has an HTTP liveness endpoint exposed that should work with GET http://localhost:8080/live. It returns 200 if the server is running.

Good callout @psiemens  👍
Per our sync, here's what we will do:

update flow-js-testing library to use HTTP endpoint /ping (alpha version)
update KittyItems to use this Alpha version of js testing lib, verify the correctness in CI environment
once access node APIs have RESTful version in the emulator, change flow-js-testing lib to use /ping endpoint instead of /live

I'll close this one and open a new task under this repo to track the more comprehensive initiative -- "implementing RESTful APIs in emualtor"


fix: explainPolicy piggyback owner's privilige

这个 PR 实现什么功能/修复什么问题:
改进：explainpolicy会携带执行者的权限信息

是否需要 backport 到之前的 release 分支:
None

/cc @zexi
/area apigateway

Codecov Report

Merging #6763 into master will decrease coverage by 0.00%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master    #6763      +/-   ##
==========================================
- Coverage    7.14%    7.14%   -0.01%
==========================================
  Files         648      648
  Lines      109565   109568       +3
==========================================
- Hits         7830     7828       -2
- Misses     101136   101140       +4
- Partials      599      600       +1

Flag
Coverage Δ

#aFlag
7.14% <0.00%> (-0.01%)
:arrow_down:

Impacted Files
Coverage Δ

pkg/cloudcommon/policy/policy.go
1.49% <0.00%> (-0.02%)
:arrow_down:

pkg/appsrv/response.go
66.17% <0.00%> (-2.95%)
:arrow_down:

pkg/cloudcommon/validators/validators_cert.go
39.77% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d1d2245...e7b198a. Read the comment docs.

/lgtm
/approve


Timezones: Add template variable support

If template variables exist, user can select them for timezone. If a correct string exists, it will use that to update timezone, otherwise, it will default to local time.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

I apologise for the late turnaround on this @drew08t . Code looks great. Just one change needed prior to merging then we can get this release out the door! 🚀

No problem! Appreciate the help!


Netlify CMS 2.0

Overview
View Issues
It's 2.0 time, people! 🔥🔥🔥
This issue serves as an all-purpose tracking center for the 2.0 release. It is the source of truth for dates, release strategy, goals, and relevant issues, and all of these things will shift (especially early on) based on the discussions that take place in the issue.
Tracking
Track the status of all 2.0 issues on the 2.0 project board.
Voting
If you're hoping to see something in 2.0 that isn't currently listed below, you have two options:

See if someone has commented with a request for the feature, and thumb up the comment. If there is no comment for it, add one. Please make sure your comment links to an issue.
Pitch in! A solid pull request implementing a new feature is the best way to get it in.

Comments receiving a high volume of thumbs up will improve the chances of the feature being prioritized.
Note: we can't guarantee that every item added to the checklists below will make it into 2.0 - only that they will be considered a very high priority. Netlify CMS is a community project, and the more folks pitch in, the closer we'll all get to seeing 100% of our 2.0 goals complete. Let's make it happen!
Timing
The release date for Netlify CMS 2.0 is July 24th.
Release Strategy
Moving forward, 2.0 work will occur on the master branch, and a dedicated release branch will be used for 1.x maintenance releases.
Goals
We've been saying for months that the themes of 2.0 are extensibility and growth. Here's what that's come to mean:
Extensibility
Netlify CMS was created to be extended - editor controls, backends, previews and more can be customized through a minimally restrictive JavaScript API. For 2.0, we want to improve that story by lowering the barrier to entry for extension authoring. Here are the stories for extensibility (checklists TBA):
Improve extension authoring docs
Provide extension starter packages

[ ] Improve widget starter (https://github.com/netlify/netlify-cms-widget-starter)

Rework extension API's

[ ] Backend API (#1134, #1171)
[ ] Preview Template API (#1041, #1311)
[ ] Widget API (https://github.com/netlify/netlify-cms/pull/1131#issuecomment-386804125)

Improve community authored extension discoverability

[ ] Provide community widgets reference (#1286)

Growth
The community around Netlify CMS is the reason for it's success. A whole lot of folks have doubled down on the JAMstack as the path forward for development on the web, and thousands of websites have been deployed using Netlify CMS for content management. The more developers use Netlify CMS, the more developers invest in the project, the more the entire community benefits.
For the growth epic, we want to address issues that are expected to have high impact on adoption for Netlify CMS. Stories include (checklists TBA):
Improve implementation docs
Improve SSG and service compatibility

[x] Bitbucket support (#525)
[x] GitLab support (#517)
[ ] Configurable entry paths (#445, #513)

Provide assistive tooling for CMS implementation

[ ] Configuration generator/validator (#1167)

Image Handling (see umbrella issue #1321)

[ ] Multiple folders for media (#1046)

I would love to see Media Library folders as discussed here. Right now it is a bit painful to maintain all images in a single folder.

Looking forward to bitbucket support!

I have been searching for a data-driven CMS and thought Gatsby hit it close, but the OpenGL requirement is a limiter. Hugo also comes close (but doesn't quite hit the mark). Many static sites are simply database views (in the traditional sense) rendered statically. Building a data-driven component into the core of the CMS without any specific dependency would be ideal. I might suggest support for everything from FADB (filesystem as database) approaches (JSON, TOML, INI) to a modular method to plugin data view renderers (or fetchers) as attempted with Gatsby. A clean data module system would allow plugging in data sources. The pattern and approach isn't new, it just currently has not been integrated cleanly into any static CMS that I am aware of.

@robmuh Netlify CMS can edit data files (TOML, JSON, and YAML currently). What specifically is missing to fulfill your requirements? Sounds like it's still going to come down to your SSG since Netlify CMS simply edits source data and does not behave as a source itself (it's literally just a React app).

Ok, that helps me understand better. When I hear CMS I usually interpret that to mean something with site generation included (or no ssg like wordpress).

Is #1032 going to make it into this release?

Could #1267  Be Added To Version 2.0

Note: I added a blurb in the summary above about voting - comments that get a lot of thumbs up will have the best chance of being prioritized.

Casting my vote for a configuration generator/validator (#1167)

How about supporting Facebook as one of the OAuth platforms to log in to Netlify CMS since it's one of the biggest social network (although Mark is getting some trouble now)

Are we considering the mono-repo setup within 2.0?
I believe this needs to be done sooner than later. @erquhart After our long discussion in chat about how the starter should be setup, I went and did some extensive heads down testing on a mono-repo starter for those wanting to do a pure React driven build of the CMS. Also, the concerns with globals (#1292) needing to be exposed to create widgets also brings this to the forefront.
Initial Findings: (Some opinionated)

There needs to be a core library for an extensible netlify-cms in a pure react app.
netlify-cms-core would have peer dependencies of the default netlify-cms react app as well as a starter example to show how to extend your own custom netlify-cms react app
The published netlify-cms app would extend upon the core library the same as an example starter and keep all the current functionality of 1.x for compatibility.
Not sure if we move to Weback 4 prior or after the mono-repo setup
NelifyCMS supported widgets, previews and editors would be published in the mono-repo and serve as the examples of a starter
netlify-cms-widget-starter should be a mono-repo the same as the netlify-cms to adhere to best practices of creating and publishing a widget, so the publisher will be familiar and will make it easier to become a contributor of the cms.

I made a valiant attempt (failed) at moving the CMS to a mono-repo using Lerna and Yarn workspaces. I will keep on this to at least have a discussion WIP branch if possible. This is a MAJOR restructure of the folder structure in the cms and would be hard to keep rebased on master.
I also attempted and succeeded at creating a mono-repo for the cms starter. The biggest issue is the dependencies being duplicated in the bundles due to a lack of externals and/or peer dependencies. This is what warrants me to suggest we take a hard look at creating a mono-repo sooner than later.

netlify-cms-widget-starter should be a mono-repo

I think it should be in cli
somethink like
netlify widget create <name>

and all netlify widgets in the netlify cms mono repo created by that.

I also would like to see the advanced(minimal) and easy(full) version, where the easy version is only need to include a js and a css(just like 1.x) and advanced version is;

it is minimal (core) version without any widget included
widget should be added manually

It would be great if we allow different orderings of the file lists, such as by last modified or date added. Right now, the ordering of the posts whatnot in a collection appear to be random.

Would love to see nested fields in the name parameter (issue 595. From a UI/UX perspective, not being able to nest makes it pretty ugly and cumbersome to the user and developer pretty quickly.

@robmuh

I oppose doing anything with Facebook, at all.

Given that Netlify CMS is written in React, that might be a problem! 🤣

Being able to define the field name to use for the slug.
At the moment it uses "title", which means you have to have a field called title in a collection, even if it doesn't make sense for the collection content. (For example, in an Authors collection, you have to use "title" as the field name for the field labelled "Author Name".

Being able to define sub-selects / sub-relations.
For example, in Posts, I have a "section" field with a relation to "Sections", so can select "News", "Sport", etc. as the post type.
For a sub-select (or relation), this would restrict the available options based on the prior selection, for example "Local News", "US News", "World News" if News is selected, but "Football", "Hockey" etc if sport is selected.
One select / relation set of choices is dependent on the value of another.

Ability to select which field to display in a relation widget, as distinct from which field is the valueFiled that is saved. (For example displaying the title of a related post, but saving the slug.)

@mbrookes can you link issues to those comments?

I saw that Forestry.io just released a 'blocks' feature that lets you choose and order includes with their own sets of properties. Essentially they're  "mini templates" inside a content section.
I'd love to see something along these lines. I'm trying to figure out how I could achieve something similar now - perhaps using the List widget.. but all the fields would have to be identical?
https://forestry.io/docs/settings/fields/blocks/

@liquidvisual didn't know about the Forestry feature, but there is a widget in the works that fits what you're describing: https://github.com/netlify/netlify-cms/pull/1169

@erquhart Wow, that's awesome - I'll keep a close eye on it. Thanks!

@erquhart I've found and linked a couple of related issues. Shall I add comments to those, or is the link sufficient?
For the parent / child select / relation suggestion (for want of a better description), I don't think there's a relevant issue to link to. Should I create a new one?

@erquhart Spot on with #1303 - I missed that one and it's exactly what I had in mind. 👍 I believe there's also an issue related to saving the slug for a relation, so I'll link that too when I find it again.
I can create a new issue for the second suggestion, as I don't thing there's a an existing or related one. (Appreciate it may be a hard nut to crack when it comes to implementation!)

That second issue is an expansion on https://github.com/netlify/netlify-cms/issues/1267, I'd recommend commenting on it with your expanded use case. Still the same general requirement of conditional fields.

I may be misunderstanding, but it seems a bit of a stretch to suggest these are related. #1267 (if I'm reading it correctly) enables an input based on a boolean. Not sure the value of that capabilty, but let's leave it to stand on its own merit.
What I'm suggesting iOTOH s that the secondary relation's collection (Local News, National News, World News, etc.) is determined by the primary relation's selection (News, Sport etc.)

Ah, didn't understand that - agreed, that's a separate issue.

I would love to see #432 added to allow larger image binaries to not always be in the repo which becomes an issue. This could also solve the image manipulation issues as a lot of external image hosts provide that functionality.

I would love to see a mobile version of the CMS (Or a responsive site)

I'd love to see multilingual/localization support (https://github.com/netlify/netlify-cms/issues/716) in 2.0

What about a GraphQL?

I'd love to see more flexible permissions settings, namely the ability to prevent certain roles from accessing/editing the posts of others and to restrict the ability to publish editorial workflow posts .

I'm curious about one thing: upgrade vs breaking changes.
I know it's early...but are you planning and leaning more toward smooth upgrades to go from 1.x to 2.0? Or is it likely that there will be a bunch of breaking changes?

Possibility to add custom widget button directly to the rich-text toolbar and choose its position (access in one click no need to click on +).

Customization of rich-text available buttons and order (show/hide button and position in toolbar).

I see this quiet useful for simple text decoration custom widget like: underline, color, font-size, font-family and such.

Ability to use a separate Git repo for assets (images, video, strings). Main repo would be for app code and the Content repo would be linked, but not actually part of the App repo.

@emveeoh, ...or maybe separate branch

I would love to see more options on how to store the data, I want the option of storing the data in the file system, or in a database. the CMS should separate the concerns of where it stores the data.

@david-genger you have the ability to register a backend today. You can write a backend for your needs and registerBackend('new-backend', NewBackendClass). There will also be some improvements on this in the next release.
This example has a backend that writes to the file-system during development.

@david-genger Right. This is something someone could do outside the CMS. A backend to write to a specific database would be maintained by the community and not be part of the CMS in most cases. I have written a backend myself to write to firebase/firestore for a project (currently private).

@talves why shouldn't this be part of the CMS?

More than one reason in my opinion:

The solution already exists for someone to add their custom backend to the CMS
Hundreds of databases out there, which one would be best maintained by the NetlifyCMS project
There would be no difference of how to create the backend whether it was included in the main project or if you or I created it.
All backends will eventually be ported outside the main bundle into their own library package whether core libraries or custom

Not sure if this is in scope, but for 'documentation' type sites, the option to create 'versions' of the site would be super helpful. e.g: see: https://docusaurus.io/docs/en/versioning.html

@keybits You could take advantage of Netlify's branch-deploy feature. It's how we intend to version the Material-UI docs site - pushing a new branch per version, so branch v1.0.0 will be served as https://v1.0.0.material-ui.com (not live at this moment).

@mbrookes aha - that's a great idea. Thanks!

Also, the list of versions shown on the versions page will be pulled from a JSON file in the github repo, so that the versions page is kept up-to-date even for earlier versions of the docs. The v1.0.0 versions page will display a link to the v1.1.0 docs (when it's released), and so on. (Commented out code here.)

More plugable/extensible/widgets creator friendly or what ever, as seen here and here #1292.
one more issue i found,
I created fontawesome and material icons widgets, both depend on react-virtualized-select
when I use both, they will include (duplicate) react-virtualized-select.
ref #328

Jekyll support.
1.0 seemed to work fine with a HUGO site, Jekyll required a workaround.

@ethancalvo Netlify CMS already works with Jekyll. Check out the jekyll-netlify-cms template or the Netlify CMS docs.

Global settings would be great. Somewhere to store prefs that can be accessed globally (by the SSG) but more importantly by the widgets. I just created a youtube widget that would be greatly enhanced by being able to hook into a general setting for an API Key. As it stands, the key is passed through a field in the widget in config.yml but needs to be passed everytime.

@hennessyevan the Widget API is being reworked under 2.0, and part of that will include allowing widgets to accept a config object at registration, so you'll be able to accept the API key there.

One more feature.
Web Notification

@ekoeryanto can you explain that one a little more by editing the request. Not sure what the use cases would be.

Update
We now have a project board for tracking status of all 2.0 issues.

@erquhart will the release date be respected ?

I have utmost respect for the release date. Missed it while trying to get some small changes merged in, but will release soon.

Heads up: 3 of the top 5 requested features for 2.0 can be solved with this PR: https://github.com/netlify/netlify-cms/pull/1602

Closing - current priorities can be tracked here: https://github.com/netlify/netlify-cms/projects/7


Iterative development with PostGraphQL

We should experiment with support for iterative development with PostGraphQL and other development workflows.
One idea is that in development we could add triggers that will notify us whenever the schema changes so we can reload the server. If we go this route we should probably also add a reloading notification to GraphiQL.
Doing this would also be super useful for refreshing GraphiQL across schema changes.

Idea 💡
What if we also allow people to run Postgres queries in the GraphiQL interface? This would be disabled in production of course.

Added support for watching Postgres (#166), now we just need GraphiQL to auto-reload it’s introspection.

We should have this with https://github.com/calebmer/postgraphql/pull/252 :+1:


Include nightly in Github CI

I noticed nightly is currently broken, might be good to actually have ci testing nightly as well.

I fixed the errors I knew how to, but I'm not really familiar with ffi in Rust yet so I had to leave some broken.

55f4fff3408d1093b874b071d0cc4a7cfdd1fd25 was correct but I already had 2a7e8acc18b05ea47264cefda87ad92e8048699d after I saw your PR. Can you remove the last commit before merging?


On mac cmd + / zooms out the view

looks to be a typo in the config:
            {
                "mac": "cmd+-",
                "key": "ctrl+=",
                "command": "workbench.action.zoomIn"
            },
            {
                "mac": "cmd+/",
                "key": "ctrl+-",
                "command": "workbench.action.zoomOut"
            },

I assume that the correct config should be:
            {
                "mac": "cmd+=",
                "key": "ctrl+=",
                "command": "workbench.action.zoomIn"
            },
            {
                "mac": "cmd+-",
                "key": "ctrl+-",
                "command": "workbench.action.zoomOut"
            },

This is especially annoying as I've become accustomed to doing cmd+/ to comment out code

I noticed the same thing and was going to report it... Drives me nuts!

Thanks to PR this will be fixed. Publishing new version soon.

This breaks all non us keyboards. See my pr here: https://github.com/waderyan/vscode-atom-keybindings/pull/17