[Bug]: Add check for backslash for CI public endpoint when running ./test.sh integ-test

Describe the bug
When a backslash exists in the CI public endpoint, e.g
./test.sh integ-test https://ci.opensearch.org/ci/dbc/bundle-build/1.2.0/998/linux/x64/
an extra backslash is appended to the metadata url leading to download error.
Downloading https://ci.opensearch.org/ci/dbc/bundle-build/1.2.0/998/linux/x64//builds/maven/org/opensearch/client/test/test/maven-metadata.xml
Below is the complete error log.
Installing dependencies in . ...
Installing dependencies from Pipfile.lock (c3492c)...
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/0 — 00:00:00
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
Running ./src/run_integ_test.py https://ci.opensearch.org/ci/dbc/bundle-build/1.2.0/998/linux/x64/ ...
2021-11-16 19:36:53 INFO     Loading ./src/test_workflow/config/test_manifest.yml
2021-11-16 19:36:53 INFO     Loading https://ci.opensearch.org/ci/dbc/bundle-build/1.2.0/998/linux/x64/dist/manifest.yml
2021-11-16 19:36:53 INFO     Loading https://ci.opensearch.org/ci/dbc/bundle-build/1.2.0/998/linux/x64/builds/manifest.yml
2021-11-16 19:36:54 INFO     TestRecorder recording logs in /usr/share/opensearch/workspace/opensearch-build/test-results/603404e77ba44d36a7b5e4372a5a27de/integ-test
2021-11-16 19:36:54 INFO     Downloading to /usr/share/opensearch/.m2/repository ...
2021-11-16 19:36:54 INFO     Downloading https://ci.opensearch.org/ci/dbc/bundle-build/1.2.0/998/linux/x64//builds/maven/org/opensearch/client/test/test/maven-metadata.xml into /usr/share/opensearch/.m2/repository/org/opensearch/client/test/test/maven-metadata.xml ...
Traceback (most recent call last):
  File "./src/run_integ_test.py", line 65, in <module>
    sys.exit(main())
  File "./src/run_integ_test.py", line 36, in main
    dependency_installer.install_maven_dependencies()
  File "/usr/share/opensearch/workspace/opensearch-build/src/test_workflow/dependency_installer.py", line 33, in install_maven_dependencies
    self.download(maven_artifacts, "builds", self.maven_local_path)
  File "/usr/share/opensearch/workspace/opensearch-build/src/test_workflow/dependency_installer.py", line 59, in download
    self.__download_or_copy(url, local_path)
  File "/usr/share/opensearch/workspace/opensearch-build/src/test_workflow/dependency_installer.py", line 66, in __download_or_copy
    urllib.request.urlretrieve(source, dest)
  File "/usr/lib64/python3.7/urllib/request.py", line 247, in urlretrieve
    with contextlib.closing(urlopen(url, data)) as fp:
  File "/usr/lib64/python3.7/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib64/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib64/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib64/python3.7/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/usr/lib64/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/usr/lib64/python3.7/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

To reproduce
 ./test.sh integ-test https://ci.opensearch.org/ci/dbc/bundle-build/1.2.0/998/linux/x64/ with a backslash in the end
The head of the commit is ac0082cc30399b20d787352a00f589797ebc8c02
Expected behavior
The metadata xml shall be successfully downloaded similar to manifest xml.
Screenshots
If applicable, add screenshots to help explain your problem.
Host / Environment
No response
Additional context
No response
Relevant log output
No response

[Triage] Need better error handling.

This needs to combine URLs better and handle the / at the end of the input URL.

This issue no longer exists. Ran the the integ-test with below params and found no issue.
./test.sh integ-test manifests/2.4.1/opensearch-2.4.1-test.yml --component ml-commons --test-run-id 3455 --paths opensearch=https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.4.1/6667/linux/arm64/tar/

and
./test.sh integ-test manifests/2.4.1/opensearch-2.4.1-test.yml --component ml-commons --test-run-id 3455 --paths opensearch=https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.4.1/6667/linux/arm64/tar

@gaiksaya please confirm if this can be closed.

I'm still seeing this issue with command ./test.sh integ-test manifests/2.4.1/opensearch-2.4.1-test.yml --component anomaly-detection --paths opensearch=https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.4.1/6667/linux/x64/tar/
2023-01-03 22:52:33 INFO     Downloading https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.4.1/6667/linux/x64/tar//builds/opensearch/plugins/opensearch-job-scheduler-<IP_ADDRESS>.zip into /tmp/tmpnsl2vrus/anomaly-detection/src/test/resources/job-scheduler/opensearch-job-scheduler-<IP_ADDRESS>.zip ...
2023-01-03 22:52:33 INFO     Removing /tmp/tmpnsl2vrus
Traceback (most recent call last):
  File "./src/run_integ_test.py", line 36, in <module>
    sys.exit(main())
  File "./src/run_integ_test.py", line 25, in main
    all_results = IntegTestRunners.from_test_manifest(args, test_manifest).run()
  File "/usr/share/opensearch/opensearch-build/src/test_workflow/integ_test/integ_test_runner.py", line 47, in run
    test_results = test_suite.execute_tests()
  File "/usr/share/opensearch/opensearch-build/src/test_workflow/integ_test/integ_test_suite_opensearch.py", line 49, in execute_tests
    self.__install_build_dependencies()
  File "/usr/share/opensearch/opensearch-build/src/test_workflow/integ_test/integ_test_suite_opensearch.py", line 59, in __install_build_dependencies
    self.__copy_job_scheduler_artifact()
  File "/usr/share/opensearch/opensearch-build/src/test_workflow/integ_test/integ_test_suite_opensearch.py", line 68, in __copy_job_scheduler_artifact
    self.dependency_installer.install_build_dependencies({"opensearch-job-scheduler": job_scheduler.version}, custom_local_path)
  File "/usr/share/opensearch/opensearch-build/src/test_workflow/dependency_installer_opensearch.py", line 42, in install_build_dependencies
    self.download_or_copy(path, local_path)
  File "/usr/share/opensearch/opensearch-build/src/test_workflow/dependency_installer.py", line 64, in download_or_copy
    urllib.request.urlretrieve(source_url, dest)
  File "/usr/local/lib/python3.7/urllib/request.py", line 247, in urlretrieve
    with contextlib.closing(urlopen(url, data)) as fp:
  File "/usr/local/lib/python3.7/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/local/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/local/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/local/lib/python3.7/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/usr/local/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/usr/local/lib/python3.7/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

I also run with the commands @rishabh6788 posted above and they are passing. After some investigation, the reason I think is the integ test run on ml-common doesn't have any build-dependencies according to the test manifest, which actually causes the issue. I will be looking into a path fix.

Closing this issue as it's fixed.


added second level domains to mailcheck

Problem
Some of the domains in the default domains have multiple valid tlds for example hotmail.com and hotmail.co.uk. Presently the way to configure mailcheck for this is to add each valid version to the domains list. This works but is cumbersome if a domain can have any valid tld.
Proposed Solution
I added second level domains to mailcheck which will be checked after comparing full domains in the same step as checking just the tld. This lets you add generic second level domains that can match any tld. If you have a domain with only a couple valid tlds they can still be added to the domain list.
I found this extra level of control quite useful when tweaking mailcheck for our need. This is essentially a cleaned up version of what we are running in production at Shopify:
https://github.com/Shopify/mailcheck/compare/shopify_mods

@kevinhughes27 Thanks for the update!


aegir is still wonky on release

I've experienced two problems:

when build happens, it runs out of memory and throws (because it has to build twice, for browser tests and for release, if we take out of the browser build, it works)
when doing the bump 11.1.0 on a release-minor becomes 111.2.0. Fortunately this time I was paying a lot of attention and stopped it before pushing to github and releasing to npm

@dignifiedquire I believe the first is because the dedup plugin stopped working and so it is taking more memory now. The second error is that gulp-bump thing, but I though it was fixed, thoughts?

I haven't seen any oom issues with webpack what so ever anymore, maybe sth changed recently.
The dedupe plugin has been dropped here: https://github.com/webpack/webpack/pull/3266 but it seems there is an alternative we can try: https://github.com/webpack/webpack/pull/3266#issuecomment-260623603 so this might be the issue you are having
When does the actual error happen? This should be independent of the two builds as they are separated and shouldn't influence each other.
There is an issue open here for the version bump here: https://github.com/stevelacy/gulp-bump/issues/62

When does the actual error happen? This should be independent of the two builds as they are separated and shouldn't influence each other.

@dignifiedquire try to run npm release-minor, it will fail on the build task, but if you disable browser tests (first build), it goes through

aegir is good again :D


feat: Update nri-bundle to version 5.0.61

Update nri-bundle to version 5.0.61

Thank you @elamaran11 and @mikemcd3912 for testing our weekly updates with a quick turnaround time every time. We appreciate the partnership with AWS and are proud of it


⚠ gorzdrav.org has degraded performance

In ac2e62b, gorzdrav.org (https://gorzdrav.org) experienced degraded performance:

HTTP code: 200
Response time: 6614 ms

Resolved: gorzdrav.org performance has improved in 9780388.


[iOS] BoxShadow flickers when scrolling

Steps to reproduce

Put a Container with BoxShadow inside SingleChildScrollView.
Make content large enough to scroll.
When scrolling, BoxShadow flickers.

Expected results
BoxShadow should have steady color without changes.
Actual results
BoxShadow flickers quite a lot.
Code sample
Code sample
import 'package:flutter/material.dart';

void main() {
  runApp(const MainApp());
}

class MainApp extends StatelessWidget {
  const MainApp({super.key});

  @override
  Widget build(BuildContext context) {
    const dimension = 300.0;

    return MaterialApp(
      home: Scaffold(
        body: SingleChildScrollView(
            child: Column(
          children: [
            const SizedBox(height: 200),
            Row(
              mainAxisAlignment: MainAxisAlignment.center,
              children: [
                Container(
                  height: dimension,
                  width: dimension,
                  decoration: BoxDecoration(
                    borderRadius: BorderRadius.circular(dimension / 2),
                    boxShadow: const [
                      BoxShadow(
                        color: Color.fromRGBO(15, 119, 109, 0.3),
                        spreadRadius: 0,
                        blurRadius: 20,
                        offset: Offset(0, 2), /
                      ),
                    ],
                  ),
                ),
              ],
            ),
            const SizedBox(height: 1500),
          ],
        )),
      ),
    );
  }
}

Screenshots or Video

Screenshots / Video demonstration
[Upload media here]

Logs
No response
Flutter Doctor output
Doctor output
Stable
Flutter (Channel stable, 3.22.0, on macOS 14.5 23F79 darwin-arm64, locale en-US)
    • Flutter version 3.22.0 on channel stable at
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 5dcb86f68f (6 days ago), 2024-05-09 07:39:20 -0500
    • Engine revision f6344b75dc
    • Dart version 3.4.0
    • DevTools version 2.34.3

Master
Flutter (Channel master, 3.22.0-35.0.pre.10, on macOS 14.5 23F79 darwin-arm64, locale en-US)
    • Flutter version 3.22.0-35.0.pre.10 on channel master at
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision ddd6c86950 (12 hours ago), 2024-05-14 22:00:17 -0400
    • Engine revision d35a1a603c
    • Dart version 3.5.0 (build 3.5.0-154.0.dev)
    • DevTools version 2.36.0-dev.5

Similar issue was reported, but closed because things worked on master. Tested and this issue on both master and stable.
https://github.com/flutter/flutter/issues/135844

Lets check partial repaint, otherwise its probably related to the blur re-write.

This is not related to partial repaint.
This issue happens when the app is run with a large radius for the BorderRadius.circular.  If it exceeds a threshold, then when _BoxDecorationPainter calls canvas.drawRRect it will go through Impeller's Canvas::DrawOval/Canvas::DrawCircle code path.
One factor contributing to the flickering is that Canvas::DrawOval/Canvas::DrawCircle sometimes uses Canvas::AttemptDrawBlurredRRect.  The blur renders inconsistently depending on whether the input content was generated by Canvas::AttemptDrawBlurredRRect or the standard method.

There also seems to be an issue involving generation of mipmaps for the blur filter input snapshot on Vulkan.
This app renders correctly on Android/Vulkan with these changes:

disable use of Canvas::AttemptDrawBlurredRRect
patch CreateSampler in sampler_vk.cc to use the default zero value for sampler_info.maxLod

or set GaussianBlurFilterContents::kBlurFilterRequiredMipCount to 1

@gaaclarke

Setting VK_LOD_CLAMP_NONE would disable mipmapping. its possible we have a bug in mip generation. Its very manual in Vulkan.
(See also: https://github.com/flutter/engine/pull/51761, https://github.com/flutter/engine/pull/51749)

Found a problem with the barriers on Vulkan. In terms of the flickering, there is a rounding issue in the Skia logic for isSimple that we depend on.
D/skia    (18362): SkRect::MakeLTRB(74, 179.666672f, 374, 479.666656f);
D/skia    (18362): const SkPoint corners[] = {
D/skia    (18362):     { 150, 149.999969f },
D/skia    (18362):     { 150, 150 },
D/skia    (18362):     { 150, 149.999969f },
D/skia    (18362):     { 150, 150 },
D/skia    (18362): };
This rrect is bouncing between this and 150 everywhere. I might just add our own check in.


Update NN/NT/TN F32 Performance

Do we have this kind of test yaml in Tensile/Tests? If not , should we add one?

Do we have this kind of test yaml in Tensile/Tests? If not , should we add one?

This tuning patch is for demo; no need to add tests for now. I will add in the future if needed.


Azure Templates reference doesn't contain api versions newer than 2020

The Reference section of this article generally doesn't contain api versions newer than 2020 even though many exist. Can this be updated to include those new api versions? Is this something that is auto-generated?
For example microsoft.compute versions here https://docs.microsoft.com/en-us/azure/templates/microsoft.compute/allversions
and the microsoft.network versions here https://docs.microsoft.com/en-us/azure/templates/microsoft.network/allversions
both only have entries from 2020 (except the compute one has single 2021 entry for cloudServices).
However there are several newer api version available which can be found by following the "View resource provider" instructions to view available api versions here .  For example in microsoft.compute  for availabilitySets the newer api versions are:

"2021-07-01",
"2021-04-01",
"2021-03-01",
"2020-12-01",
"2020-06-01",

but the newest api version for availabilitySets  in these docs here  is 2020-12-01.
Thank you for taking a look!

@lintFan Thanks for reaching out! AFAIK API specifications in these articles are auto-generated. Tagging @tfitzmac and @mumian to confirm the same and to help provide any additional insights w.r.t your question.

@tfitzmac @mumian Any additional insights on this please?

@lintFan - thanks for this feedback. Those 2 resource providers have now been updated. Other resource providers are also in the process of being updated. I have a new generation process for this documentation that will make it easier to keep these articles up-to-date.
#please-close

@tfitzmac great thank you!


[SPARK-33073][PYTHON][3.0] Improve error handling on Pandas to Arrow conversion failures

What changes were proposed in this pull request?

This improves error handling when a failure in conversion from Pandas to Arrow occurs. And fixes tests to be compatible with upcoming Arrow 2.0.0 release.
Why are the changes needed?

Current tests will fail with Arrow 2.0.0 because of a change in error message when the schema is invalid. For these cases, the current error message also includes information on disabling safe conversion config, which is mainly meant for floating point truncation and overflow. The tests have been updated to use a message that is show for past Arrow versions, and upcoming.
If the user enters an invalid schema, the error produced by pyarrow is not consistent and either TypeError or ArrowInvalid, with the latter being caught, and raised as a RuntimeError with the extra info.
The error handling is improved by:

narrowing the exception type to TypeErrors, which ArrowInvalid is a subclass and what is raised on safe conversion failures.
The exception is only raised with additional information on disabling "spark.sql.execution.pandas.convertToArrowArraySafely" if it is enabled in the first place.
The original exception is chained to better show it to the user (only for Spark 3.1+ which requires Python 3)

Does this PR introduce any user-facing change?

Yes, the error re-raised changes from a RuntimeError to a ValueError, which better categorizes this type of error and in-line with the original Arrow error.
How was this patch tested?

Existing tests, using pyarrow 1.0.1 and 2.0.0-snapshot, and Python 2 with 0.15.1

Thank you, @BryanCutler !

cc @HyukjinKwon @dongjoon-hyun , I tested this locally with Python 2

Test build #129492 has started for PR 29962 at commit 5c3c195.

Test build #129492 has finished for PR 29962 at commit 5c3c195.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/129492/
Test PASSed.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34098/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34098/

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/34098/
Test FAILed.

Merged to branch-3.0.
Thanks @BryanCutler and @dongjoon-hyun.

Thanks @HyukjinKwon and @dongjoon-hyun !


vagrant up errors

==> default: W
==> default: :
==> default: Failed to fetch http://security.ubuntu.com/ubuntu/dists/trusty-security/main/source/Sources  Hash Sum mismatch
==> default: W
==> default: :
==> default: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/trusty-updates/main/source/Sources  Hash Sum mismatch
==> default: W
==> default: :
==> default: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/trusty-updates/main/binary-amd64/Packages  Hash Sum mismatch
==> default: W
==> default: :
==> default: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/trusty-updates/multiverse/binary-amd64/Packages  Hash Sum mismatch
==> default: E: Some index files failed to download. They have been ignored, or old ones used instead.
The SSH command responded with a non-zero exit status. Vagrant
assumes that this means the command failed. The output for this command
should be in the log above. Please read the output to determine what
went wrong.

That's an error running apt-get update against the standard Ubuntu servers and thus not related to Zulip; this could in theory happen if you run it while a mirror is updating, but could also be caused by other things, e.g. a malicious server posing as archive.ubuntu.com or a broken mirror server.
If it's an update issue, trying again should work.


Implementing expiration tolerance in OAuth clients

Hello,
As I understand it, a token is renewed when it is expired.
We can have the following scenario :

OAuth Client calls a REST Service (accessToken is not expired - but quite)
REST Service receives the call and when validating the accessToken, the token has expired.

Could we implement an expiration tolerance (i.e. client-side), meaning the client will refresh the token a few moments (configurable) before its expiry ?

This seems like the usual case where clients attempt to refresh their tokens something like 70% of the way into the lifetime.


DOCS-5691 publish for raji

Description

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate)

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist

[ ] I have updated the documentation accordingly.
[ ] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes if appropriate.
[ ] All new and existing tests passed.

@jonathan-ly the build completed after upgrading to larger runner and migrating to node 20. Hopefully that buys us a bit more time to explore other performance optimizations.


PriorityQueue's Length method returns wrong(?) number of elements

When a PriorityQueue is already closed or dropped the Length method still returns the number of elements available before close/drop.
Maybe 0 or an error would be a better return value in this case.

Hmm, I'm open to hearing ideas on the best way to handle it, let's have that has the first option.
I'm thinking maybe there could be an error returned for each function if the stacks/queues are closed? Something like ErrDBClosed. This would return for all functions if the queue is enclosed, i.e. Enqueue, Dequeue, Peek, Update, etc. Right now, for those functions, it returns the default LevelDB errors.
If this new error is added, should it be returned for the Length function as well?

Nevermind on adding the error to the Length function, that should only return uint64.

Ok I think I'll merge the two ideas, should have a new commit here within an hour or so. Thank you!

Fixed this with commit 39d78f4aade884bbbf0c7de5977c6a597d03ed6e, thank you again!


Catches InvalidFileException when loading environment file with dotenv

Dotenv may throw an exception when loading a .env file, either InvalidPathException or InvalidFileException. At present LoadEnvironmentVariables is only catching InvalidPathException. This means that when you provide an invalid .env file Laravel outputs an unhelpful error message, e.g:
APP_NAME=Laravel Framework

Fatal error: Uncaught ReflectionException: Class config does not exist in /app/vendor/laravel/framework/src/Illuminate/Container/Container.php:767 Stack trace: #0
/app/vendor/laravel/framework/src/Illuminate/Container/Container.php(767): ReflectionClass->__construct('config') #1
/app/vendor/laravel/framework/src/Illuminate/Container/Container.php(646): Illuminate\Container\Container->build('config') #2
[...]

Many are confused by this error: 1, 2, 3, 4, 5, 6.
This change captures the additional error, meaning that when invalid .env file is provided (usually because a value containing a space lacks quotation marks) Laravel will fall back to the default application config.

I think that when this error occurs Laravel should not continue executing[1] because it is a very difficult situation to debug without an error, and it's unlikely a developer will want their application to run without a .env — and if they do, they can create an empty .env file — but I haven't added it in case there is a use case I am not aware of that necessitates allowing the application run even with an invalid .env file provided. Any thoughts on terminating execution here?
[1] As far as I know the only way to output an error here would be dd($e) because the error handling functionality isn't loaded at this point in the bootstrap process.

How do you trigger an InvalidFileException in this case?

The exception is thrown when a value that contains a space is not enclosed in quotation marks, e.g:
APP_NAME=Laravel Framework

https://github.com/vlucas/phpdotenv/blob/master/src/Loader.php#L220
    // Unquoted values cannot contain whitespace
    if (preg_match('/\s+/', $value) > 0) {
        throw new InvalidFileException('Dotenv values containing spaces must be surrounded by quotes.');
    }

+1 for this. It's hard to debug because current Exception message says nothing about .env file.

Surely this will make it harder to debug?

Although Laravel does create a .env file on install and Laravel does by default depend on the APP_KEY value in .env to be secure, it is possible that some Laravel applications out there do not have a .env file and so to implement my preferred solution would potentially break some applications (until they added an empty .env file).
The alternative is to keep the current behaviour and add additional behaviour for InvalidFileException, e.g:
try {
    (new Dotenv($app->environmentPath(), $app->environmentFile()))->load();
} catch (InvalidPathException $e) {
    //
} catch (InvalidFileException $e) {
    dd('Fatal Error: Values in .env containing spaces must be surrounded by quotes.');
}

An application does not have a .env file: the default configuration is loaded
An application has an invalid .env file: the error message ("Values in .env containing spaces must be surrounded by quotes.") is dumped.

I have updated the pull request to use the backwards compatible behaviour — missing .env does not trigger an error, an error is only triggered when the .env is invalid.

I don't think this is right. There are way more reasons parsing can fail than just double quotes missing!

@GrahamCampbell Are you sure? As far as I can tell the only place the InvalidFileException is used is when checking the file for values containing spaces that are missing quotes: https://github.com/vlucas/phpdotenv/blob/master/src/Loader.php#L251 — so the only way this error can occur is if a value is missing quotes.

@GrahamCampbell That would be my preference, if there is a way to handle errors correctly then that would be much better. However, as far as I can tell, at this point in the bootstrap process Laravel doesn't have access to anything necessary to error properly (i.e: the exception system depends on valid configuration) so it would require changes to the architecture. I could very well be wrong there, though, are there ways to error properly here?
Re: dumping exception message vs. a custom error message, the exception message does not explicitly state the issue is with the .env file and some developers (novice, esl) may not make the connection between the package name "Dotenv" and the file ".env", as the exception message is: "Dotenv values containing spaces must be surrounded by quotes.".
That said, anything (whether it's dumping the exception, or a custom error message, or even loading the default configuration) is an improvement over the current non-descript uncaught fatal error that causes so much confusion, so if there's a different approach to this problem that is better than mine then I support it.

This also seems to occur if you have an error (In my case a call to a undefined function) within config.php.


set(Max|Min)Backoff not working

Description
Executable.set(Max|Min)Backoff() are not working on main. I didn't test on any release.
Steps to reproduce
It's easy to reproduce against a down mainnet node (or just changing an up node's port to be invalid):
    Client client = Client.forNetwork(Maps.of("<IP_ADDRESS>:50211", AccountId.fromString("0.0.24")));
    client.setOperator(OPERATOR_ID, OPERATOR_KEY);
    client.setRequestTimeout(Duration.ofSeconds(3));

    new AccountBalanceQuery()
        .setAccountId(OPERATOR_ID)
        .setMaxAttempts(4)
        .setMaxBackoff(Duration.ofMillis(500L))
        .setMinBackoff(Duration.ofMillis(500L))
        .execute(client, Duration.ofSeconds(3));

Additional context
Logs where you can see the min backoff starts at 250ms not 500ms and the maxbackoff goes to 2000ms not 500ms. Can also see the request takes 10s regardless of 3s timeout values:
2022-01-14T10:42:33.719-0600 [main] WARN com.hedera.hashgraph.sdk.AccountBalanceQuery - Retrying node 0.0.24 in 250 ms after channel connection failure during attempt #1
2022-01-14T10:42:44.421-0600 [main] WARN com.hedera.hashgraph.sdk.AccountBalanceQuery - Retrying node 0.0.24 in 500 ms after channel connection failure during attempt #2
2022-01-14T10:42:55.438-0600 [main] WARN com.hedera.hashgraph.sdk.AccountBalanceQuery - Retrying node 0.0.24 in 1000 ms after channel connection failure during attempt #3
2022-01-14T10:43:07.006-0600 [main] WARN com.hedera.hashgraph.sdk.AccountBalanceQuery - Retrying node 0.0.24 in 2000 ms after channel connection failure during attempt #4
Exception in thread "main" com.hedera.hashgraph.sdk.MaxAttemptsExceededException: exceeded maximum attempts for request with last exception being
	at com.hedera.hashgraph.sdk.Executable.execute(Executable.java:209)
Elapsed: 44.07 s
	at com.hedera.hashgraph.sdk.Query.execute(Query.java:29)
	at TransferCryptoExample.main(TransferCryptoExample.java:47)
Caused by: java.lang.IllegalStateException: Failed to connect to node 0.0.24
	at com.hedera.hashgraph.sdk.Executable$GrpcRequest.reactToConnectionFailure(Executable.java:486)
	at com.hedera.hashgraph.sdk.Executable.execute(Executable.java:223)
	... 2 more
Caused by: java.lang.IllegalStateException: Failed to connect to node 0.0.24

Hedera network
mainnet
Version
main
Operating system
macOS

What you are seeing here is not the executable's min/max backoff, instead it's the unhealthy node backoff, which is separate.
The 250 ms min backoff is for the node, not for the executable.  In other words, the node is marked unhealthy for 250 ms, and it will not be used by any executable during that period.
Because that backoff is for the node, not for the executable, the executable's min/max backoffs are not used.
I am not sure if the current behavior is ideal, but I'm pretty sure it is at least correct to not use the executable's min/max backoff here.

That sounds like an implementation detail. That's not how it should work from the user's perspective. The min/max backoff should control the overall retry interval for unsuccessful requests. It doesn't matter if a node is unhealthy or its a retryable application error. If I configure the below, I as a user, expect up to 4 attempts with 0.5s in between for a total of up to 2s in time taken:
    new AccountBalanceQuery()
        .setAccountId(OPERATOR_ID)
        .setMaxAttempts(4)
        .setMaxBackoff(Duration.ofMillis(500L))
        .setMinBackoff(Duration.ofMillis(500L))
        .execute(client, Duration.ofSeconds(3));

If the behavior doesn't match that, it should be changed.


chore: Adds GCP secret manager API

Provide a description of what has been changed

Adds GCP secret manager for Postgres SQL connection string

Checklist

[x] Commits are signed with Developer Certificate of Origin (DCO)

Related to https://github.com/kedacore/keda/pull/4864

I have made the requisite changes, can you PTAL now? Thanks!


Send EpiTrax deindentified info to test connection

We should send some deindentified NV results to NV's new EpiTrax connection to make sure things are working and they're handling our data correctly.
@MauriceReeves-usds or @brick-green could you add any more details here?

Sent fake HL7 messages via email to start the validation process while we wait for whitelisting via CDC palo alto

update from user Linda King<EMAIL_ADDRESS>on 8/25 @ 5:58 PM:
Hey Brick,
No, we don't have any specifics for the file name. Thanks so much for the sample messages, I will be reviewing and getting back to you soon.
Take care.
Linda

Message queued to send to NV.

Test message sent to end point and they confirmed receipt!


Use ISO-861 Duration to determine the date/time that Cassandra Source should start reading data

Issue #278 created the capability to configure the size of the time slice when retrieving data from Cassandra to publish to Kafka. Ideally the default starting date/time would be configurable using ISO-861 duration  instead of the current default of Jan 1, 1900.
https://github.com/datamountaineer/stream-reactor/blob/74f239baaa5f1dca8e7d4f5607186484268ee040/kafka-connect-cassandra/src/main/scala/com/datamountaineer/streamreactor/connect/cassandra/source/CqlGenerator.scala#L34-L35
The following issue (#301) would make the setting of the value configurable.
This would be similar to the FTP connector
Willing to submit PR

Note: the commit allows a different starting date/time to be set but is not using ISO-861 Duration. Do we still want to update this to use ISO-861 Duration?


Log to Sentry if unusual requests happen

🛠 Changelog

Add RequestLog for grouping SQL logs
Log to Sentry if long request happen
Log to Sentry if requests with many SQL queries are made
Log to Sentry if requests with bug request bodies are made

🧪 Test plan

Automated tests

🚧 Dependencies

None

🎫 Tickets

TILA-3325

Rebased this on top of main so that it can be used to solve issues with performance in production.

Merging this PR with #1211 for simplicity.


python310Packages.gflanguages: 0.5.13 -> 5.0.4

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for python310Packages.gflanguages is: Python library for Google Fonts language metadata
meta.homepage for python310Packages.gflanguages is: https://github.com/googlefonts/lang
meta.changelog for python310Packages.gflanguages is: https://github.com/googlefonts/lang/releases/tag/v5.0.4
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes
Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 5.0.4 with grep in /nix/store/h67y1ldygkhzm5di79gmlj8fz0bdp696-python3.10-gflanguages-5.0.4
found 5.0.4 in filename of file in /nix/store/h67y1ldygkhzm5di79gmlj8fz0bdp696-python3.10-gflanguages-5.0.4

Rebuild report (if merged into master) (click to expand)

8 total rebuild path(s)

8 package rebuild(s)

First fifty rebuilds by attrpath
fontbakery
python310Packages.fontbakery
python310Packages.gflanguages
python310Packages.shaperglot
python311Packages.fontbakery
python311Packages.gflanguages
python311Packages.shaperglot
shaperglot

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/h67y1ldygkhzm5di79gmlj8fz0bdp696-python3.10-gflanguages-5.0.4 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A python310Packages.gflanguages https://github.com/r-ryantm/nixpkgs/archive/1f01cde66152eda5ebd4eea778ba4bb44d527f55.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/1f01cde66152eda5ebd4eea778ba4bb44d527f55#python310Packages.gflanguages

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/h67y1ldygkhzm5di79gmlj8fz0bdp696-python3.10-gflanguages-5.0.4
ls -la /nix/store/h67y1ldygkhzm5di79gmlj8fz0bdp696-python3.10-gflanguages-5.0.4/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

6 packages built:

fontbakery (python311Packages.fontbakery)
python310Packages.fontbakery
python310Packages.gflanguages
python310Packages.shaperglot
python311Packages.gflanguages
shaperglot (python311Packages.shaperglot)

Maintainer pings
cc @danc86 for testing.

python310Packages isn't build by Hydra any more, should be safe to close.


Cherry-pick landed PR from LLVM

The last integrate breaks the compilation of internal models because a PR didn't make to the last integrate: https://github.com/llvm/llvm-project/pull/82843 The PR landed already on the LLVM side. Would it make sense to just cherry-pick it or should we wait until next integrate?

The recent llvm integrate pins it to a commit (which was landed 6 hours ago). Maybe you can coordinate with @qedawkins and see if you can bump it to include your commit?

My understanding was that breakages are good enough justification for a cherry-pick, although if it was that recent we could try just bumping straight to the commit in question?

I have no problem at all with cherry picking for breakages.

I just found that the cherry-pick == bump LLVM submodules. The recent one is just one behind @dcaballe 's commit.

ah darn, that's unfortunate, then doubly approved :)

ok I will let you guys figure it out then, but I am ok with whatever works given that there are breakages.

It is landed in https://github.com/openxla/iree/pull/16609, closing the PR


managed client subscriber not recieving messages in the form xxxx/#

When subscribing to the topic xxxx/# the client does not throw the messages out to the subscriber object.
subscription is made just like any other
  var filter = new MqttTopicFilterBuilder()
    .WithTopic( topic )
    .WithQualityOfServiceLevel( (MqttQualityOfServiceLevel)subscription_data.qos_level )
    .Build();

  await _mqtt_client.SubscribeAsync( filter );

and messages recieved via "mqtt_client.UseApplicationMessageReceivedHandler"
inside "UseApplicationMessageReceivedHandler" is the following code:
foreach ( var subscription in _subscriptions[ topic ] ) {
	subscription?.callback?.Invoke( payload );
}

expected behavoiur,
The object that subscribed via "SubscribeAsync" should recieve messages from xxxx/foo and xxxx/bar, but doesnt, because while "args.ApplicationMessage.Topic" never shows in the form "xxxx/#" only for each topic name.  Inside "UseApplicationMessageReceivedHandler" there is no way of telling when to send message to subscriber object apart from going through stored subscriptions and checking each one individually.

I'm not sure if I understand the problem. The topics inside the message received handler are of course topics, not topic filters like xxx\#.

I agree that you cannot use wildcards in topics. So you will never get your expected result. This is by design and not supported in MQTT at all.

Hm. Given that comment on the topic property:

I would have expected to get that back too. It says "uses to filter". Not "result of a match".


cant remove top border while have a label

I am trying to get rid of border when i focus to input but i still have top border.
My main goal changing that focus color to custom color. But when i change to focus color i still have that top border as blue.
<Input
variant='outlined'
label='Static'
containerProps={{
className:
'!min-w-[0px] !w-[200px] mt-2 dark:text-white !dark:focus:border-white border-none focus:ring-0',
}}
className='focus:border-blue-gray-200'
labelProps={{ className: 'before:border-0 peer-focus:border-transparent' }}
/>

Hi @FurkanSezal, I'm too lazy to give you a copy/past solution but looking shortly in my Chrome Developer Tools, I saw this on one of my Inputs.:
<label class="flex w-full h-full select-none pointer-events-none absolute left-0 font-normal !overflow-visible truncate peer-placeholder-shown:text-blue-gray-500 leading-tight peer-focus:leading-tight peer-disabled:text-transparent peer-disabled:peer-placeholder-shown:text-blue-gray-500 transition-all -top-1.5 peer-placeholder-shown:text-sm text-[11px] peer-focus:text-[11px] before:content[' '] before:block before:box-border before:w-2.5 before:h-1.5 before:mt-[6.5px] before:mr-1 peer-placeholder-shown:before:border-transparent before:rounded-tl-md before:border-t peer-focus:before:border-t-2 before:border-l peer-focus:before:border-l-2 before:pointer-events-none before:transition-all peer-disabled:before:border-transparent after:content[' '] after:block after:flex-grow after:box-border after:w-2.5 after:h-1.5 after:mt-[6.5px] after:ml-1 peer-placeholder-shown:after:border-transparent after:rounded-tr-md after:border-t peer-focus:after:border-t-2 after:border-r peer-focus:after:border-r-2 after:pointer-events-none after:transition-all peer-disabled:after:border-transparent peer-placeholder-shown:leading-[3.75] text-gray-500 peer-focus:text-gray-900 before:border-blue-gray-200 peer-focus:before:!border-gray-900 after:border-blue-gray-200 peer-focus:after:!border-gray-900">Anzahl </label>
The <label> is responsible for this upper part of the border. I changed some peer-focus:.... utilities in the Dev Tools and could change the appearance of the border part. If You carefully do the same in the Dev Tools, you probably can solve your task.
For example add peer-focus:before:!border-**red**-500 to the labelProps className and you get a red upper border left from the label text.
Good luck.


Add Template workflows for tawilkinson/Warhammer-Fantasy-6th-edition

To add GitHub Actions workflows to your repository:

create this issue
add a new comment with the content below
replace [repo name] with your repository name in the format owner/repo, e.g. BSData/test.

/template-workflows-pr tawilkinson/Warhammer-Fantasy-6th-edition

/template-workflows-pr tawilkinson/Warhammer-Fantasy-6th-edition


Exception handling changes exception types unexpectedly

I was surprised that my 'except OSError' did not catch network related errors from my application's 'request' call.  The root error was ConnectionResetError, which is a subclass of OSError, and so I would have expected that catching OSError would catch it.  But it gets turned into a ServerDisconnectedError, which is not a subclass of OSError.  Now, it would be simple enough and logical enough to make DisconnectedError a subclass of OSError.  However, the ServerDisconnectedError in turn gets turned into a ClientResponseError (using raise from) for reasons I don't understand, so even if DisconnectedError were an OSError the OSError would still be masked.  In fact I note that the clauses in that same area of client.py explicity turn OSErrors into ClientOSErrors, which I see has been fixed to be a subclass of OSError, but something that started out as an OSError has gotten converted to something else.  Based on looking at that part of the code, I suspect there may be similar problems with masking HTTPErrors.
(I'm not clear on why there are so many custom exception classes, but I wasn't part of the development so I won't comment further on that :)

This may be another issue, but...I just printed out the error message from the ClientResponseError I got, and it is...absolutely nothing.  Not exactly useful.  I guess I'll have to log the entire exception chain in order to find out what went wrong, which is something I'd rather not do in production, since I don't want to log exceptions when the problem is just a network timeout.
I think it would be much more useful to just let the exceptions bubble up.  At the very least the error message needs to include the error message of the exception being masked.
To put it another way, the value of exception hierarchies is to allow the application to decide to respond to more specific exceptions, or to instead respond to a boad group of exceptions.  Because aiohttp catches the specific exceptions and masks them under a generic exception, the application is prevented from deciding to take action specific exceptions.  My application would have to chase the exception chain in order to find out what really when wrong when I get a ClientResponseError, yet I can't on the other hand decide to handle all network related errors by catching OSError because ClientResponseError is not part of a logical exception hierarchy.

Well, maybe we need cleanup our exception hierarchy but when you use traceback.print_exc() for caught exception you see exception values for all nested exceptions on your chain, isn't it?
IIRC the main reason for not deriving from OSError is: we created custom exception hierarchy. Some exceptions are not OSErrors but parser errors or timeouts. But all of them are derived from ClientError base class.
Anyway, if you have a suggestion about exceptions refactoring please share your opinion.

Yeah, I'm going to try to look in to making a suggestion after I get my code working :)
A timeout should be a subclass of TimeoutError, shouldn't it?  And the base exception in the chain is a server disconnection, and that's an OSError, as I said.  So maybe the bug is that that base error is incorrectly getting turned into something that is not an OSError?  I'll try to investigate further, since I have a unit test that exercises this.  (Note that my code icurrently has to special case ClientResponseError and print out the full traceback so I can see the real error, whereas for those errors that are wrapped in ClientOSError, I can just log the error message...andit is OSError I'm catching).

I've added more information for raising ClientOSError.
See 0a5b068
About TimeoutError: ClientTimeoutError is derived from asyncio.TimeoutError.
The problem is asyncio.TimeoutError is not derived from standard TimeoutError because is has no errno code and asyncio.TimeoutError is logically not OSError.

#1661


feat: Implementation of  memorydb api

This is the initial moto implementation of AWS MemoryDB API.

@bblommers Thanks for reviewing, I merged the latest master. If the tests fail in server mode, do I need to spell out the URL paths in urls.py? Would that fix it?

Hi @archinksagar, yes, manually adding the URL's usually fixes the ServerMode tests. Let me know if it doesn't, or if you run into other issues, and I can have a closer look.

@bblommers  how to I get all the correct URL paths for memorydb?

@archinksagar You can run the MotoServer locally, and Moto will tell you. Within the virtual environment that you develop Moto in, run:
moto_server

And then run the tests against it:
TEST_SERVER_MODE=true pytest -sv tests/test_memorydb

The server logs will then tell you which URL's are being visited, and which requests are failing.

@bblommers Server mode tests were failing because it did not find the correct URL pattern in werkzeug_app. I added few changes to that file. Could you review again and let me know if the changes are right in werkzeug_app.py. Thankyou.


Hiding the social icons in the sidebar

Feature request:
Hiding the social icons in the sidebar in mobile view
Describe the solution you'd like:
I think hiding the social icons in the left side in mobile view will be good since the same icons are available in the footer also.
Describe how this feature will be useful to our readers:
This will help in increasing the width of main content in small screen size. Currently, the side icons are taking some space.
Additional context:

I can work on this issue if this is a needed feature

i would love to work on this issue under GSSOC'24 @niteshseram


DB2/OpenSSL False positives

Hi
I am analysing a tool as part of a bug bounty program and i have ended analysing a horrible DB2 driver  (version <IP_ADDRESS>). The report shows a lot of false positives. Looks like it is ignoring the version and is returning all the known vulnerabilities for DB2, as CVE-2007-3676, CVE-2007-2582 or CVE-2008-4692 that were in the very first versions of DB2 only
Apparently, that version only has two known vulnerabilities
https://www.cvedetails.com/vulnerability-list/vendor_id-14/product_id-3424/version_id-194076/IBM-DB2-<IP_ADDRESS>.html
I have seen something like that with openssl libraries as well, where it is reporting vulnerabiltiies from 1999 in the 1.1.0e version

Can you tell me the GAV coordinates of the DB2 driver you are looking at?  Also, can you tell me what the identified CPE value in the dependency-check report is?

Hi
com.ibm.db2.jcc
db2jcc
4.14.137
CPE (confidence is LOW)

(cpe:/a:ibm:db2:-)
https://web.nvd.nist.gov/view/vuln/search-results?adv_search=true&cves=on&cpe_version=cpe%3A%2Fa%3Aibm%3Adb2
cpe:/a:ibm:db2_connect:10.1
cpe:/a:ibm:db2
cpe:/a:ibm:db2:9.5
cpe:/a:ibm:db2:9.5:fp3a
cpe:/a:ibm:db2:9.5:fp4a
software>cpe:/a:ibm:db2:9.7
cpe:/a:ibm:db2:9.5:fp2a
cpe:/a:ibm:db2:9.5:fp3b
cpe:/a:ibm:db2:9.5:fp6a
cpe:/a:ibm:db2:9.5:fp6
cpe:/a:ibm:db2:9.5:fp5
cpe:/a:ibm:db2:9.7:fp2
cpe:/a:ibm:db2:9.7:fp1
cpe:/a:ibm:db2:9.5:fp2
cpe:/a:ibm:db2:9.5:fp1
cpe:/a:ibm:db2:9.7:fp3
cpe:/a:ibm:db2:9.5:fp4
cpe:/a:ibm:db2:9.5:fp3

Is the jar in a public repo?  The only version that is close that I can find referenced in mvnrepository is com.ibm.db2.jcc:db2jcc:4.14.137.
Any chance you could share the HTML report for just the single JAR file?

The other item I would point out is that dependency-check generally reports a lot of FP for database drivers. This is because there are very few (if any) instances where the DB driver has a CPE that is different than the server. With dependency-check we have taken the stance that these will be reported - they may not be an issue for your application, but may indicate an issue in your environment (i.e. outdated server software). See issue #266 and #623 for more information.

Hi
This is a private repo. This one you have referred is version 10.1
I cant share the html report as in the pipeline, this file is not accessible from jenkins (could be a nice improvement though)
I think the problem is exactly that... Its picking it as cpe:/a:ibm:db2:9.7:fp2 when its fp11). Not sure if anything could be done to improve it
Same thing happens with openssl
Thanks

I opened #811 to track adding the update and edition matching.

Closing this issue and tracking the enhancement via #811.


statement.d: for => foreach

refactor only

What do you think about adding a simple opApply(index, T) to Array, so we can use it with foreach?

Add this to root.Array:
    extern (D) T[] opSlice()
    {
        return data[0..dim];
    }

Then it becomes foreach (i, s; cs.statements)

Auto-merge toggled on


Mongo adapter

Add adapter for mongo with suggestion to install mongodb/mongodb.

This seems to be missing a test.


Theme causing arrows to change BPM?

Whenever I use this theme, the left and right arrows in game causes the scroll speed to increment up and down (faster with right arrow and slower with left arrow). Even remapping the keys of left and right arrow don't make it stop. I'm not sure what causes it, but the game plays fine without the theme.

This is not an OutFox issue, this is the theme having special functions to change the speed and cover lane distance with the Menu buttons. It seems you don't have dedicated buttons set first and declared, so they're acting as both at the same time, which is the default for regular themes, not a lot of themes do this.
To enable dedicated keys, go to in the input options (Input options > Advanced Input Options) and change Dedicated Menu Buttons to "Dedicated Buttons". Ensure you already have set dedicated buttons before this is changed however, otherwise, you won't be able to move through the menus!
The theme performs change of speeds via Code messages done in its metrics.
CodeSpeedUp="MenuRight"
CodeSpeedDown="MenuLeft"


Update docs

Done:

Fixed rst formatting errors
Updated copyright
Added/clarified instructions to install process

Questions:

Are you ok if I re-name some files, to remove spaces?
Are you ok if I move the Table of Contents hierarchy around a bit?

I can do these in a separate PR if you want to see what it would look like. Please let me know. Thanks!

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Questions:

Are you ok if I re-name some files, to remove spaces?
Are you ok if I move the Table of Contents hierarchy around a bit?

I can do these in a separate PR if you want to see what it would look like. Please let me know. Thanks!

Sure thing on both, so long as the linking within the document continues to work after the changes are finished, and the various documents don’t get renamed in the tables of contents to have underscores rather than spaces!
Your call on whether to do as a separate branch/PR. Whatever you think would fit your workflow best.

OK, updated the hierarchy a bit as well, so with that I'm done on this PR. Let me know any edits/changes/whatevers, otherwise I think its ready for your review and/or merge!

Thanks! I just got my Tilt, so I'll start working on that this week. If you're around, I may ask/need some help setting it up, if you have a few minutes

I just installed Fermentrack for the first time and will be starting a brew in a couple hours. I have a Tilt and have been using brewpi (legacy) with tilt integration for the past year.  I'd be interested in testing the dev branch if that would help.

Nevermind, lol.  I've configured to update from the dev branch.  Thanks!

Dev and master should now both be up to date. If you have any issues, let me know - it’s probably a bug. :)

Don't forget to update docs.fermentrack.com!

Apparently I lied, and never merged the PR to update master. Let me do that today.


.prettier. configuration is gone

i saw a PR to add .prettier configuration - why is it gone?

Hi there,
As you can read in the thread of PR 54, after migrating from TSlint to ESlint, it was already covering formatting good enough, so prettier was not adding much value on top of it. I tried to integrate it too but felt kind of useless.
Javi

Thanks @javieraviles. Let me close this


Cache Long as strings in CommandEncoder in order to reduce allocations and improve performance

Profiling our application showed that most of our allocations, over 11% of the total allocation in our application is happening in org/redisson/client/handler/CommandEncoder.writeArgument in the Long.toString method which allocates both string and byte array.
From what I understand from the code the cases where we call writeArgument the arguments are quite small so caching up to 100k should be enough to cover all (?) cases, I'm guessing we can make this configurable but I'm not sure if it's a good idea.
I would've added some benchmarks if I knew where to find such.

It's a good idea

Please correct all checks

Should the cache be filled dynamically? I think at Redisson startup we can get CPU spike in environments with limited resources.

Shouldn't the cache be filled dynamically? I think at Redisson startup we can get CPU spike in environments with limited resources.

How limited? It's only 10000 Strings.
We can use something like Caffeine cache instead of statically in a list.
But one thing I wonder in general @mrniko - I'm trying to understand the distribution of possible values which we encode in writeCharSequence and Long.toString methods:
Line #92: out.writeCharSequence(Long.toString(len), CharsetUtil.US_ASCII);
Where len is the number of arguments in command, so I expect this number to be quite small usually.
Then there are 3 calls to write argument:
Line #96 & #98:
String name = commandMapper.map(msg.getCommand().getName());
writeArgument(out, name.getBytes(CharsetUtil.UTF_8));
if (msg.getCommand().getSubName() != null) {
    writeArgument(out, msg.getCommand().getSubName().getBytes(CharsetUtil.UTF_8));
}

It seems to me that in both cases I expect the value (the command name) to be quite small as well.
And then there is line #103:
for (Object param : msg.getParams()) {
    ByteBuf buf = encode(param);
    writeArgument(out, buf);

Not sure about the size of objects here, maybe you can shed some light.
My point is that if the distribution is really wide then maybe the cache is not really helpful, but if the number are usually small we can even reduce the size of it.
Are there any benchmarks I can run to test it?

I've been doing a little bit more digging into this, since I don't have a "real" data which I can test this, I added some code to generate some statistics based on the tests, and like I suspected almost all (99%, usually more than that) the values are very small so I think that even if we'll set the cache size to 1000 it should be enough to get a hit rate of way over 90% and probably wouldn't create and issues even in environment with limited constraints, what do you think?

I agree with 1000

Changed to 1000

BTW - This a profiling run of our application (2 minutes using async-profiler, ~32k samples)

@tomerarazy
No.
Thanks for the contribution!

by the way, did you evaluate allocation with getBytes for command names at 115 line?

by the way, did you evaluate allocations with getBytes for command names at 115 line?

No, but if command names are repeating it's a good idea to cache them as well.
I'll test it and if it looks good I'll create another PR.

@mrniko I updated our production with the latest version and we're getting about 94% hit rate in the cache.


Update Quantum_Approximate_Optimization_Algorithm.ipynb

Fix spellings and formatting for title.
Issue #, if available:
Description of changes:
Testing done:
Merge Checklist
Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.
General

[ x] I have read the CONTRIBUTING doc
[x ] I used the commit message format described in CONTRIBUTING
[x ] I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

[x ] I have added tests that prove my fix is effective or that my feature works (if appropriate)
[ x] I have checked that my tests are not configured for a specific region or account (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Codecov Report
Patch and project coverage have no change.

Comparison is base (1c0819b) 100.00% compared to head (517e5d5) 100.00%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@            Coverage Diff            @@
##              main      #114   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           26        26
  Lines         1028      1028
  Branches       181       159   -22
=========================================
  Hits          1028      1028

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Dynamically Adding/Removing Fields

I'm looking to use Pode.Web to create a page that would allow me to dynamically add/remove fields (similar to these projects: OpenControl-Editor/example and JSON Editor/example). Based on reading the documentation, I'm pretty sure that I need to use the Pode.Web Events functionality with the "Click" event. But I'm having trouble getting it to work. Since the objects don't have IDs or references, how can I tell Pode.Web to add a field or a collection of fields as a child to a specific field/object and/or remove those fields when an even happens?

Hi @IvanDrag0,
The type of dynamically adding layouts/elements you're talking about doesn't exist yet in Pode.Web, but I am working on it.
You can "mimic" the behaviour by adding the layouts/elements you need upfront, and then use Hide-PodeWebComponent and Show-PodeWebComponent to simply hide/show the objects. You can register a Load page event via Register-PodeWebPageEvent to hide any initial objects, and then use Register-PodeWebEvent and the Click event - or event just a New-PodeWebButton - to then show/hide objects afterwards. You'll need to give all layouts/elements an -Id for this to work.

Hi @IvanDrag0,
Could you elaborate a bit or give an example. I would like to show/hide the password input line by checkbox. thanks a lot in advance.
New-PodeWebTextbox -id "pass_id" -Name 'Password' -Type Password -PrependIcon Lock
New-PodeWebCheckbox -Name 'Eigenes Password' -Inline |
Register-PodeWebEvent -Type Click -ScriptBlock {
Show-PodeWebComponent -Id "pass_id"
}

Hi @eugen257,
Off the top of my head I believe it should be something like the following:
New-PodeWebTextbox -id "pass_id" -Name 'Password' -Type Password -PrependIcon Lock

New-PodeWebCheckbox -Name 'Eigenes Password' -Inline |
    Register-PodeWebEvent -Type Changed -ScriptBlock {
        if ($WebEvent.Data['Eigenes Password'] -ieq 'true') {
            Show-PodeWebComponent -Id "pass_id"
        }
        else {
            Hide-PodeWebComponent -Id "pass_id"
        }
    }

You can check the value of the checkbox when it's changed, and run either Hide or Show accordingly.
Hope that helps! :)

Hi @Badgerati,
it is working. Thanks a lot .
Is there a way to hide (not show) the password string on boot?

Hi @eugen257,
Do you mean hide the whole textbox on page load, or just the contents of the textbox?
If it's the former, then you can pipe Add-PodeWebPage into Register-PodeWebPageEvent:
$page = Add-PodeWebPage -Name PageName -ScriptBlock {}

$page | Register-PodeWebPageEvent -Type Load -ScriptBlock {
    Hide-PodeWebComponent -Id "pass_id"
}

This will hide the textbox on page load.

Hi @Badgerati,
I am getting the following error. What I don't understand.
Register-PodeWebPageEventInternal : The argument for the "Page" parameter cannot be validated. The argument is NULL. Provide a valid value for the argument and rerun the command out.
In C:\Program Files\WindowsPowerShell\Modules\Pode.Web\0.8.1\Public\Events.ps1:126 characters:19
 -Page $Page

Hi @eugen257,
Sorry, I missed a parameter on Add-PodePage, you'll need to pass it -PassThru so it returns the page for you to use.

Hi @Badgerati,
Thank you, everything is working

Hi @Badgerati,
I hurried))) not everything works as it should.
When "Textbox" is hidden, title, Helptext, Icon, PrependText are not hidden.
The "Credential" form doesn't work either, it doesn't have an "Update" option.
What can you advise?

I have used Textbox in a Container and hide the Container completely. Problem solved.
Thanks

The initial question asked in this issue will be resolved by #440.
The above question about hiding elements on creation was resolved by #467 (once released).
The last question about Update functions for Credentials will be resolved by #415.

Closing this issue in favour of the above 3.


Throw Exception on error

Hey, I have done a gradle update (was really outdated) and added a checked exception in case of an error. Currently the caller can not know if the api call was successful, it was only logged.
BTW: I would really appreciate a jar pushed to maven, github packages or something like that :)
And some dependency updates would nice as well, but didn't had time for it yet.

I pushed the dependency updates in this PR

Added some more improvements.


Problem with empty folders

Currently we have a problem in the templates. Template generates a lot of empty folders but git can't add empty folders to repo and it means that our teammates will download project without some folder.
I think we have two ways to fix it:

Put .gitkeep file in each of folders
Leave it as is

We need put .gitkeep in each folder if we really need these directories

@gregoryvit Okay, I'll fix it after fix #24


Hint: Shift enter for new line

This is not obvious you can do that (alt, ctrl, cmd don't work), but super useful.

Maybe support alt as well because that works in macOS text fields?

The reason


Update RequestAuthentication to use TargetRef instead of workloadSelector

Part of #46360
Related to #46560, #46844, #46845
Find where workloadSelector is used (explicitly or implicitly) and add checks for the existence of a targetRef. Defaulting to the current label selector if targetRef doesn't exist for the policy.
Some things to note:

RBAC and policy enforcement in general is a listener concern in Envoy
There may be resources precomputed based on namespace or workload selectors to provide a more effective lookup in Ambient . As seen here for Authorization Policy
targetRef applies to ingress gateways and waypoints
Validation is done to ensure the policies and the targetReferent is valid

Part of https://github.com/istio/istio/issues/46360
Related to https://github.com/istio/istio/pull/46560, https://github.com/istio/istio/issues/46844, https://github.com/istio/istio/issues/46845


ng-bootstrap Modal and md-button compatibility

since md-dialog has no tools for input ( no ways to access dialog instance also )
I had to use ng-bootstrap NgbModal
and this cause problem with mdButton
when this button is not focused before click, it's okay
but when focused, if I click and modal came, then "changed" property of button(with md-button) element cause "hasChangedAfterItWasCheckedException" since changed become false right after modal came
Is there any wise solution?

@alanpurple Please follow the issue template if you're reporting a bug. I can't tell from your description what's actually going on.
WRT dialog, you get the instance of the component from dialogRef.componentInstance

@jelbourn  thx, MdDialog has no trouble with MdButton,  only NgbModal has a trouble with that.
solved


docs: website article

pre-commit.ci autofix

pre-commit.ci autofix

I don't get why this was required — this should be the default. Possibly because it was failing on a commit from itself, and pre-commit doesn't know whether it might be in a loop, so disables that?

Excellent! I'll take a pass later.
Would you prefer this is from "the project" or from you? I notice it doesn't have your name at the moment. I would think it's the sort of piece that could come from an individual (rather than some official project documentation), but no strong view. If we have it from the project, we can adjust some of the language from "I"
(I went through a phase of doing that through PRQL's docs as PRQL became a proper project, lmk if you still see any remnants!)

Would you prefer this is from "the project" or from you?

Actually this can remain as coming from me - a few times I speak in first person and give some personal opinions may not apply to the whole prql team.

pre-commit.ci autofix

@aljazerzen I just found this, have a look! Functional Programming with SQL Query lambdas, functions and pipelines #6

The whole repo seems to be about this: SQL Functional Programming
I haven't done more than glance at it, mostly because I find the SQL really doesn't lend itself to taking a quick look and getting a good idea of what it's about (a great motivation for PRQL), but it seems very similar in aim and spirit (if not syntax) to PRQL and your article in particular.

Oh my god - that's exactly what I'm talking about. A lot of useful material! Thanks!

From that article:

In SQL-99 every query is, at it's base, hard coded to a global namespace of tables and columns. This means that there is no mechanism to cleanly separate the query transformation from the actual data that is being processed.

For me this is a key point, and from my understanding, still an issue with our current name resolution. See for example @aljazerzen 's comment here where it is said:

... we'd be looking up rel.salary in rel. In cases you provided, no relation had a column rel.salary, so it would be also produce an error.

I believe that creating function scope local table names and then restricting column name resolution to that scope is essential to enabling reusable functions in PRQL.
I also think that this links back to the Monadic structure of our transformation pipelines and the actions of the bind operator, which I unfortunately cannot elucidate properly yet and remains a hunch for me at this stage.
The rest of the Benefits section of that article are basically the same motivations that are shared by PRQL.

Hi team, nice to know that someone has some value from my reflections on functional programming with SQL!  I don't love SQL syntax either - so for me, PRQL is an exciting development.
It is a bit hard to follow the thread above to understand where your function definition/ reference discussions landed. Could someone sketch out how a reference to summarise and the definition of it would land in PRQL?
Usage
SELECT *
from summarise((SELECT country_id summary_id, state_id id FROM states),
               (SELECT state_id id, population amount FROM population_by_states)
              );

Definition
CREATE VIEW summarise(summary_id, amount) AS
    GIVEN (mapping(summary_id, id), rawdata(id, amount))
    SELECT mapping.summary_id, sum(rawdatea.amount) amount
    FROM rawdata rawdata
             JOIN mapping mapping ON rawdata.id = mapping.id
    GROUP BY mapping.summary_id;
)

I'm keen to catch up with the conversation.
David

Hi @DavidPratten ,
Welcome and great to have you part of the conversation!
I did promise to raise a new issue for you, apologies for not getting to that yet. We're trying to tie up a couple of loose ends in order to get the 0.4 release out this weekend. There will be actually a number of changes that impact the "table" (aka CTE) and transform definitions which has bearing on what you are interested in.
Our functions actually work amazingly great for the most part and many things that appear to be something different on the surface, e.g. relations and transforms, are actually functions underneath, following the functional programming paradigm.
What I see as an outstanding issue for us, is exactly as you described so well in the piece that I quoted from your article:

In SQL-99 every query is, at it's base, hard coded to a global namespace of tables and columns. This means that there is no mechanism to cleanly separate the query transformation from the actual data that is being processed.

My understanding is the compiler currently needs to be able to resolve names in the global namespace of tables and columns at function definition time which in some ways leaves us in the same position as SQL. @aljazerzen is really our compiler expert and mage though so he would have to confirm that.
I advise though that we put a temporary hold on this discussion until 0.4 is released so that the public documentation matches the current state of the compiler that the core team are referencing in the discussions. 0.4 should come out soon (this weekend probably) so we can pick things up after that.
I hope you find that acceptable.
P.S. You can read @aljazerzen 's article that this thread is about here: A functional approach to relational queries

@DavidPratten , since we found each other through Torsten Grust's twitter post about "Making Recursive CTEs more lovable", you may also want to take a look at the current discussion around the proposed Recursive CTE syntax for PRQL and weigh in on that.
That discussion can be found here: https://github.com/PRQL/prql/issues/407#issuecomment-1380921570

It is a bit hard to follow the thread above to understand where your function definition/ reference discussions landed. Could someone sketch out how a reference to summarise and the definition of it would land in PRQL?

Yeah, this thread does not give good context. Your function summarize would look like this:
func summarise mapping rawdata -> (
    from rawdata = _param.rawdata
    join mapping [==id]
    group mapping.summary_id (
        aggregate [amount = sum rawdata.amount]
    )
)

summarise (
    from states | select [country_id, summary_id, state_id]
) (
    from population_by_states | select [id = state_id, amount = population]
)

Don't mind the _param param thing, name resolving is a work in progress.

Thanks @aljazerzen nice.   A couple of follow-ups to start with.
Polish notation
summarise (
    from states | select [country_id, summary_id, state_id]
) (
    from population_by_states | select [id = state_id, amount = population]
)

This looks a bit like Polish Notation where parameters are not grouped by ( ) -  is this a pattern used in other parts of PRQL, or just here?
Declaring column names and types
In the PRQL example the column names are being drawn from the underlying table in global name space.
To disconnect the function body from the global namespace and have a function that is "write-once use anywhere" the function declaration needs to include column names and types just like any other relation (table) declaration.
In my article, I ignored types, but if they were included the GIVEN clause might look this:
GIVEN (mapping(summary_id varchar, id bigint), rawdata(id bigint, amount float))

What is the PRQL equivalent of this?
On invocation are we matching names or ordinal position?
On invocation of the function then there is a choice, match columns by ordinal position or by name.  e.g. SQL UNION uses ordinal position match. Alternatively, the matching could be on column name.  This would force explicit renaming e.g. id = state_id of input relations to use the names in the function's column declaration.
David

Polish notation

This is our function call notation that starts with the function name and continues with arguments separated by whitespace. It's explained here and here.

Declaring column names and types

At the moment we don't have functioning type annotations, but they are on the map.
Functions are meant to be "write once, use anywhere" and this is achieved by expecting the input relation to have a certain structure. Right now, this contract expressed implicitly by just using references to the columns, but the plan is to 1) allow adding type annotation to the functions 2) automatically infer the columns (maybe types) from the function body.

On invocation are we matching names or ordinal position?

Relations in PRQL have columns ordered and optionally named. This is reflected in their type signature. For now, we only had to do matching for append (which is UNION ALL) and we haven't explicitly decided on how to do matching exactly. As a first iteration, I implemented matching based on position, but we will have to revisit that some time soon.


Provide some utilities for other rules

Some of the things in https://github.com/sindresorhus/eslint-plugin-ava/pull/10 could be useful for other rules and should be extracted into util.js.

@sindresorhus Okay I'll extract common structure between existing test-ended and prefer-power-assert rules.


Documentation for Installing from Source

Though the documentation says just run go build, the following packages are still telling me I need to use go get prior to the build.  Am I doing something wrong here or is this in fact the necessary steps?
[/repositories/git/opensource/confd]$ go build
config.go:14:2: cannot find package "github.com/BurntSushi/toml" in any of:
	/usr/local/go/src/github.com/BurntSushi/toml (from $GOROOT)
	/repositories/git/opensource/confd/src/github.com/BurntSushi/toml (from $GOPATH)
confd.go:10:2: cannot find package "github.com/kelseyhightower/confd/backends" in any of:
	/usr/local/go/src/github.com/kelseyhightower/confd/backends (from $GOROOT)
	/repositories/git/opensource/confd/src/github.com/kelseyhightower/confd/backends (from $GOPATH)
confd.go:11:2: cannot find package "github.com/kelseyhightower/confd/log" in any of:
	/usr/local/go/src/github.com/kelseyhightower/confd/log (from $GOROOT)
	/repositories/git/opensource/confd/src/github.com/kelseyhightower/confd/log (from $GOPATH)
confd.go:12:2: cannot find package "github.com/kelseyhightower/confd/resource/template" in any of:
	/usr/local/go/src/github.com/kelseyhightower/confd/resource/template (from $GOROOT)
	/repositories/git/opensource/confd/src/github.com/kelseyhightower/confd/resource/template (from $GOPATH)

[/repositories/git/opensource/confd]$ ./build
Building confd...
config.go:14:2: cannot find package "github.com/BurntSushi/toml" in any of:
	/usr/local/go/src/github.com/BurntSushi/toml (from $GOROOT)
	/repositories/git/opensource/confd/src/github.com/BurntSushi/toml (from $GOPATH)
confd.go:10:2: cannot find package "github.com/kelseyhightower/confd/backends" in any of:
	/usr/local/go/src/github.com/kelseyhightower/confd/backends (from $GOROOT)
	/repositories/git/opensource/confd/src/github.com/kelseyhightower/confd/backends (from $GOPATH)
confd.go:11:2: cannot find package "github.com/kelseyhightower/confd/log" in any of:
	/usr/local/go/src/github.com/kelseyhightower/confd/log (from $GOROOT)
	/repositories/git/opensource/confd/src/github.com/kelseyhightower/confd/log (from $GOPATH)
confd.go:12:2: cannot find package "github.com/kelseyhightower/confd/resource/template" in any of:
	/usr/local/go/src/github.com/kelseyhightower/confd/resource/template (from $GOROOT)
	/repositories/git/opensource/confd/src/github.com/kelseyhightower/confd/resource/template (from $GOPATH)

If this is in fact the correct approach, could we add it to the documentation, the ./build script and/or perhaps the new Vendor subdirectory?  I know its pretty obvious on how to get the packages updated, but not having it in the build from source documentation lead me to think I was doing something wrong at first....might help others avoid the same suspicion (unless I am doing something wrong!).

It doesn't look like you have GO15VENDOREXPERIMENT=1 set. Which version of Go are you using?

after that ./build is the way to go.

Doh, I forgot to include that...
[/repositories/git/opensource/confd]$ go version
go version go1.6 darwin/amd64

Try this:
GO15VENDOREXPERIMENT=1 ./build

That should get you started :)

[/repositories/git/opensource/confd]$ export GOPATH=`pwd`
[/repositories/git/opensource/confd]$ GO15VENDOREXPERIMENT=1 ./build
Building confd...
config.go:14:2: cannot find package "github.com/BurntSushi/toml" in any of:
	/usr/local/go/src/github.com/BurntSushi/toml (from $GOROOT)
	/repositories/git/opensource/confd/src/github.com/BurntSushi/toml (from $GOPATH)
confd.go:10:2: cannot find package "github.com/kelseyhightower/confd/backends" in any of:
	/usr/local/go/src/github.com/kelseyhightower/confd/backends (from $GOROOT)
	/repositories/git/opensource/confd/src/github.com/kelseyhightower/confd/backends (from $GOPATH)
confd.go:11:2: cannot find package "github.com/kelseyhightower/confd/log" in any of:
	/usr/local/go/src/github.com/kelseyhightower/confd/log (from $GOROOT)
	/repositories/git/opensource/confd/src/github.com/kelseyhightower/confd/log (from $GOPATH)
confd.go:12:2: cannot find package "github.com/kelseyhightower/confd/resource/template" in any of:
	/usr/local/go/src/github.com/kelseyhightower/confd/resource/template (from $GOROOT)
	/repositories/git/opensource/confd/src/github.com/kelseyhightower/confd/resource/template (from $GOPATH)

huh. I wasn't able to reproduce this.
$ git status | head -n 1
On branch master
$ docker run -itv $PWD:/go/src/github.com/kelseyhightower/confd -w /go/src/github.com/kelseyhightower/confd golang:1.6 bash
Unable to find image 'golang:1.6' locally
1.6: Pulling from library/golang

Digest: sha256:bf9c6e96109d5b6db3789f5a2e93305481fb50485e34fac97eb28867aeaa8605
Status: Downloaded newer image for golang:1.6
root@b1ae684e5678:/go/src/github.com/kelseyhightower/confd# ./build
Building confd...
root@b1ae684e5678:/go/src/github.com/kelseyhightower/confd# ./bin/confd --version
confd 0.12.0-dev

[~/repositories]$ docker run -itv /Users/jbkc85/repositories/confd:/go/src/github.com/kelseyhightower/confd -w /go/src/github.com/kelseyhightower/confd golang:1.6 bash
root@ac2f813505ac:/go/src/github.com/kelseyhightower/confd# ls
CHANGELOG		 Godeps       README.md  clean	    config_test.go  install	 node_var.go  vendor
CONTRIBUTING.md		 LICENSE      backends	 confd.go   contrib	    integration  resource     version.go
Dockerfile.build.alpine  MAINTAINERS  build	 config.go  docs	    log		 test
root@ac2f813505ac:/go/src/github.com/kelseyhightower/confd# export GOPATH=`pwd`
root@ac2f813505ac:/go/src/github.com/kelseyhightower/confd# ./build
Building confd...
config.go:14:2: cannot find package "github.com/BurntSushi/toml" in any of:
	/usr/local/go/src/github.com/BurntSushi/toml (from $GOROOT)
	/go/src/github.com/kelseyhightower/confd/src/github.com/BurntSushi/toml (from $GOPATH)
confd.go:10:2: cannot find package "github.com/kelseyhightower/confd/backends" in any of:
	/usr/local/go/src/github.com/kelseyhightower/confd/backends (from $GOROOT)
	/go/src/github.com/kelseyhightower/confd/src/github.com/kelseyhightower/confd/backends (from $GOPATH)
confd.go:11:2: cannot find package "github.com/kelseyhightower/confd/log" in any of:
	/usr/local/go/src/github.com/kelseyhightower/confd/log (from $GOROOT)
	/go/src/github.com/kelseyhightower/confd/src/github.com/kelseyhightower/confd/log (from $GOPATH)
confd.go:12:2: cannot find package "github.com/kelseyhightower/confd/resource/template" in any of:
	/usr/local/go/src/github.com/kelseyhightower/confd/resource/template (from $GOROOT)
	/go/src/github.com/kelseyhightower/confd/src/github.com/kelseyhightower/confd/resource/template (from $GOPATH

I made sure to do it through Docker to have the same environment :-)

oh, wow.  This was due to the fact i was running ```export GOPATH=`pwd````.  Why on earth would that happen?

Still doesn't work on basic MacOSX though:
[~/repositories/confd]$ git status | head -n1
# On branch master
[~/repositories/confd]$ go version
go version go1.6 darwin/amd64
[~/repositories/confd]$ echo $GOPATH

[~/repositories/confd]$ ./build
Building confd...
config.go:14:2: cannot find package "github.com/BurntSushi/toml" in any of:
	/usr/local/go/src/github.com/BurntSushi/toml (from $GOROOT)
	($GOPATH not set)
confd.go:10:2: cannot find package "github.com/kelseyhightower/confd/backends" in any of:
	/usr/local/go/src/github.com/kelseyhightower/confd/backends (from $GOROOT)
	($GOPATH not set)
confd.go:11:2: cannot find package "github.com/kelseyhightower/confd/log" in any of:
	/usr/local/go/src/github.com/kelseyhightower/confd/log (from $GOROOT)
	($GOPATH not set)
confd.go:12:2: cannot find package "github.com/kelseyhightower/confd/resource/template" in any of:
	/usr/local/go/src/github.com/kelseyhightower/confd/resource/template (from $GOROOT)
	($GOPATH not set)

And here through Docker, it works fine...
[~/repositories/confd]$ docker run -itv /Users/jbkc85/repositories/confd:/go/src/github.com/kelseyhightower/confd -w /go/src/github.com/kelseyhightower/confd golang:1.6 ./build
Building confd...
[~/repositories/confd]$ ls ./bin/
confd

@jbkc85 Sorry for the friction and it seems we need to clearly mark that confd require Go 1.6 to build from source. We are using the new support for the vendor directory and now that Go 1.6 has been released I want to target it.

@kelseyhightower no worries, but you actually already have it clearly marked that confd requires Go 1.6.  I have Go 1.6 installed on my Mac (freshly installed) and still got the issues from above...however when I use the Docker container that @bacongobbler used, it works perfectly fine.  It was very confusing to me (not to mention that also, when you use GOPATH=pwd in the Docker container, it messes it up too).  This might be a Go issue versus something of just Confd...I just wanted to point it out as I only have Confd to test against currently :-)

You need a directory structure like this:
$GOPATH
|	src/
|	|	github.com/kelseyhightower/confd/
|	|	|	vendor/
|	|	|	|	...

It doesn't work if you have the confd folder elsewhere in your filesystem.
Thats also the reaseon why
root@ac2f813505ac:/go/src/github.com/kelseyhightower/confd# export GOPATH=`pwd`

doesn't work.

@HeavyHorst that would do it.  Thank you for the explanation!
I guess i was just thrown off by:
git clone https://github.com/kelseyhightower/confd.git
cd confd
go build

Which causes errors as seen above...Versus it being something like:
[~/go]$ go get github.com/kelseyhightower/confd
[~/go]$ go build github.com/kelseyhightower/confd

Once again, I am not sure if this is due to a setup with Go, but given I had the issues I wanted to raise them with you guys just in case.  Thank you for your patience and help walking me through it!

thanks everyone for helping on this one :-)

@kelseyhightower  Why is this issue closed? This should totally replace the misleading instructions on the README.md (which still end up in failure if you follow them).
If it weren't for this issue, I'd never have been able to install confd from source.

@nfantone please feel free to open a PR with any proposed changes to the README.

@bacongobbler Was afraid of this answer. As much as I'd like to, I am sorry, but I won't be doing that. For a variety of reasons:

I have absolutely no idea of go.  This is my first time running the tool. Ever.
I just found this thread and copy-paste my way out of installing.
I can't really say if this actually are the real steps I should follow. As I said, I just run them as is.
No idea why they work or why they need to be like that.
No idea why instructions on the README.md don't work in the first place, either.

Please, correct me if I'm wrong, but you, on the other hand, being a collaborator, could just corroborate and write the proper instructions with much less effort. And they would be much better written, I am sure.
I'm sure you understand.

I actually closed this ticket because I figured that it was just a 'me' thing.  I went ahead and adjusted the README where I thought it needed to be adjusted, at least to what I had experienced while cloning/installing confd for myself.  Please feel free to adjust or ask questions where needed, otherwise im happy to contribute.
@nfantone - though I am sure you already know this, based on your last comment I would just like to share something.  I am also quite new when it comes to Go, and haven't really contributed to any Open Source project before.  That being said, this should never make you think that you can't contribute.  There are as you mentioned collaborators assigned to this project, who of course are writing the code and initial documentation, but people like you and I can easily come in and contribute to things like the README - in which they can review, adjust, and push to the master repo when they feel it is ready.  Never feel bad about a PR being rejected, edited, and feel fantastic about a 4 line commit that gets accepted.  Its all about the community and collaboration, and the more we have the better project we get :-).

I just pushed https://github.com/kelseyhightower/confd/commit/6386163046ddad465498d6e002bd8e5ab3748365 which should help clarify things. Thanks @jbkc85 for your help!

That being said, this should never make you think that you can't contribute.

@jbkc85 Don't misunderstand me: I'm with you on the sharing bit. I've contributed my fair share to open source projects in the past. I manage some of them myself, that are even published to npm. This is not bragging: anyone with an internet connection can do that. The thing is, I believe you mistook "open source" with "anyone can change things". Even you and I, by coming here and posting our comments in this issue are collaborating. Heck, it even ended in a PR and a change to the official docs!

Its all about the community and collaboration, and the more we have the better project we get :-).

This is part of the confusion. More does not mean better. So you found out some go commands that worked for you. And you did share it. That's awesome. Do you know what they imply? Do you know if those were the right commands to type? Which versions were you using? What other system packages you had installed? How do you know if they just worked on your environment? If you do know all of this, then great! Feel free to PR the confd guys. But, if not (which is my case, at least), I'd limit to opening and issue and/or write a comment. There are more qualified people out there who can write that piece of documentation far better than me.


Assertion 'ConfirmEqual' prints null as empty string in default message

Describe the bug
Assertion 'ConfirmEqual' prints null as empty string in default message.
Steps to reproduce
Run:
_ = "".ConfirmEqual(null);

See error:
Expected '' but got ''.

Expected behavior
The displayed error message should print null as null and not an empty string.
Additional information
N/A
Screenshots
No response
Additional context
No response

Update:
After merging #169
Running the same code, returns error:
Assertion ConfirmEqual failed: Expected , but got ""


gzip: filename.tgs.gz: No such file or directory

Apologies in advance in case this is a silly doubt
"F:\Telegram Emojis" is my local directory with the .tgs files in it
I installed docker for desktop
I'm on Windows 11
The console output is as follows:
C:\Users\Monster>docker run --rm -v "F:\Telegram Emojis":/source edasriyan/lottie-to-gif
Unable to find image 'edasriyan/lottie-to-gif:latest' locally
latest: Pulling from edasriyan/lottie-to-gif
6299ae3fea5d: Pull complete
264cbb337611: Pull complete
664f776008c6: Pull complete
38765ae832f3: Pull complete
0a81288c9a12: Pull complete
Digest: sha256:329fe35c5a0a57134a6bcf38b344cb1efd165b5f5e3150fed0c13e40aac2af24
Status: Downloaded newer image for edasriyan/lottie-to-gif:latest
Converting /source/No mouth.tgs...
gzip: mouth.tgs.gz: No such file or directory
terminate called after throwing an instance of 'std::runtime_error'
what():  can not load lottie animation
/usr/bin/lottie_common.sh: line 106:    15 Aborted                 $SCRIPT_DIR/lottie_to_png --width $WIDTH --height $HEIGHT --fps $FPS --threads $THREADS --output $TMP_PATH $LOTTIE_PATH
error: The following required arguments were not provided:
...
USAGE:
gifski --output <a.gif> --quiet --fps  --height  --width  --quality <1-100> ...
For more information try --help
C:\Users\Monster>
To Reproduce
Steps to reproduce the behavior:

Download this file to ...
Run this command ...
Open that file in ...
See error

Expected result
A clear and concise description of what you expected to happen.
Actual result
A clear and concise description of what happened instead.
Console copy-paste
Please copy-paste command run and program output
docker run --rm -v /tmp/test-files:/source edasriyan/lottie-to-webp
Converting /source/11.json...
Done.

Converting /source/08.tgs...
Done.

Segmentation fault...

Desktop (please complete the following information):

OS: [e.g. iOS]
Arch: [e.g. arm, x86_64]

Additional context
Add any other context about the problem here.

Issue resolved, It doesn't work if any of the file names have a [space] in them (even though I share the PATH with quotes). Please check if this exception can be handled.
I'm super new to docker (unfamiliar with the syntax) so I was extra skeptical when you mentioned 'One-line command' in your readme,
I was extra skeptical when the One-line command didn't mention a dockerfile (The Basic Docker tutorials speak about a local dockerfile). You called it from the cloud XD (Haven't reached that part yet)
You sir are terrifyingly brilliant!!
Thank you for your work @ed-asriyan


lazy=true seems unnecessarily lazy?

Currently, lazy = true defaults to always creating a FileArray, which means that any read requires re-opening the dataset, and calling GDAL again if using e.g a VRT.
Consider this benchmark on a VRT of Copernicus DEM.  dem is the raster, opened with lazy = true:
julia> @be dem[$(1:10), $(1:10)] seconds=2
Benchmark: 3 samples with 1 evaluation
        927.189 ms (107 allocs: 6.812 KiB)
        939.257 ms (107 allocs: 6.812 KiB)
        941.771 ms (107 allocs: 6.812 KiB)

That's pretty bad.  Now, see what happens when I simply run the benchmark inside open(dem):
julia> open(dem) do d
       @be d[$(1:10), $(1:10)] seconds=2
       end
Benchmark: 877 samples with 1 evaluation
 min    2.165 ms (76 allocs: 4.125 KiB)
 median 2.177 ms (76 allocs: 4.125 KiB)
 mean   2.185 ms (76 allocs: 4.125 KiB)
 max    3.119 ms (76 allocs: 4.125 KiB)

Users will initially try lazy = true, see that the performance is horrible, and move on with their lives - when we could have 2 OOM better performance on reads!
Since we have finalizers in Julia, it seems like we could make lazy = true at least open the GDAL dataset, and lazy = false would eagerly read into memory as it currently does?  Or is there something I am missing here?

This is a difference of use cases. If you have only a few files it is no problem to keep the file open in the background, but with a few hundred you get memory issues and at some point GDAL cant handle it anymore.

Aha, didn't realize that case was also a thing.  Too used to working with Kerchunk I guess :D
We probably need a lot more documentation around lazy, open and read then.  open also doesn't work by itself (it needs an associated function or open(ras) do A) so allowing fast_ras = open(ras) would be great as well.

Yeah, Rasters has a "no open files" policy.
I had a lot of issues hitting is open file limits early in. Like 400 and and that's it on Linux.
So they are really lazy.
We should add a Raster constructor with a function argument so it combines with open smoothly. But not sure I want to allow actually open files. With GDAL that means hanging C objects.

Isn't that what IDataset is for in ArchGDAL, for instance?  It has an attached finalizer.  Same for any ArchGDAL/GeoDataFrames geometry...

relying on people using close

sorry to belabour this point - but isn't the point of a finalizer that you don't need to use close?  The first GC sweep after the object goes out of scope will do it for you.  For interactive REPL use it would leave a hanging pointer, but then something was always going to do that.
I get your point about file limits though.  Maybe this is something to do only if lazy = false, or maybe we can have some other keyword that controls this...

You don't need to use close if you don't care about open files I guess.
We could add open that just gives you the GDAL array wrapped in a Raster...
But what's wrong with using an open closure?
I never find myself limited by this. I don't really get the use case.

Anything interactive is a problem?  You have to do open(ras) do R; ...; end as opposed to directly working with an object.  It's more that this puts less cognitive load on the user.
I personally rarely, if ever, am in a position to use ArchGDAL with closures - if you're writing and debugging analysis code, it's very hard to inspect intermediate results within a closure!

Do you have an example of this kind of workflow? I guess I don't understand why you need to open the Raster for analysis... What works on the open file that won't work on the raw raster?


VALUES clause support

Is your feature request related to a problem? Please describe.
Hi there! It looks like Jet doesn't support VALUES clause. My personal use case is batch update statements, for instance
UPDATE public.some_table
SET name = (d.v)::text
FROM (
         VALUES (1, 'name1'), (2, 'name2')
     ) AS d(id, v)
WHERE some_table.id = (d.id)::bigint;

or
WITH d(id, v) AS (VALUES (1, 'name1'), (2, 'name2'))
UPDATE public.some_table
SET name = (d.v)::text
FROM d where some_table.id = (d.id)::bigint;

Describe the solution you'd like
Public API support for VALUES clause.

Hi @v-byte-cpu. VALUES is not supported currently. You can use this workaround for now.

VALUES Statements Support add in release v.2.12.0 (wiki)


Wanted to add client certificate and key in OkHTTP

I wanted to add client certificate and key in OkHTTP, when we are hitting with any request that client cert need to associated with that request.

Please use stackoverflow for discussions like this. The issue tracker is for bugs and feature requests.