Error: You do not have an SDK

Getting this error upon make
==> Error: You do not have an SDK in /Applications/untitled folder/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs.
Here are the contents of /Applications/untitled folder/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs.
iPhoneSimulator.sdk	iPhoneSimulator11.2.sdk

There are two sdk's there. However, my simulator is on 11.0, not the 11.2 sdk above.
Where do I get these sdks, and why are the ones already there not sufficient?
Using Xcode 9.2 (9C40b) with a iPhone 8 Plus 11.2 Simulator

What if you move Xcode.app out of a space-included directory (untitled folder)? It may handle spaces in a directory path incorrectly.

Yes that worked :D
Heres the full output (lots of warnings, but it works)
I have a bash script executed when I build it in Sublime Text, which checks if the simulator is running (if not, open it), auto copies the dylibs to the right folder, and repsprings, so some of the output is related to that
Simulator already running
> Making all for tweak simjectExampleTweak…
==> Preprocessing simjectExampleTweak.xm…
==> Compiling simjectExampleTweak.xm (x86_64)…
==> Linking tweak simjectExampleTweak (x86_64)…
clang: warning: libstdc++ is deprecated; move to libc++ with a minimum deployment target of iOS 7 [-Wdeprecated]
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(chkstk.S.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(chkstk2.S.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(cpu_model.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(divmodsi4.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(divsi3.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(divtc3.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(extendhfsf2.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(int_util.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(mulodi4.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(mulosi4.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(muloti4.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(os_version_check.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(truncdfhf2.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(truncsfhf2.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(udivmodsi4.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(udivsi3.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(atomic.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(atomic_flag_clear.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(atomic_flag_clear_explicit.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(atomic_flag_test_and_set.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(atomic_flag_test_and_set_explicit.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(atomic_signal_fence.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(atomic_thread_fence.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(emutls.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
==> Stripping simjectExampleTweak (x86_64)…
==> Preprocessing simjectExampleTweak.xm…
==> Compiling simjectExampleTweak.xm (i386)…
==> Linking tweak simjectExampleTweak (i386)…
clang: warning: libstdc++ is deprecated; move to libc++ with a minimum deployment target of iOS 7 [-Wdeprecated]
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(chkstk.S.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(chkstk2.S.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(cpu_model.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(divmodsi4.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(divsi3.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(divtc3.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(extendhfsf2.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(int_util.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(mulodi4.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(mulosi4.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(os_version_check.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(truncdfhf2.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(truncsfhf2.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(udivmodsi4.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(udivsi3.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(atomic.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(atomic_flag_clear.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(atomic_flag_clear_explicit.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(atomic_flag_test_and_set.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(atomic_flag_test_and_set_explicit.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(atomic_signal_fence.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(atomic_thread_fence.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
ld: warning: object file (/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/lib/darwin/libclang_rt.ios.a(emutls.c.o)) was built for newer iOS version (6.0) than being linked (5.0)
==> Stripping simjectExampleTweak (i386)…
==> Merging tweak simjectExampleTweak…
==> Signing simjectExampleTweak…
Service configured for next launch.
respring_simulator (C) 2016 Karen Tsai (angelXwind)
Injecting appropriate dynamic libraries from /opt/simject...
Respringing...


Add new git repos, isolate awesome-lint test in its own job and add test to check octocat links

Add new git repos (1 round of research)
Isolate awesome-lint job
Create pt-en-comparison with the existing test that count the number of items + a new one that compares the octocat links
Minor tweaks: fix typo + add badge to English version to match the line numbers with the Portuguese version

Close #11

Sorry about the delay, this PR looks fine to me!

It's alright! Actually, it was my bad not waiting for your feedback. I'll be very busy in the next few days, so I decided to merge it quickly to not let it floating during this week, and I didn't want to hurry you. Feel free to point any problem or suggestion!


Fix link to pure JAX example

00_linear_regression_pure_jax.py  is not found in the basic subfolder of examples but in the jax subfolder.

Thanks!


Fix typos in "Galaxy Installation with Ansible" tutorial

Fix a few typos in the "Galaxy Installation with Ansible" tutorial.

Should this be merged to gat or main? Will changes from gat be merged to main at a later point?


Implement abstract base class for ASDF extensions

Right now AsdfExtension implements __subclasshook__ so that it can recognize foreign extensions even if the foreign extension doesn't inherit directly from AsdfExtension. In most cases, it is not appropriate for foreign extensions to inherit from AsdfExtension.
However, using an abstract base class that can be used as a parent for both AsdfExtension and other extensions would make this much cleaner. It would allow invalid extension definitions to be checked much earlier than they are right now.
cc @Cadair

I will take a swing at this.

Closing this issue. I opened it in the midst of a momentary lapse of reason. Ideally, I think that AsdfExtension should not implement __subclasshook__ so that foreign extensions are required to inherit it explicitly, but it's probably not that important.


Change isinstance/issubclass to accept PyClassRef

To do this I also changed the return type of PyValue::class() and PyContext::xxx_type() methods to PyClassRef.

Codecov Report

Merging #723 into master will decrease coverage by 1.45%.
The diff coverage is 72.22%.

@@            Coverage Diff             @@
##           master     #723      +/-   ##
==========================================
- Coverage   47.02%   45.56%   -1.46%
==========================================
  Files          81       81
  Lines       16054    16309     +255
  Branches     3894     3978      +84
==========================================
- Hits         7549     7432     -117
- Misses       6702     7072     +370
- Partials     1803     1805       +2

Impacted Files
Coverage Δ

vm/src/function.rs
32.84% <ø> (-1.49%)
:arrow_down:

wasm/lib/src/convert.rs
0% <ø> (ø)
:arrow_up:

vm/src/obj/objmemory.rs
21.42% <0%> (-10.16%)
:arrow_down:

vm/src/stdlib/types.rs
0% <0%> (ø)
:arrow_up:

vm/src/obj/objcode.rs
30.76% <0%> (-22.97%)
:arrow_down:

vm/src/exceptions.rs
39.24% <0%> (+0.73%)
:arrow_up:

vm/src/obj/objobject.rs
48.42% <0%> (+0.5%)
:arrow_up:

vm/src/stdlib/re.rs
29.33% <0%> (+2.22%)
:arrow_up:

vm/src/obj/objfunction.rs
43.54% <0%> (-23.9%)
:arrow_down:

wasm/lib/src/browser_module.rs
0% <0%> (ø)
:arrow_up:

... and 64 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b7fa08e...983cb9e. Read the comment docs.


Fix parsing errors from S3

When attempting to retrieve a non-existing record from a key-value store, we get an error directly from S3, which is XML-formatted, instead of from our API servers, which return it in JSON. This fixes the parsing of those errors, enabling correct behaviour (returning None) for those missing records.
I've also set the default Accept header to application/json, */*, in order to prefer JSON responses and maintain compatibility with the JS client.

@Equidem This just fixes the parsing of the errors which we get from S3 into the ApifyApiError.message field, the actual check for whether it's a "not found" error is in https://github.com/apify/apify-client-python/blob/master/src/apify_client/_utils.py#L120, and that checks for both options, if the error was JSON or if it was not. No information is actually lost here.

@Equidem This just fixes the parsing of the errors which we get from S3 into the ApifyApiError.message field, the actual check for whether it's a "not found" error is in https://github.com/apify/apify-client-python/blob/master/src/apify_client/_utils.py#L120, and that checks for both options, if the error was JSON or if it was not. No information is actually lost here.


Add StableHLO complex sqrt to stablehlo-complex-math-expander pass

As in the title.
The existing implementation of sqrt on complex inputs uses polar form of complex sqrt which is inaccurate/incorrect on about 28 % of uniformly distributed samples over all complex plane.  The corresponding JAX complex sqrt accuracy statistics is as follows:
test_unary[sqrt-jax-cuda-complex64-default] maximal ULP difference: 5
ULP difference == 0: 1060571
ULP difference == 1: 1008268
ULP difference == 2: 31136
ULP difference == 3: 686
ULP difference == 4: 129
ULP difference == 5: 10

test_unary[sqrt-jax-cpu-complex64-default] maximal ULP difference: 2
ULP difference == 0: 1348868
ULP difference == 1: 751504
ULP difference == 2: 428

It is interesting to note that although the same algorithm is used for both CUDA and CPU platforms, then the expected maximal ULP difference is 4 (obtained from applying the algorithm to numpy arrays). Hence

the accuracy of complex sqrt on CPU is better than expected, likely due to fusing
the accuracy of complex sqrt on CUDA is worse than expected because CUDA sqrt produces slightly different results from std sqrt on float inputs.

@GleasonK , please review.


change TargetInfo to ReadOnlyTargetRules

This change removes the warning when compiling the plugin for engine version 4.16 and above.

Thanks for this @metahusk, by the way.  Hopefully @pixley / @DescendentStudios accepts it at some point.


SimpleJson is not compiled with SIMPLE_JSON_READONLY_COLLECTIONS

All my existing types for REST serialization use IReadOnlyCollection<T> for nested collections, and refactoring that to any other type (no matter if ICollection<T>, T[] or List<T>) introduces a huge amount of strong PITA problems.
So please define SIMPLE_JSON_READONLY_COLLECTIONS either directly in code or in whatever project which includes and compiles that file.

The Blazor team is focusing on the core Blazor functionality (see #966) and the JSON serialization is not part of that, so this isn't going to be addressed at this point. The use of SimpleJson is really a quick-and-dirty solution for getting basic JSON serialization into Blazor, and it has some issues.
If you need better control over JSON serialization, add the Newtonsoft.Json package to your Blazor client, and use that. You will need to turn off the linker to get it to work however, see #370
Add this to the  Blazor .csproj
<BlazorLinkOnBuild>False</BlazorLinkOnBuild>

I was able to get Newtonsoft working on my 0.5.1 test project with that fix.

Writing that answer and searching for #966 and #370 definitely took more time than just enabling a conditional feature which SimpleJson brings along out of the box. I don't think compatibility with .net < 4.5 is a requirement for Blazor.


Members events tab design – Mobile

Screens for "Borrowing Activities" and "Membership Activities" both in scrolled and uncscrolled versions have been added to Zeplin. (for mobile and tablet)

The sort options for columns for mobile are now eliminated from the interface and will be revisited next week to look for potentially larger icons/ways of sort navigations if possible.

The "Status" column under "Borrowing Activities" has been moved to the left of the "Start-date" and "End-date" columns. A conversation during the usability testing has been weighed into this decision as it was also discussed during one of the project meetings.


Fix Avro CVE

Problem
https://confluentinc.atlassian.net/browse/CC-22818
https://confluentinc.atlassian.net/browse/SECOPS-15591
Solution
Update common version and remove unused pinned avro version.
Fix schema for transaction.avro

Does this solution apply anywhere else?

[ ] yes
[ ] no

If yes, where?
Test Strategy
mvn dependency:tree | grep org.apache.avro
[INFO] |  +- org.apache.avro:avro:jar:1.11.3:compile

Testing done:

[ ] Unit tests
[ ] Integration tests
[ ] System tests
[ ] Manual tests

Release Plan

Build - https://jenkins.confluent.io/job/Confluent Public Repo PR builder/job/kafka-connect-datagen/view/change-requests/job/PR-136/4/

Build - https://jenkins.confluent.io/job/Confluent Public Repo PR builder/job/kafka-connect-datagen/view/change-requests/job/PR-136/5/


Net types not explicitly declared, but default_nettyp set to none

With 08cd6eba30b5b59bbd33a808a9173203d7a7175b and 581068fea64d0d978f060a259cfbb2756bc88e90 the compiler directive ``default_nettype none` was introduced to some (not all) files, including all testbench files.
The directive requires that the type of nets of all ports in the modules must be defined.
The design of the management soc uses implicit port type declarations most of the time. Moreover the produced netlists from yosys do not include port type declarations!
Using any other simulator than Icarus Verilog, above described problem will result in "missing net type declarations" errors.
This problem is closely related to the google/skywater-pdk#198, where the use of implicit net type declarations was already discussed but without a resolution.
Due to the use of implicit declarations everywhere, I would suggest to remove the compiler directive ``default_nettype none`.

If we could get the PDK libraries to explicitly declare wires, then this discussion would be resolved.
For example:
change output Y;
to output wire Y;
Better yet, just migrate to the Verilog 2001 convention of fully qualified port names.


UploadOperation is using unnecessary GetUploadLinkOperation

After going through the code of UploadOperation.cs and GetUploadLinkOperation.cs, is it actually necessary to issue a separate web request to get the upload link? According to MSDN, it is perfectly valid to generate the upload link client-side, by simply using the following format:

https://apis.live.net/v5.0/{ID}/files/

Where {ID} is folder ID, such as folder.ca1234567ad234c2.
Constructing the upload link in code (instead of issuing a dedicated request to the server) would be much faster and also cleaner solution.

@paya-cz You are absolutely right, upload operation could be updated to detect if you provided that object with a folder id and then attempt to upload to that location avoiding the round-trip on checking the existence of the folder.  I'd be happy to accept a pull request that offered this kind of functionality.

At this time we will not be adding a change to reduce the roundtrips, feel free to submit a pull request and refresh this issue.


fix columns alignment and margin top

Test URLs:

Before: https://main--moleculardevices--hlxsites.hlx.page/applications/cell-counting/cell-counter-a431-cells
After: https://cell-counter-a431-cells--moleculardevices--hlxsites.hlx.page/applications/cell-counting/cell-counter-a431-cells

Aren't these styles a bit too generic? Do they apply to all places where columns are used? Would it make sense to namespace it a bit?


add ssh-agent plugin to base plugins

Adding ssh-agent plugin. Refer to #613 for more details.
/assign @gabemontero @bparees making this change if you determine it satisfies the 80% criteria.

/ok-to-test

@gabemontero When I built the docker image locally, I had to delete my volume in order for the ssh-agent plugin to actually get the plugin installed. Is there some caching going on with this jenkins job?

@gabemontero reverted the test. I understand on the pending decision. I did this in case the decision is yes.

this time the red hat registry flaked ...

/retest

Hey @Gl4di4torRr - following up here.  After some internal polling, sorry, but I'm going to refrain from adding ssh-agent to the base jenkins image.  As discussed before, follow the intended pattern of building your own images off of the openshift/jenkins image, using one or our extension mechanisms, to add ssh-agent or other plugins that facilitate this scenario.
thanks

@gabemontero no worries. glad to help out where i can.


docker deploy failure

Describe the bug
Use docker and set -p 3010:3000  -p 8000:8000, ip is <IP_ADDRESS>.
Open http://<IP_ADDRESS>:3010 , find console print WebSocket connection to 'ws://localhost:8000/event/?EIO=4&transport=websocket' failed:, page rendering is OK but can't be interacted.
To Reproduce
Use juwan92/pynecone:1 image as base, run a new empty project with pc init and pc run --env prod, default pcconfig.py
Specifics (please complete the following information):

Python Version: 3.8.16
Pynecone Version: 0.1.24
OS: 46~20.04.1-Ubuntu x86_64 GNU/Linux
Browser (Optional): Edge

Try to configure api_url in pcconfig.py to also use your IP instead of localhost, seems to me it's a problem with how you access it.
Let me know if that fix the problem.

Set api_url to my server ip, then ws failure is gone.
However, ip isn't public, can't be access by others.

Set api_url to my server ip, then ws failure is gone.
However, ip isn't public, can't be accessed by others.
How about pc deploy can solve it?

Just change your api_url to the website url which is public

I'm having the same problem.
When built on the linux server side and accessed from the linux/windows client, both buttons did not work, and after testing, I found that it was unable to set off the corresponding event.
However, when it is built on windows server side, it can be accessed from linux/windows client side, both can be normal.
It should be noted that the configuration of pcconfig is the same in linux server and windows server, but the situation that it does not work properly in linux occurs.
Please, let me know if there is a way to fix the problem.

Thanks for sharing @jackhappystar, lmk if this resolves your issue.


Node.js Structural Comparisons

Article checklist
NOTE: (Please ensure that you have only one open issue + linked pull request at a time. This will ensure that we complete the article in a timely manner from inception to publishing.)
Preliminary Checks - Formatting and Structure

[ ] Does your article follow any of the suggested structure formats? - see example formats
[x] Is your article properly formatted in Markdown? - see Github Markdown guide
[x] Have you used the correct folder and file structure? - see Contribution Guidelines
[x] Is your article over 750 words? - Tool: Wordcounter.net
[ ] Have you used our preferred heading structure? - H3 (###) and up
[x] Does your article provide enough value and detail about your topic? - Articles should be clear, accurate, and fully explained.
[ ] Can your article be understood by beginners? - Assume the audience is smart but has no prior exposure to the common terminology in your article.
[x] Have you included a hero image that is 600x400 pixels and have the copyright to use it? - Tools: Pixlr Image Editor for resizing and Unsplash for Creative Commons images
[x] In your PR - add a comment that states "This fixes #(enter your Idea Content Form #)" to link and close your Idea Content (issue) when the PR is merged.

Grammar & plagiarism checks

[ ] Have you spellchecked and grammar checked your article? - Tools: Write&Improve - ProWritingAid -  Grammarly
[ ] Please place your article through a 3rd party plagiarism checker? We suggest using Quetext, this tool is free and has a daily limit. We typically accept articles with 10% or less.
[ ] Have you checked your article for readability? - Tool: Hemmingway
[x] Have you added sources for quotes and images that aren't yours?

Technical checks

[x] Are your code snippets properly formatted for syntax highlighting - see Syntax guide

Contribution guidelines
For first-time contributors and for more details, see Contributing Guidelines
This fixes #1928.

Hi @cyberShaw
I had to make another PR because of some conflicts. In other words, the PR should have been published and pushed upstream.
I will have to close this. The proper PR number is #2045 .

Hi @cyberShaw
I had to make another PR because of some conflicts. In other words, the PR should have been published and pushed upstream.
I will have to close this. The proper PR number is #2045 .

Understood, I've assigned myself to the new PR 👍🏼


Creation of sample pool and volume group for fileshare

What this PR does / why we need it:
Create sample pool and volume group for fileshare
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
Special notes for your reviewer:
Release note:

Coverage decreased (-0.02%) to 28.285% when pulling 7e363f701652b58807661d1a082b24a756980e71 on PravinRanjan10:development into e62cde3f3c3db4c43413197fd5e00f62d0ece53c on opensds:development.

Please add DeleteFileShareAclOpts in dock and controller.

Please address the ci errors.


Dragonrage: Watch writinguniverse.com

Dragonrage requests the watch of the watch_keyword writinguniverse\.com. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

writinguniverse\.com has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by cigien in Charcoal HQ


kapp tries to list resources from deprecated api versions

What steps did you take:
I tried deploying an app that consists of a horizontal pod autoscaler (api version v2) on a Kubernetes cluster v1.25.
What happened:
kapp tries to make a  list call to /apis/autoscaling/v2beta2/horizontalpodautoscalers which is deprecated and there's no resource in my app with this apiVersion.
What did you expect:
I expected kapp to only list the api versions that are part of my app.
Anything else you would like to add:
Currently we scope list calls to unique GKs, so we try to make a list call for all versions. We need to be smart enough to exclude such deprecated apis.
Environment:

kapp version (use kapp --version):
OS (e.g. from /etc/os-release):
Kubernetes version (use kubectl version)

Vote on this request
This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.
👍 "I would like to see this addressed as soon as possible"
👎 "There are other more important things to focus on right now"
We are also happy to receive and review Pull Requests if you want to help working on this issue.

We should be conscious about how this affects diffing during migrating CRs from to a newer api version. We should probably verify how ignoring older versions would affect diffs and the end result.

this blocks automatic upgrades on gke when using kapp

@100mik gke checks for uses of deprecated apis in the last 30 days and blocks auto-upgrading the k8s cluster if there are any so when we're using kapp all the time GKE never auto-upgrades our clusters off of deprecated versions.

That is helpful, we might have to think about how we can have better control knobs for this behaviour after gauging the impact on not relying on the cluster resources


TCK cannot require Contains, EndsWith, IgnoreCase, Like, Null, StartsWith for Document and Graph, TCK cannot require Between, True, False for Key-Value

fixes #781

@KyleAure @njr-11 I am still facing issues using the newest version, e.g.:
https://github.com/eclipse/jnosql-extensions/blob/main/jnosql-data-tck-runner/pom.xml#L175-L186
And I still have a problem with the:
testContainsInString

And similar.

Furthermore, what do you think if we simplify to only RELATIONAL and NoSQL databases?

Furthermore, what do you think if we simplify to only RELATIONAL and NoSQL databases?

As we currently have it, there are a large number of cases where vendors can skip TCK test assertions when run against Key-Value and a much lesser number assertions that are skipped for Graph and Document.  If you simplify to NoSQL, but still want Key-Value to pass, then we would need to start allowing Graph and Document to skip a large number of assertions, meaning the TCK would not be enforcing the specification requirements that it should be for those database types, and vendors would be able to certify implementations that pass on NoSQL even though they don't do what the spec says to.  We should not make a change like that.


Redshift COPY should use the columns defined

Currently the Redshift COPY is not using the defined columns (if present) - these are only used in table creation. This means that the COPY job relies upon the order of the columns being the same as in the CSV file.
This is a big problem, as it can cause data to be inserted into the wrong columns.
Example
For example, imagine we are importing two fields from a pull from another DB via a CSV in S3:
order_id, us_dollars_float
The DB maintainers then decide they will drop the column of us_dollars_float and instead send it as an integer in cents, so we now receive:
order_id, us_dollars_cents_int
However we don't want to drop the old column in our Redshift table yet as we still have reports using it that we can't change right now.
So our RS table looks like:
order_id, us_dollars_float, us_dollars_cents_int
This means if we now do the COPY without specifying the columns we will insert in to the old column with the new data (now in cents!), as it only uses the column ordering.
I.e. Luigi will do:
COPY our_table FROM 'ourfile.csv' ...
Whereas we want:
COPY our_table (order_id, us_dollars_cents_int) FROM 'ourfile.csv'...
This is a big problem for us, so I'll submit my Pull Request immediately.

See PR #2245

Merged now :) If using the columns variable be sure to set them to match your Redshift table.


[textarea]: sync with @carbon/react version

Duplicate of https://github.com/carbon-design-system/carbon-for-ibm-dotcom/issues/10234


File transfer is slow in Linux docker environment, where as it is working normal in windows

FTP Server OS: Linux
FTP Server Type: FTPS
Client Computer OS: Linux docker
FluentFTP Version: latest version
Framework: .NETCore 5
File transfer is slow in Linux docker environment, where as it is working normal in windows
For example: File Upload is taking 30 seconds, where as same source is working in 2~3 seconds in windows server.
Logs :
2023-06-03 10:09:49.681 +00:00 [Information] File Processed with name:-'myfile.txt'
2023-06-03 10:09:49.682 +00:00 [Information] File Writer : Create Only
2023-06-03 10:09:49.682 +00:00 [Information] >         UploadStream("myfile.txt", Overwrite, False)
2023-06-03 10:09:49.682 +00:00 [Information] >         FileExists("myfile.txt")
2023-06-03 10:09:49.683 +00:00 [Information] >         GetNameListing("/myftps/path")
2023-06-03 10:09:49.683 +00:00 [Information] >         OpenDataStream("NLST /myftps/path", 0)
2023-06-03 10:09:49.683 +00:00 [Information] >         OpenPassiveDataStream(PASV, "NLST /myftps/path", 0)
2023-06-03 10:09:49.683 +00:00 [Information] Command:  PASV
2023-06-03 10:09:49.689 +00:00 [Information] Response: 227 Entering Passive Mode (10,87,85,231,241,52). [5ms]
2023-06-03 10:09:49.689 +00:00 [Information] Connecting to IP #1= ***:61748
2023-06-03 10:09:49.696 +00:00 [Information] Command:  NLST /myftps/path
2023-06-03 10:09:49.703 +00:00 [Information] Response: 150 Here comes the directory listing. [6ms]
2023-06-03 10:09:49.703 +00:00 [Warning] SSL Buffering force disabled, is .NET 5.0 and later
2023-06-03 10:10:04.721 +00:00 [Information] FTPS authentication successful, lib = .NET SslStream, cipher suite = Tls12 (Aes256, 	TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, DiffieHellman, 0) [15.018s]
2023-06-03 10:10:04.722 +00:00 [Information] >         CloseDataStream()
2023-06-03 10:10:04.735 +00:00 [Information] Response: 226 Directory send OK. [15.038s]
2023-06-03 10:10:04.735 +00:00 [Information] >         OpenWrite("myfile.txt", Binary)
2023-06-03 10:10:04.735 +00:00 [Information] >         OpenDataStream("STOR myfile.txt", 0)
2023-06-03 10:10:04.735 +00:00 [Information] >         OpenPassiveDataStream(PASV, "STOR myfile.txt", 0)
2023-06-03 10:10:04.735 +00:00 [Information] Command:  PASV
2023-06-03 10:10:04.740 +00:00 [Information] Response: 227 Entering Passive Mode (10,87,85,231,240,101). [4ms]
2023-06-03 10:10:04.741 +00:00 [Information] Connecting to IP #1= ***:61541
2023-06-03 10:10:04.746 +00:00 [Information] Command:  STOR myfile.txt
2023-06-03 10:10:04.753 +00:00 [Information] Response: 150 Ok to send data. [7ms]
2023-06-03 10:10:04.753 +00:00 [Warning] SSL Buffering force disabled, is .NET 5.0 and later
2023-06-03 10:10:19.774 +00:00 [Information] FTPS authentication successful, lib = .NET SslStream, cipher suite = Tls12 (Aes256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, DiffieHellman, 0) [15.021s]
2023-06-03 10:10:19.787 +00:00 [Information] Response: 226 Transfer complete. [15.041s]
2023-06-03 10:10:19.874 +00:00 [Information] File Create Status :Successfully

Here's the sample source used for this application
private void UploadFile()
{
	var ftpConfig = new FtpConfig()
	{
		ValidateAnyCertificate = true,
		LogToConsole = true,
		TransferChunkSize = 100000
	};

	using var conn = new FTPClient(ipaddress, username, password, config: ftpConfig, logger: new FtpLogAdapter(_logger));
	conn.Config.EncryptionMode = FtpEncryptionMode.Implicit;
	conn.Config.CheckCapabilities = false;
	conn.Config.NoopInterval = 0;
	conn.Config.SslBuffering = FtpsBuffering.On; //SslBuffering is force disabled for .Net5
	conn.Config.SslProtocols = System.Security.Authentication.SslProtocols.Tls12;
	conn.Config.SocketKeepAlive = true;
	conn.ValidateCertificate += new FtpSslValidation(OnValidateCertificate);
	conn.AutoConnect();

	var datawithstringformat = GenerateStreamFromString("mystring value");
	conn.UploadStream(fstream, filename);
}

private static Stream GenerateStreamFromString(string s)
{
	var stream = new MemoryStream();
	var writer = new StreamWriter(stream);
	writer.Write(s);
	writer.Flush();
	stream.Position = 0;
	return stream;
}

private static void OnValidateCertificate(BaseFtpClient control, FtpSslValidationEventArgs e)
{
	e.Accept = true;
}

Looks like SSL Authentication (Handshake etc.) is taking ages:
2023-06-03 10:10:04.753 +00:00 [Warning] SSL Buffering force disabled, is .NET 5.0 and later
2023-06-03 10:10:19.774 +00:00 [Information] FTPS authentication successful, lib = .NET SslStream, cipher suite = Tls12 (Aes256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, DiffieHellman, 0) [15.021s]

Looks like SSL Authentication (Handshake etc.) is taking ages:
2023-06-03 10:10:04.753 +00:00 [Warning] SSL Buffering force disabled, is .NET 5.0 and later
2023-06-03 10:10:19.774 +00:00 [Information] FTPS authentication successful, lib = .NET SslStream, cipher suite = Tls12 (Aes256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, DiffieHellman, 0) [15.021s]

Whether is it taking different port than default 990 for SSL authentication?

When I tried to upload multiple files using same connection, for each file SSL authentication is happening and taking 15 seconds. Whether is it possible to suppress the authentication after the first time ?

I don't know the reason for running faster in window server, but I found 2 mistakes with the code, which makes delay in Linux

I didn't pass the port 990 in FtpClient
using var conn = new FTPClient(ipaddress, username, password, config: ftpConfig, logger: new FtpLogAdapter(_logger));
I used conn.AutoConnect();

Even though I've mentioned the Encryption Mode as Implicit, AutoConnect won't bother about that and it tries for all possible connections like Port 21, 22, and 990
Now working modified code:
using var conn = new FTPClient(ipaddress, username, password, port: 990, config: ftpConfig, logger: new FtpLogAdapter(_logger));
conn.Config.EncryptionMode = FtpEncryptionMode.Implicit;
conn.Connect();

Even If I mention port and use AutoConnect, it ignores the port info and tries 21 by default.
If I explicitly mentioned all information and use Connect, it directly tries the same config and it works normally without any delay.
If we know all configurations, then it would be better to use Connect() / Connect(true).

Thanks a lot for this explanation. I understand what you are saying. And yes - some configuration options (like implicit and others) are not taken into account to shorten the processing logic of autoconnect. Maybe this would be something for the todo-list.


Return of the Old Versions Bug

I was looking through some blueprints earlier today, and I found that the versions in Bubbleprints was out of date. Comparing blueprints to patch notes, it looks like the version that I have in Bubbleprints is 1.3.2c? I assume this is a silly typo in generating the binz file again. No idea how long the bug has been around.
Currently using version 0.25.0.
Current binz file loaded:

Example of an outdated blueprint, Guid fd2e11ebb8a14d6599450fc27f03486a had a new import etude (InevitableDarkness_KilledOnUnfair) added for DLC1 in 1.3.6d. Said etude is not in the viewable files and is missing from the imported etudes:

I don't have any particular reason I need this fixed in a hurry, but it would be nice to have it fixed before the binz file for 1.4 (and especially before the binz file for EE) is generated. Thanks!

Hmm. Possibly broke something by accident, pushed 1.4.0 can you check it is actually right? :-(

Checking the files, MainCampaign is still a pre-1.3.6d version as it is missing the InevitableDarkness_KilledOnUnfair import. Checked for some new files that were added in 1.4 and could not find them. So yeah, the problem still appears to be there on my end.

Ah fuck, I'll have a look to see what I've messed up.

Ah no worries. The new binz has the updated version of the MainCampaign file. Checking some additional files, it looks like the files include bugfixes from 1.4, so it appears to be fully up to date.
Thanks for the work!

No worries I literally just had a hard coded path to the wrong wrath install 😔

PS: I usually respond a lot quicker on discord (the wrath one in mod-dev-technical or mod-user-general) just cos github notifications absolutely suck.


[Question]: Ensemble Querying / Advanced Retrieving from VectorStore

Question Validation

[X] I have searched both the documentation and discord for an answer.

Question
Hi,
i've stored a lot of Nodes in a QdrantVectorStore.
https://docs.llamaindex.ai/en/stable/examples/query_engine/RetrieverRouterQueryEngine/
https://docs.llamaindex.ai/en/stable/examples/query_engine/ensemble_query_engine/
I'm not sure if I'm confusing things. If so, I'm sorry - please explain it to me.
I can't index my data with every start of my application because it is using a large knowledge index (separation of concerns).
Q1:
Is it possible to create a keyword index from a VectorStore? The data is already there and also keywords could be used through the keyword extractor.
The documentation suggests using the nodes. I don't understand why I need to load the nodes again if I already stored them in a VectorStoreIndex.
Q2:
How do I know what data the retriever is using? does it only use the embedded user query or can I use a pre-filtered search (e.g. keywords) to retrieve the relevant documents?

indexing.py
config.embed_model = HuggingFaceEmbedding(model_name="mixedbread-ai/mxbai-embed-large-v1")
config.node_parser = SentenceSplitter.from_defaults(chunk_size=1024, chunk_overlap=100)

list_of_documents = [
  Document(
      text="...",
      metadata=[...]
  ),
...
]

nodes = config.node_parser.get_nodes_from_documents(
  list_of_documents,
  show_progress=DEBUG
)

client = QdrantClient()
vector_store = QdrantVectorStore(client=client, collection_name="large_index")
storage_context = StorageContext.from_defaults(vector_store=vector_store)

VectorStoreIndex(
    embed_model=config.embed_model,
    storage_context=storage_context,
    nodes=nodes,
    show_progress=DEBUG
)

retrieving.py
config.llm = Ollama(model="mistral")
client = QdrantClient()
vector_store = QdrantVectorStore(client=client, collection_name=collection__name)
index = VectorStoreIndex.from_vector_store(vector_store=vector_store)
query_str = "I want to watch a movie about space"
chat_engine = index.as_chat_engine(
    llm=config.llm,
    verbose=DEBUG
)
streaming_response = chat_engine.stream_chat(query_str)
for token in streaming_response.response_gen:
    print(token, end="")

Q1: You need the nodes, because you need to extract and map keywords to nodes. This does not work with a vector index. Some vector dbs offer hybrid search, which is similar-ish to keyword search.
Q2: It uses all the data in your configured index/namespace/collection. If you apply metadata filters, those filter the data before performing the top-k search

Thanks for the explanation. That was helpful.


HIVE-25800: Improvement in loadDynamicPartitions() to not load all partitions from HMS for managed table

What changes were proposed in this pull request?
HIVE-20661 added an improvement in loadDynamicPartitions() api in Hive.java to not add partitions one
by one in HMS. It used to fetch all the existing partitions for a table from HMS and compare that with
dynamic partitions list to decide old and new partitions to be added to HMS (in batches). The call to
fetch all partitions has introduced a performance regression for tables with large number of
partitions (of the order of 100K).
This is fixed for external tables in HIVE-25178. However for ACID tables there is an open Jira HIVE-25187.
Until we have an appropriate fix in HIVE-25187,we can skip fetching all partitions. Instead, in the
threadPool which loads each partition individually,call getPartition() to check if the partition already
exists in HMS or not. This will introduce additional getPartition() call for every partition to be loaded
dynamically but does not fetch all existing partitions for a table anymore.
Why are the changes needed?

Does this PR introduce any user-facing change?
No
How was this patch tested?
Since it is an improvement in existing logic, therefore relying on existing tests.

@lcspinter @rbalamohan @nrg4878 : Please review and provide your feedback.

@kgyrtkirk : Please review and provide your feedback.

The following test is failing and it seems unrelated to the patch. I have triggered the run again:
org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException Caught exception while initializing the SqlSerDe)
 at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:1301)
 at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:1306)
 at org.apache.hadoop.hive.ql.ddl.table.create.CreateTableOperation.createTableNonReplaceMode(CreateTableOperation.java:140)
 at org.apache.hadoop.hive.ql.ddl.table.create.CreateTableOperation.execute(CreateTableOperation.java:98)
 at org.apache.hadoop.hive.ql.ddl.DDLTask.execute(DDLTask.java:84)
 at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:212)
 at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:105)
 at org.apache.hadoop.hive.ql.Executor.launchTask(Executor.java:365)
 at org.apache.hadoop.hive.ql.Executor.launchTasks(Executor.java:338)
 at org.apache.hadoop.hive.ql.Executor.runTasks(Executor.java:249)
 at org.apache.hadoop.hive.ql.Executor.execute(Executor.java:110)
 at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:348)
 at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:204)
 at org.apache.hadoop.hive.ql.Driver.run(Driver.java:153)
 at org.apache.hadoop.hive.ql.Driver.run(Driver.java:148)
 at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:164)
 at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:230)
 at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:256)
 at org.apache.hadoop.hive.cli.CliDriver.processCmd1(CliDriver.java:201)
 at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:127)
 at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:422)
 at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:353)
 at org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:726)
 at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:696)
 at org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:114)
 at org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:157)
 at org.apache.hadoop.hive.cli.split8.TestMiniLlapLocalCliDriver.testCliDriver(TestMiniLlapLocalCliDriver.java:62)

Fix has been merged. Please close the PR

Thank you @lcspinter for the approval and @nrg4878 for merging it. Commit merged to master: https://github.com/apache/hive/commit/63c6d8ba70dfa59e791e1e49f44629c22fb41b7f


Update readme, typo in name file

A simple fix with a typo in the name file.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Thank you very much for your contribution!
Please sign the CLA before we can merge this PR, appreciate it!

@huan done !!!


Able to parse urlencoded data in body

This fixes https://github.com/ParsePlatform/parse-server/issues/101
I wasn't sure how to run the test suite. Could you explain this here: https://github.com/ParsePlatform/parse-server/blob/master/CONTRIBUTING.md?

Was able to run the tests just now, they failed. Will look into it further


Update sbt-scoverage to 1.9.0

Updates org.scoverage:sbt-scoverage from 1.8.2 to 1.9.0.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scoverage", artifactId = "sbt-scoverage" } ]

labels: sbt-plugin-update, semver-minor

Codecov Report

Merging #375 (e21fafc) into master (f218f17) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #375   +/-   ##
=======================================
  Coverage   91.12%   91.12%
=======================================
  Files          17       17
  Lines         293      293
  Branches        1        1
=======================================
  Hits          267      267
  Misses         26       26

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f218f17...e21fafc. Read the comment docs.


Oracle throws ORA-00972: identifier is too long

Describe the bug
NotificationPortlet attempts to create tables that exceed the Oracle Object Name limit in versions previous to 12.2. Databases need to be configured with COMPATIBLE set to 12.2.
To Reproduce
Steps to reproduce the behavior:

Set up an Oracle database to back uPortal
Build and run uPortal ./gradlew portalInit portalOpen
Grep NotificationPortlet.log for ORA-00972

Expected behavior
Tables successfully created.
Additional context
https://docs.oracle.com/en/database/oracle/oracle-database/12.2/sqlrf/Database-Object-Names-and-Qualifiers.html#GUID-75337742-67FD-4EC0-985F-741C93D918DA

Root cause is that JPA annotations were configured to create intermediary tables for joins. Solution is to reconfigure to use foreign key relationships directly between tables.


Update early access timeline, remove announcement banner

Update timeline to reflect revised estimates 😅
Remove announcement banner -- been up since August and felt it was getting stale / annoying

@jefferenced The nice thing about our site on NextJS / Vercel is we can preview it and comment directly on the drafted site build using the link in this comment

Visit Preview

That's awesome, we had a similar feature 'view deployment' (preview) with Fly.io, so I'm very comfortable with this kind of setup!

@jefferenced Thanks for the feedback, updated!

Refs #2519


Add spatial filter actions in Redux

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

10 of 16 (62.5%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.2%) to 70.246%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

packages/react-redux/src/slices/cartoSlice.js
10
16
62.5%

Totals

Change from base Build<PHONE_NUMBER>:
-0.2%

Covered Lines:
973

Relevant Lines:
1294

💛  - Coveralls


non-disruptive waypoint additions

Describe the feature request
Adding a waypoint proxy should not cause disruption to traffic. We've pitched ambient as "only pay for what you use."  We've encouraged adoption of the ztunnel first and later adding waypoint as you need. If adding the waypoint is a disruptive event it could dampen enthusiasm for this approach.
At present adding a waypoint seems to create a race between the waypoint becoming ready and proxies being configured to use the waypoint. Relevant ztunnel logs for this type of event:
2023-08-10T18:46:46.830246Z DEBUG outbound{id=6a3bf7d5d78bbad01004ec89a7276823}: ztunnel::state: fetch address network_addr.address=<IP_ADDRESS>
2023-08-10T18:46:46.830258Z DEBUG outbound{id=6a3bf7d5d78bbad01004ec89a7276823}: ztunnel::state: found upstream Upstream{reviews-v3-6b64666d78-fxf2s with uid Kubernetes//Pod/bookinfo/reviews-v3-6b64666d78-fxf2s:9080 via <IP_ADDRESS>:15008 (HBONE)} from VIP <IP_ADDRESS>
2023-08-10T18:46:46.830265Z DEBUG outbound{id=6a3bf7d5d78bbad01004ec89a7276823}: ztunnel::state: fetch address network_addr.address=<IP_ADDRESS>
2023-08-10T18:46:46.830268Z DEBUG outbound{id=6a3bf7d5d78bbad01004ec89a7276823}: ztunnel::state: VIP <IP_ADDRESS>:15008 has no healthy endpoints
2023-08-10T18:46:46.830271Z DEBUG outbound{id=6a3bf7d5d78bbad01004ec89a7276823}: ztunnel::state: waypoint upstream not found wl.name=reviews-v3-6b64666d78-fxf2s
2023-08-10T18:46:46.830359Z  WARN outbound{id=6a3bf7d5d78bbad01004ec89a7276823}: ztunnel::proxy::outbound: failed dur=215.671µs err=unknown waypoint: failed to find waypoint for workload: reviews-v3-6b64666d78-fxf2s

Similar behavior has been observed for traffic where the client has a sidecar as well but it's less common.
Describe alternatives you've considered
Ask users to deploy waypoints up front even if they don't need them.
Affected product area (please put an X in all that apply)
[X] Ambient
[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
Affected features (please put an X in all that apply)
[ ] Multi Cluster
[ ] Virtual Machine
[ ] Multi Control Plane
Additional context
After considering some options I think potential solutions are:

Make istiod more context aware. If waypoint is deployed but there is no policy used which requires us to send traffic to waypoint do not configure proxies to target the waypoint for that service. Only when there is a policy to enforce at waypoint would it be configured. Advise users to deploy waypoint prior to applying any policy.
Implement a grace period approach to allow time for waypoints to become ready. This should likely be entirely in istiod where proxies are not configured to use the waypoint to reach ns/sa workloads until grace period expires or waypoints become ready. It could potentially also be implemented in the data plane but this seems like unnecessary work being done in the data path and is likely undesirable as a result.
both, could consider doing both to provide some extra assurances that users won't be impacted even if they do things out of order

My original expectation was that we would "ratchet" the readiness of gateway. Once it becomes ready once, we start sending to it. If it later becomes unready we stop sending to it. I think there is some TODOs around this already.
One mechanism for that can be to not allocate the gateway.status.address until its ready once.

I'd agree, I think once it becomes ready then our obligation is to use it. If for some reason there's an outage later on down the line it's an outage, we can't bypass it at that point

Yep, same with ztunnel initial rollout as well. Ben looked at a lot into that, not sure the current status

What is the easiest way to tell if the waypoint is ready once?

I am not sure if we want Istiod to be context aware, I'd wait till users ask for it.  Plus like we discussed, how do we know if users deploy waypoint for L7 telemetry but not for policy.

* Make istiod more context aware. If waypoint is deployed but there is no policy used which requires us to send traffic to waypoint do not configure proxies to target the waypoint for that service. Only when there is a policy to enforce at waypoint would it be configured. Advise users to deploy waypoint prior to applying any policy.

not sure if this is related-to/encompassed-by https://github.com/istio/ztunnel/issues/311 - but linking for reference:
https://github.com/istio/ztunnel/issues/311

My original expectation was that we would "ratchet" the readiness of gateway. Once it becomes ready once, we start sending to it. If it later becomes unready we do NOT stop sending to it. I think there is some TODOs around this already.

I'm assuming there would be some kind of timeout here? I'd expect istio L7 config to be eventually consistent. If the waypoint never becomes ready if we had L7 auth I'd eventually expect that to start failing, even in the ratchet scenario

That's a reasonable argument. If k8s operators think that unready waypoints are monitored closely enough that operators will notice and address, then I'm ok with that.

I think I'm slightly in favor of a timeout because we're dealing with policy. In general, my preference is to fail in an obvious way. If the waypoint isn't ready in ~5 minutes, then yeah it's time for the system to act up to bring attention to the broken state.

I don't think I have ever seen something in Kubernetes where if you don't resolve an issue within 5 minutes it decides to punish you by causing an outage... it seems like not a very safe idea

Well, the outage would be scoped to that particular namespace/service account depending on the granularity chosen. My thinking is, if we don't start sending to the waypoint after some upper bound, that creates:

An attack vector where if a noisy neighbor hogs node resources and cluster-autoscaler is taking awhile, any policy for the waypoint in question won't be enforced
A situation where a user "just has to know" that ztunnel won't send to waypoint if it isn't ready to correctly debug unexpected behavior.

The current promise we have is "Once a config is fully applied [eventual consistency], its always applied". This is actually trickier than it sounds, since we never tell a user a config has been applied or not. But in general in Kubernetes this is the pattern - you can never just blindly apply and walk away, always check it became Ready after.
The same, IMO, applies here. we can encourage this as well by making istioctl x waypoint apply also wait (or --wait), and document for manual installs how to wait for it to be ready. So they don't have to know magically -- there is a well defined pattern for how they know

+1 to the above, but the "just know" I'm talking about is referring to ztunnel behavior. IIUC, what's being proposed is for ztunnel to not send traffic to a waypoint until it's ready (once it's ready once, traffic is always routed there even if it becomes unready again). If there's no upper bound on that readiness check, any policy applied to that waypoint will be bypassed in a silent, undetectable way. Sure it's best practice to make sure to check ready, but imagine an ambient cluster with hundreds of services and a hundred waypoint gateways. You've gotta check that particular Gateway's status to see if the waypoint is programmed and worse, you must look at the pods managed by that Gateway to see if it's Ready. By eventually sending traffic to the waypoint to force 503s, we surface the error to the user. How else to they discover the pods aren't ready (say in the cluster autoscaler scenario i mentioned)

No, my suggestion is Istiod does not configure a Waypoint at all until its Ready (the first time).

you must look at the pods managed by that Gateway to see if it's Ready.

Istiod is already doing this, see https://github.com/istio/istio/pull/46500

How else to they discover the pods aren't ready

The Gateway status will report this

No, my suggestion is Istiod does not configure a Waypoint at all until its Ready (the first time).

Ack - I thought istiod would configure the waypoint and ztunnel just won't send until ready. I'm good with this 👍🏾

https://github.com/istio/istio/pull/46540 is merged, can this be closed?

I think it can close. We can reopen or create a new issue if we find it's needed in 1.19 as well.


TODO: 5.11 Disable ability to login to another user's active and locked session

I believe this is off by default, but want a way to audit/remediate if not.

This was not off by default on my Sierra machine.
Simple search and replace with sed in /etc/pam.d/screensaver


Working on NULL inspect response message issue

This PR contains multiple breaking changes unfortunately.

I was working on these changes, and I have just published a draft PR with the last changes.


Release Cryostat 2.1 on OperatorHub.io

Now that the 2.1 upstream release is available (see github tags), Cryostat 2.1 can be released on OperatorHub. Similar to the 2.0 release, you'll need to complete the prerequisites and open a PR with the updated operator.
To speed up the approval process, you can also run the operator pre-submission tests as explained here. These tests are identical to the tests that will run when you open the PR. I think this script worked last time for 2.0:
# (assuming you have podman already installed)
# setup steps
sudo dnf install ansible
sudo dnf install kubernetes-kubeadm
sudo swapoff -a # [disable swap for kubelet to work](https://stackoverflow.com/a/52196985)
systemctl enable kubelet.service
systemctl start kubelet

# This command will run the test suite. Notice the container tool is Docker:
OPP_CONTAINER_TOOL=docker OPP_AUTO_PACKAGEMANIFEST_CLUSTER_VERSION_LABEL=1 OPP_PRODUCTION_TYPE=k8s \
bash <(curl -sL https://raw.githubusercontent.com/redhat-openshift-ecosystem/community-operators-pipeline/ci/latest/ci/scripts/opp.sh) \
all \
operators/cryostat-operator/2.0.0 \
jan-law/community-operators \
cryostat-operator

If the new interns are interested, I was thinking it would be good for them to get involved with the OperatorHub release. Hareet should also be available to help when he gets back on the 26th too

2.1.1 has now been released. Any takers for this?

2.1.1 has now been released. Any takers for this?

@tthvo @maxcao13 ?

Yeah, I want to try this, @tthvo do you want to do it as well? I think we are going to definitely need some guidance.

yeh sounds great! we can both work on this! @maxcao13

Here's the issue for the original release on OperatorHub.io that @jan-law and I worked on. The PR for the release can be found here.
The issue I've linked contains some context to get you started. Much of the discussion on the issue has to do with errors we encountered running Cryostat on Kubernetes (as opposed to OpenShift) that have since been resolved but there is still useful info. regarding the packaging process for the release.

Me and @tthvo are having some trouble getting the tests through locally. The error we both see is the one shown here http://pastebin.test.redhat.com/1059686
@hareetd and @jan-law helped us through, but we couldn't get past it. Interestingly the tests passed on @hareetd Fedora 36 local with Podman 4, but we have are not sure why it does not work for us. We've tried restarting our computers and docker as well.
Things we have tried:
sudo dnf install ansible
sudo dnf install kubernetes-kubeadm
sudo swapoff -a
systemctl start docker
systemctl enable kubelet.service
systemctl start kubelet
minikube start --driver=kvm2

OPP_CONTAINER_TOOL=docker OPP_AUTO_PACKAGEMANIFEST_CLUSTER_VERSION_LABEL=1 OPP_PRODUCTION_TYPE=k8s bash <(curl -sL https://raw.githubusercontent.com/redhat-openshift-ecosystem/community-operators-pipeline/ci/latest/ci/scripts/opp.sh) \
all \
operators/cryostat-operator/2.1.1 \
maxcao13/community-operators \
cryostat-operator

My forked repo and branch is here
https://github.com/maxcao13/community-operators/tree/cryostat-operator
Fedora 35:
Podman version: 3.4.7
Docker version 20.10.17, build 100c701
Minikube version: v1.25.2

It turned out the script skipped the task to create the registry if kind is not available on PATH.  We noticed sudo seems to run in a different environment as PATH is different and does not include $HOME/.local/bin so kind should not be put there.
https://github.com/redhat-openshift-ecosystem/operator-test-playbooks/blob/master/upstream/roles/reset_kind/tasks/reset.yml
Also, running kind v0.12.0` appears to cause:
TASK [reset_kind : Configuring Kind with registry] **********************************************************************************************************
fatal: [localhost]: FAILED! => {"changed": true, "cmd": "/tmp/kind-with-registry.sh", "delta": "0:00:31.701774", "end": "2022-06-20 20:06:53.879448", "msg": "non-zero return code", "rc": 1, "start": "2022-06-20 20:06:22.177674", "stderr": "Error: No such container: kind-registry\nCreating cluster \"operator-test\" ...\n • Ensuring node image (kindest/node:v1.21.1) 🖼  ...\n ✓ Ensuring node image (kindest/node:v1.21.1) 🖼\n • Preparing nodes 📦   ...\n ✗ Preparing nodes 📦 \nERROR: failed to create cluster: could not find a log line that matches \"Reached target .Multi-User System.|detected cgroup v1\"", "stderr_lines": ["Error: No such container: kind-registry", "Creating cluster \"operator-test\" ...", "

So, we downgraded kind to v0.11.0 as suggested here.
https://github.com/kubernetes-sigs/kind/issues/2460

Our previous on OperatorHub :))

Did we miss anyone?
---
updateGraph: semver-mode
reviewers:
 - ebaron
 - andrewazores
 - hareetd
 - jan-law
 - jiekang
 - maxcao123
 - tthvo

@hareetd @jan-law Do we also have to run OpenShift tests outlined here?

This release for OperatorHub.io is just for kubernetes users I think. I just checked the OpenShift operator catalog and the 2.1.1 release is already there.

Did we miss anyone?

Looks good to me!

Our preview on OperatorHub :))

@ebaron it looks like the example Flight Recording and Recording CRDs (custom resource definitions) are still visible on the preview. I thought those were deprecated?

Full steps for passing all OperatorHub tests
On Fedora 35:
Have kind v0.11.0 in $PATH (we had it as /usr/local/bin/kind)
Have Podman (v3.4.7)
sudo dnf install ansible #(v.2.9.27)
sudo dnf install kubernetes-kubeadm #(GitVersion:"v1.22.7")
systemctl start docker #(v.20.10.17)
sudo swapoff -a
systemctl enable kubelet.service #(Kubernetes v1.22.7)
systemctl start kubelet

If OLM resource error occurs, use
/tmp/operator-test/operator-sdk olm uninstall
To clean sources run
bash <(curl -sL https://raw.githubusercontent.com/redhat-openshift-ecosystem/community-operators-pipeline/ci/latest/ci/scripts/opp.sh) clean
Removing testing container 'op-test' ...
Removing kind registry 'kind-registry' ...
Deleting cluster "operator-test" ...
Removing cert dir '/home/mcao/.optest/certs' ..

Run test suites:
OPP_CONTAINER_TOOL=docker OPP_AUTO_PACKAGEMANIFEST_CLUSTER_VERSION_LABEL=1 OPP_PRODUCTION_TYPE=k8s bash <(curl -sL https://raw.githubusercontent.com/redhat-openshift-ecosystem/community-operators-pipeline/ci/latest/ci/scripts/opp.sh) \
all \
operators/cryostat-operator/2.1.1 \
maxcao13/community-operators \
cryostat-operator

@jan-law @hareetd We have some questions while going through the PR template.

Example templates for all Custom Resource Definitions intended to be used

This question, we need to wait for Elliott as here

Is your new CSV referenced in the appropriate channel defined in the package.yaml or annotations.yaml ?

We are not sure we understand this and how to check if valid?

Our preview on OperatorHub :))

@ebaron it looks like the example Flight Recording and Recording CRDs (custom resource definitions) are still visible on the preview. I thought those were deprecated?

That's right. They're deprecated and we use the operators.operatorframework.io/internal-objects annotation to hide them from view in the OpenShift Console. It appears that OperatorHub.io doesn't support this annotation yet though: https://github.com/k8s-operatorhub/operatorhub.io/issues/14.
Users will see warnings about those types being deprecated when using them on the command line at least.

Is your new CSV referenced in the appropriate channel defined in the package.yaml or annotations.yaml ?

We are not sure we understand this and how to check if valid?

All of our operator releases should use the stable channel going forward. This is because we only want to support the latest release. The head of the stable channel will always contain the latest released version of our operator.
package.yaml seems to refer to the old packagemanifests format, so you don't need to worry about that.
annotations.yaml is what we use with the bundle format. For the 2.1.1 release, that would be this file: https://github.com/cryostatio/cryostat-operator/blob/v2.1.1/bundle/metadata/annotations.yaml. The stable channel is listed as the only channel, and also the default:
https://github.com/cryostatio/cryostat-operator/blob/88ee94131b1fcbf5a3587f06822cfd9de45732ac/bundle/metadata/annotations.yaml#L7-L8
I think in our case, we just have to ensure that the channels listed in annotations.yaml are what we expect them to be.

Full steps for passing all OperatorHub tests

Thanks @maxcao13! This will be a very useful resource when we need to do our next release.

Did we miss anyone?
---
updateGraph: semver-mode
reviewers:
 - ebaron
 - andrewazores
 - hareetd
 - jan-law
 - jiekang
 - maxcao13
 - tthvo

Looks good!

Thanks @ebaron we will open the PR in a few minutes and send the link here

It's showing up here now: https://operatorhub.io/operator/cryostat-operator
Thanks for doing this!


Pin stays on the map when results of subsequent queries return

Hi! thanks for the amazing project. I have encountered what I think is a bug, because i think it should be irrelevant to my choice of geocoder (Nominatim).
To reproduce the bug:

search for "NUS singapore" (I have a map of singapore), press enter.
click on the one that says National University of Singapore from the dropdown list, it zooms in closer to the place;
click enter again and the map zooms in even closer;
search for the next query "NUH singapore" (do not clear manually clear search box), which has 2 results from the drop down. But when I zoom out the map, the pin for National University of Singapore is still there.

Please see attached screen recording for the reference.
Thank you!
https://user-images.githubusercontent.com/25223376/158390405-0ef0a3e5-9a54-490a-ad88-ef80b603d3c2.mp4

Thanks for reporting. Can you share a stackblitz or something similar?

@wipfli sure. demo is here: https://stackblitz.com/edit/web-platform-fszogw?file=index.html
you will need a maptiler API key to access the basemap. Do you have one?

Does the one for the MapLibre examples work? get_your_own_OpIi9ZULNHzrESv6T2vL from https://maplibre.org/maplibre-gl-js-docs/example/canvas-source/

Yes it will work. I've just updated stackblitz.

Cool, I can reproduce the bug you reported. Is there any chance that you can fix this behavior? I would be happy to review a pull request...

Maybe one has to loop through features. Here is a different example where I made only one marker being highlighted: https://codesandbox.io/s/hopeful-elion-w04kb6

How is the different example related to the bug?
I will try to read code more closely when I have the bandwidth. If i can come up with a fix i will submit a PR

Maybe it is not really related :S

Cool, thanks @xinyuluo !

Is it possible to add this nominatim example to either the main readme of this repo or the API.md?
I think nominatim is a probably the most known free geocoder in the OSM echo system.
Also I think it will be super cool to add this to Maputnik.

The code after #118 should be fully typescript, which should help in solving these kind of cases, and if not, refactoring would be a lot easier.
I see that there are promises returning from the geoApi, and there might be a race condition in this case, or a missing marker.remove call.
It should be fairly easy to solve this I hope.
Sorry for the late response...

Can you reproduce this in the following example page:
https://maplibre.org/maplibre-gl-js/docs/examples/geocoder/
I can't reproduce it...

No response, closing for now.
If this is still relevant, please let me know and I'll reopen.


JS Test Cases

I have done the Dark mode test case

I have done the Merging Functionality test case

I completed the sorting functionality

@aditya4196 Please verify

verified the tests on chrome browser


Demo page not working?

I'm trying to see the demo at http://madscript.com/boron/ but I can't access the site, anyone else has this same problem?

Yes, scam / ad link.


Email can't be sent - msmtp

First off thanks for this image.  Saved me a TON of time having to quickly migrate to FPBX 15 and PJSIP. :)
Everything is working well except for one snag, email.  When trying to send I get errors like the following:

I saw a prior closed issue about msmtp being in the base image and can see it's included, however it doesn't seem to be providing service.  I've tried explicitly adding ENABLE_SMTP=TRUE but no go.  I figure I'm missing something simple.  Any pointers?
Thanks!

I reported about this situation: https://issues.freepbx.org/browse/FREEPBX-21595
proxy is good idea, I will make PR for this.

https://github.com/tiredofit/docker-freepbx/pull/149

Update : I've made a couple of steps...
After setting ENABLE_SMTP to true, as noted in another comment, I got this:
s6-supervise (child): fatal: unable to exec run: Permission denied,
s6-supervise 15-socat: warning: unable to spawn ./run - waiting 10 seconds

I shelled into the running image, and ran ls -al /etc/services.d/15-socat/run
No user had execute set!
So I ran chmod a+x /etc/services.d/15-socat/run
The image then finished booting up.  I also set SMTP_HOST to my on LAN SMTP relay (namshi/smtp image)
Next problem : I can send emails by writing a test file and catting it to msmtp, but I have to use:
msmtp -f<EMAIL_ADDRESS>Else it returns:
msmtp: account default from /etc/msmtprc: envelope-from address is missing
So I know the relay via socat is working, and my relay host config is ok - but whenever I generate a voicemail, I don't get an email...email is enabled, my address is valid - so I presume the msmtp is arguing.
There is no from address configured in /etc/msmtprc currently.
Even changing the freepbx mailcmd in voicemail admin to /usr/sbin/sendmail -t -f<EMAIL_ADDRESS>doesn't generate an email....but I can't see an error trapped anywhere

A couple weeks ago the socat service was brought in to solve some hardcoded quirks with the backup module. I'll push a new copy of freepbx immediately with the +x bit on for that service. I'm trying to think back at what I had to do to get voicemail emails going - it's been a while. I use tiredofit/postfix-relay for my outbound smtp (even to a local LAN based SMTP) but what you are using should be fine as well. I'll do some digging in the asterisk source to see if something has changed. I'll also investigate this FROM-Address error. I did notice it on one of my other images lately so looks like we'll have to add another var.
In the meantime, there is an documented var called SMTP_ENABLE_GMAIL which adds the entry auto_from on which may give you better results at least from the command line.
Will push this new image with working socat right away and then determine what we need to do with MSMTP following that.

Hmm, not sure what's going on here.
Last night it managed to send me an email about upgradable modules, so that's something.
Voicemail notifications? Nope.
I noted it sent the email from asterisk@freepbxengine - freepbxengine is the hostname I gave the Docker Container, but I can't otherwise see where the address gets/got set? (unless it picked it up via the sendmail invocation as the calling user)

Right, cracked it (sort of)
Everytime I run msmtp manually with just a rcpt address (/usr/bin/sendmail<EMAIL_ADDRESS>I got an error about the envelope-from address missing from /etc/mstmprc
So I updated my /etc/mstmprc thus:
### Automatically generated on container start. See documentation on how to set!
account default
from<EMAIL_ADDRESS>host <IP_ADDRESS>
port 25
domain docker
maildomain local
tls off
tls_starttls off
tls_certcheck off
...and then I changed the freepbx voicemail send command to this:
/usr/sbin/sendmail -t
And tested, and it worked!

Addendum - if you could add "apt-get update" and "apt-get install dos2unix" to the package I'd be immensely grateful
(I have a custom email to mp3 script which otherwise runs, but uses dos2unix) :)

Thanks Peter. Normally an application would send the From header as well but as we've seen with Freepbx it's skipping a few key functions and uses different methods for different areas.
I'd be more than happy to include it in the package if you wanted to submit it as a PR. We can certainly add environment variables to be modified upon container bootup as well.
So what I'll do today is update the Debian base image with the from statement in msmtprc (env var). It'll take about 3 days for Docker Hub to finish off building all the dependent images on it, but we should have solved this email issue right after. I'll also put in dos2unix for you in the meantime.

I've not forked/pulled your codebase to generate a PR.
For the MP3/transcription It's a single file, so I could paste a marked up version in this chain?

Sure that works!

Ok - here is sendmailmp3.sh
This can live anywhere as long as users know the path - so they can set the FreePBX VM Sendmail cmd path to it
ATM I'm using a copy in my mounted /data folder
ENV vars I've set
VM_TRANSCRIBE - can be TRUE or FALSE (basically only does stuff if set to TRUE - suggest default of FALSE if not set
VM_TRANSCRIBE_APIKEY - IBM Watson API Key (URL comments inline - an account is needed) - should not be blank
VM_TRANSCRIBE_MODEL - see comments in line for valid values URL - suggest default of en-GB_NarrowbandModel
#!/bin/sh
# Asterisk voicemail attachment conversion script
# Revision history :
# 22/11/2010 - V1.0 - Creation by N. Bernaerts
# 07/02/2012 - V1.1 - Add handling of mails without attachment (thanks to Paul Thompson)
# 01/05/2012 - V1.2 - Use mktemp, pushd & popd
# 08/05/2012 - V1.3 - Change mp3 compression to CBR to solve some smartphone compatibility (thanks to Luca Mancino)
# 01/08/2012 - V1.4 - Add PATH definition to avoid any problem (thanks to Christopher Wolff)
# 16/07/2015 - V1.5 - Handle natively GSM WAV (thanks to Michael Munger)
# 01/08/2020 - V2.0 - Modded for revised IBM Watson URLs and Docker tiredofit/docker-freepbx

# Requires
#  dos2unix
#  sed
#  awk
#  lame

# set PATH
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

# create a temporary directory and cd to it
TMPDIR=$(mktemp -d)
cd $TMPDIR

# dump the stream to a temporary file
cat >> stream.org

# get the boundary
BOUNDARY=$(grep "boundary=" stream.org | cut -d'"' -f 2)

# cut the file into parts
# stream.part - header before the boundary
# stream.part1 - header after the bounday
# stream.part2 - body of the message
# stream.part3 - attachment in base64 (WAV file)
# stream.part4 - footer of the message
awk '/'$BOUNDARY'/{i++}{print > "stream.part"i}' stream.org

# if mail is having no audio attachment (plain text)
PLAINTEXT=$(cat stream.part1 | grep 'plain')
if [ "$PLAINTEXT" != "" ]
then

  # prepare to send the original stream
  cat stream.org > stream.new

# else, if mail is having audio attachment
else

  # cut the attachment into parts
  # stream.part3.head - header of attachment
  # stream.part3.wav.base64 - wav file of attachment (encoded base64)
  sed '7,$d' stream.part3 > stream.part3.wav.head
  sed '1,6d' stream.part3 > stream.part3.wav.base64

  # convert the base64 file to a wav file
  dos2unix -o stream.part3.wav.base64
  base64 -di stream.part3.wav.base64 > stream.part3.wav

  # convert wave file (GSM encoded or not) to PCM wav file
  sox stream.part3.wav stream.part3-pcm.wav

  if [ "$VM_TRANSCRIBE" = "TRUE" ]
  then

        CURL_OPTS=""
        # Do not change the API_USERNAME
        API_USERNAME="apikey"
        # Set the API_PASSWORD to relevant $ENV variable
        # IBM Watson PAYG account with speech transcription enabled - is required
        # More info at https://www.ibm.com/uk-en/cloud/watson-speech-to-text
        # See also http://nerdvittles.com/?page_id=25616 for more info on how to setup the IBM account
        API_PASSWORD="$VM_TRANSCRIBE_APIKEY"
        # Set the voice model to a valid IBM model
        # All models available via https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models
        API_MODEL=$VM_TRANSCRIBE_MODEL

        curl -s $CURL_OPTS -k -u $API_USERNAME:$API_PASSWORD -X POST \
            --limit-rate 40000 \
            --header "Content-Type: audio/wav" \
            --data-binary @stream.part3.wav \
            "https://stream.watsonplatform.net/speech-to-text/api/v1/recognize?continuous=true&model=$VM_TRANSCRIBE_MODEL" 1>audio.txt

        # Extract transcript results from JSON response
        TRANSCRIPT=`cat audio.txt | grep transcript | sed 's#^.*"transcript": "##g' | sed 's# "$##g'`

  fi

  # convert PCM wav file to mp3 file
  # -b 24 is using CBR, giving better compatibility on smartphones (you can use -b 32 to increase quality)
  # -V 2 is using VBR, a good compromise between quality and size for voice audio files
  lame -m m -b 24 stream.part3-pcm.wav stream.part3.mp3

  # convert back mp3 to base64 file
  base64 stream.part3.mp3 > stream.part3.mp3.base64

  # generate the new mp3 attachment header
  # change Type: audio/x-wav or audio/x-WAV to Type: audio/mpeg
  # change name="msg----.wav" or name="msg----.WAV" to name="msg----.mp3"
  sed 's/x-[wW][aA][vV]/mpeg/g' stream.part3.wav.head | sed 's/.[wW][aA][vV]/.mp3/g' > stream.part3.mp3.head

  # generate first part of mail body, converting it to LF only
  mv stream.part stream.new
  cat stream.part1 >> stream.new
  cat stream.part2 >> stream.new

  if [ "$VM_TRANSCRIBE" = "TRUE" ]
  then
        echo "--- Automated transcription result ---" >> stream.new
        echo "$TRANSCRIPT" >> stream.new
  fi

  cat stream.part3.mp3.head >> stream.new
  dos2unix -o stream.new

  # append base64 mp3 to mail body, keeping CRLF
  unix2dos -o stream.part3.mp3.base64
  cat stream.part3.mp3.base64 >> stream.new

  # append end of mail body, converting it to LF only
  echo "" >> stream.tmp
  echo "" >> stream.tmp
  cat stream.part4 >> stream.tmp
  dos2unix -o stream.tmp
  cat stream.tmp >> stream.new

fi

# send the mail thru sendmail
cat stream.new | sendmail -t

# remove all temporary files and temporary directory
rm -Rf $TMPDIR

Alright, I've made a few minor mods to the script and put it in place inside the source tree. I'm going to wait for a couple hours before I commit to make sure Docker Hub doesn't go out of sequence but we should see this tagged as release 15-5.0.4 in the next 36 hours.

Excellent stuff. Did you set an ENV for the msmtprc?

Yes, that's in the top level base image of tiredofit/debian now, with the variable SMTP_FROM which is where the holdup on pushing this new image out - Docker Hub only gives me 1 core to build with and when working at the very top level of my images it has to walk through every dependent image and rebuild. On my private build server it takes approximately 7 hours on a 16 core system, but that is also including the Alpine variants which most of my images are using, so it's a waiting game at this stage.

Image is live now. Success on the mail sending issue?

Downloading now, will update.

Updated (few days off)
I can see the watson_transcribe script, but it's not working correctly with all ENVs set.  Voicemail received but not transcription.
Also, I set FOP to be enabled and it won't autostart.  A manual kick of /usr/local/fop2/fop2_server -D gets it going

Weird - And it is working when you manually inject? Are we positive that it is executing? A simple check would be to add a
echo "Script executed at $(date)" >> /tmp/watson.log line on line 2 after the #!/bin/bash declaration.

I'll check in a mo - just having the normal fun of a reboot and all my
containers firing out of order....
On Sat, 15 Aug 2020 at 14:43, Dave Conroy<EMAIL_ADDRESS>wrote:

Weird - And it is working when you manually inject? Are we positive that
it is executing? A simple check would be to add a
echo "Script executed at $(date)" >> /tmp/watson.log line on line 2 after
the #!/bin/bash declaration.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/tiredofit/docker-freepbx/issues/146#issuecomment-674397887,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAPGIYY7YJ3IR4BDXYYCLADSA2GHBANCNFSM4N2UEWUA
.

So, my script (which lives outside sbin) is leaving a folder in /tmp owned by asterisk:asterisk
Your script (in sbin) is leaving a folder owned by root:root - which I suspect asterisk can't read/write into ( 1 x 0 byte file in it)
The sbin/ownership may be the issue - I forced the /usr/sbin copy to be chown asterisk:asterisk and I'm getting files in the folder, but the audio.txt file is missing (which is called by Curl to IBM) - so checking that now

Weird - SMTP_FROM didn't make it into the buster source tree yet I explicitly added it. I'm pushing a new release of the base image now for it.

Right, found the issues with the watson_transcribe
a) you def need to chown asterisk:asterisk the script in /usr/sbin
b) change the line in the script that reads:
curl -s "$CURL_OPTS" -k -u $API_USERNAME:"$API_PASSWORD" -X POST \
to
curl -s "$CURL_OPTS" -k -u $API_USERNAME:$API_PASSWORD -X POST \
(lose the quotes around API_PASSWORD - as they are passed to IBM which bounced them.

Have a peek at this updated script which if you have the environment variable CONTAINER_LOG_LEVEL=DEBUG set will step through the script:
#!/usr/bin/with-contenv bash

# Asterisk voicemail attachment conversion script
# Revision history :
# 22/11/2010 - V1.0 - Creation by N. Bernaerts
# 07/02/2012 - V1.1 - Add handling of mails without attachment (thanks to Paul Thompson)
# 01/05/2012 - V1.2 - Use mktemp, pushd & popd
# 08/05/2012 - V1.3 - Change mp3 compression to CBR to solve some smartphone compatibility (thanks to Luca Mancino)
# 01/08/2012 - V1.4 - Add PATH definition to avoid any problem (thanks to Christopher Wolff)
# 16/07/2015 - V1.5 - Handle natively GSM WAV (thanks to Michael Munger)
# 01/08/2020 - V2.0 - Modded for revised IBM Watson URLs and Docker tiredofit/docker-freepbx
# 01/08/2020 - V2.1 - Further tweaks to integrate within tiredofit/docker-freepbx

# Requires
#  dos2unix
#  sed
#  awk
#  lame

# Grab functions and defaults from Docker Image
source /assets/functions/00-container
source /assets/defaults/10-freepbx

print_debug "Set Path"
# set PATH
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

print_debug "Create Temporary Directory"
# create a temporary directory and cd to it
TMPDIR=$(mktemp -d)
cd "$TMPDIR"
print debug "Now working in ${TMPDIR}"

# dump the stream to a temporary file
print_debug "Dumping Stream to Temporary File"
cat >> stream.org
print_debug "Temporary file is $(du -k "$stream.org" | cut -f 1) kilobytes
"
print_debug "Getting Boundaries"
# get the boundary
BOUNDARY=$(grep "boundary=" stream.org | cut -d'"' -f 2)

# cut the file into parts
# stream.part - header before the boundary
# stream.part1 - header after the bounday
# stream.part2 - body of the message
# stream.part3 - attachment in base64 (WAV file)
# stream.part4 - footer of the message
print_debug "Cutting file into parts"
awk '/'$BOUNDARY'/{i++}{print > "stream.part"i}' stream.org

# if mail is having no audio attachment (plain text)
PLAINTEXT=$(cat stream.part1 | grep 'plain')
if [ "$PLAINTEXT" != "" ] ; then
print_debug "File has no audio attachment"

  # prepare to send the original stream
  cat stream.org > stream.new
print_debug "Sending to new Stream"
# else, if mail is having audio attachment
else
print_debug "File has audio attachment"
  # cut the attachment into parts
  # stream.part3.head - header of attachment
  # stream.part3.wav.base64 - wav file of attachment (encoded base64)
  print_debug "Cutting Attachment into parts"
  sed '7,$d' stream.part3 > stream.part3.wav.head
  sed '1,6d' stream.part3 > stream.part3.wav.base64

  # convert the base64 file to a wav file
  print_debug "Converting Base64 information into WAV file"
  dos2unix -o stream.part3.wav.base64
  base64 -di stream.part3.wav.base64 > stream.part3.wav

  # convert wave file (GSM encoded or not) to PCM wav file
  print_debug "Convert WAV file to PCM"
  sox stream.part3.wav stream.part3-pcm.wav

  if var_true "$ENABLE_VM_TRANSCRIBE" ; then
        print_debug "Voicemail Transcription Enabled"
        CURL_OPTS=""
        # Do not change the API_USERNAME
        API_USERNAME="apikey"
        # Set the API_PASSWORD to relevant $ENV variable
        # IBM Watson PAYG account with speech transcription enabled - is required
        # More info at https://www.ibm.com/uk-en/cloud/watson-speech-to-text
        # See also http://nerdvittles.com/?page_id=25616 for more info on how to setup the IBM account
        API_PASSWORD="$VM_TRANSCRIBE_APIKEY"
        # Set the voice model to a valid IBM model
        # All models available via https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models
        API_MODEL=$VM_TRANSCRIBE_MODEL
        print_debug "Using API_MODEL ${VM_TRANSCRIBE_MODEL}"

        print_debug "Sending Payload to Watson"
        curl -s "$CURL_OPTS" -k -u $API_USERNAME:$API_PASSWORD -X POST \
            --limit-rate 40000 \
            --header "Content-Type: audio/wav" \
            --data-binary @stream.part3.wav \
            "https://stream.watsonplatform.net/speech-to-text/api/v1/recognize?continuous=true&model=$VM_TRANSCRIBE_MODEL" 1>audio.txt

        print_debug "Extracting Transcript Results"
        # Extract transcript results from JSON response
        TRANSCRIPT=$(cat audio.txt | grep transcript | sed 's#^.*"transcript": "##g' | sed 's# "$##g')
  fi

  # convert PCM wav file to mp3 file
  # -b 24 is using CBR, giving better compatibility on smartphones (you can use -b 32 to increase quality)
  # -V 2 is using VBR, a good compromise between quality and size for voice audio files
  print_debug "Converting WAV to MP3"
  lame -m m -b 24 stream.part3-pcm.wav stream.part3.mp3

  # convert back mp3 to base64 file
  print_debug "Converting MP3 to Base64"
  base64 stream.part3.mp3 > stream.part3.mp3.base64

  # generate the new mp3 attachment header
  # change Type: audio/x-wav or audio/x-WAV to Type: audio/mpeg
  # change name="msg----.wav" or name="msg----.WAV" to name="msg----.mp3"
  print_debug "Creating Mp3 Attachment Header"
  sed 's/x-[wW][aA][vV]/mpeg/g' stream.part3.wav.head | sed 's/.[wW][aA][vV]/.mp3/g' > stream.part3.mp3.head

  # generate first part of mail body, converting it to LF only
  print_debug "Generate 1st part of mail body"
  mv stream.part stream.new
  cat stream.part1 >> stream.new
  cat stream.part2 >> stream.new

  if var_true "${VM_TRANSCRIBE}" ; then
        print_debug "Showing Transcribe Details"
        echo "--- Automated transcription result ---" >> stream.new
        echo "$TRANSCRIPT" >> stream.new
  fi

  print_debug "Generate third part of mail body"
  cat stream.part3.mp3.head >> stream.new
  dos2unix -o stream.new

  print_debug "Appending Base64 MP3 to mail body"
  # append base64 mp3 to mail body, keeping CRLF
  unix2dos -o stream.part3.mp3.base64
  cat stream.part3.mp3.base64 >> stream.new

  # append end of mail body, converting it to LF only
  print_debug "Append end of mail body"
  echo "" >> stream.tmp
  echo "" >> stream.tmp
  cat stream.part4 >> stream.tmp
  dos2unix -o stream.tmp
  cat stream.tmp >> stream.new
fi

# send the mail thru sendmail
print_debug "Sending stream via sendmail"
cat stream.new | sendmail -t

# remove all temporary files and temporary directory
print_debug "Cleaning up"
rm -rf "$TMPDIR"

This is a dual issue :)
Notwithstanding the permission/quote issue - my docker had lost the VM_ENABLE_TRANSCRIBE option....restarting...

OK. let me know. I'm going to bypass Docker Hub auto builds and build my own copy and can push new versions immediately to get you working with both SMTP_FROM and the transcribe script and FOP2.

Try ENABLE_VM_TRANSCRIBE for your env var.

That one was my bad - changed.
Found another issue in the script
The first TRANSCRIBE check is:
 if var_true "$ENABLE_VM_TRANSCRIBE" ; then
but lower down, there is
if var_true "${VM_TRANSCRIBE}" ; then
        print_debug "Showing Transcribe Details"
        echo "--- Automated transcription result ---" >> stream.new
        echo "$TRANSCRIPT" >> stream.new
  fi

That does the append and has no matching value :)

Ack I missed it. Sorry about that.

I'm building a new version and will let you know when it is in place.
Fixed -
Script based on our discussions
SMTP_FROM
FOP2 startup

Coolio - and no worries, not your script originally, and it's been pinned
down :)
I'm running the Chrome FOP plugin so that'a a handy canary if FOP doesn't
start as it has a scrolling error text that vanishes once it signs on
On Sat, 15 Aug 2020 at 15:29, Dave Conroy<EMAIL_ADDRESS>wrote:

I'm building a new version and will let you know when it is in place.
Fixed -
Script based on our discussions
SMTP_FROM
FOP2 startup
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/tiredofit/docker-freepbx/issues/146#issuecomment-674405161,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAPGIY7Y3EKPZYN2FF7OOS3SA2LVRANCNFSM4N2UEWUA
.

My locally built copy uploading now to Docker Hub as tiredofit/freepbx:15-latest
Docker Hub should follow with it's own tagged images of version 5.0.6.


Merge existing Lambda Function Configurations when adding notifications to access logs S3 bucket

Is your feature request related to a problem? Please describe.
When using an existing S3 bucket for Access logs, and it already contains Lambda Function notifications, these are removed by the CustomResource and reset to and empty array before adding in the new ones for this solution.
Describe the feature you'd like
Existing LambdaFunctionConfigurations configured outside this solution are not removed when configuring the access log S3 bucket notifications
Additional context
Refer to the custom_resource.py > add_s3_bucket_lambda_event() for where the existing config is lost.  Whereas for other configuration types (e.g. TopicConfigurations), they are preserved

Thanks for reporting the issue. This has been added to our backlog for future releases.

S3 event notifications only allows one suffix like LambdaFunctionConfiguration for overlapping prefix for the same event type. So it is by design to remove all existing LambdaFunctionConfigurations before adding the new ones used by the solution.


collection of tagged union bug fixed.

Fixes the bug when a field of a tagged union collection type:
class FloatSubModel(BaseXmlModel):
    type: Literal['float'] = attr()
    data: float

class StringSubModel(BaseXmlModel):
    type: Literal['string'] = attr()
    data: str

TaggedUnion = Annotated[Union[FloatData, StringData], Field(discriminator='type')]

class Model(BaseXmlModel):
    collection: list[TaggedUnion]

Fixes the issue https://github.com/dapper91/pydantic-xml/issues/206

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 91.89%. Comparing base (3b81aeb) to head (a62b518).

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##              dev     #207      +/-   ##
==========================================
+ Coverage   91.83%   91.89%   +0.06%
==========================================
  Files          29       29
  Lines        1617     1617
==========================================
+ Hits         1485     1486       +1
+ Misses        132      131       -1

Flag
Coverage Δ

unittests
91.89% <ø> (+0.06%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.