SPIRV legalization fails for RW texture reads in HLSL

Hello!
I have the following trivial compute shader that I compile with glslang (https://github.com/KhronosGroup/glslang/commit/386b4fcb28e5cc2987c26c1250f0ba8200eed8fa) and then optimize and legalize with SPIRV-Tools (https://github.com/KhronosGroup/SPIRV-Tools/commit/f0c96f40c79d03e42688aae48642a4759a5e376f):
RWTexture2D<float>Tex2D;

[numthreads(8, 8, 1)]
void TestCS()
{
    Tex2D[int2(0,0)] = Tex2D[int2(0,0)] + 1.0; // Legalization fails
}

I use spvtools::Optimizer as follows:
spvtools::Optimizer SpirvOptimizer(SPV_ENV_VULKAN_1_0);
SpirvOptimizer.RegisterLegalizationPasses();
std::vector<uint32_t> LegalizedSPIRV;
SpirvOptimizer.Run(SPIRV.data(), SPIRV.size(), &LegalizedSPIRV);

but the optimizer fails. If I replace [] with Load in the right side of the assignment, the optimizer succeeds:
    Tex2D[int2(0,0)] = Tex2D.Load(int2(0,0)) + 1.0; // OK

If I use 4-component texture, the optimizer also succeeds with []:
RWTexture2D<float4>Tex2D; // Also OK

[numthreads(8, 8, 1)]
void TestCS()
{
    Tex2D[int2(0,0)] = Tex2D[int2(0,0)] + float4(1, 1, 1, 1);
}

The validation fails here, but is there such a limitation?
Here is the failing SPIRV:
const uint32_t spirv_fail[] = {
    119734787,65536,524298,25,0,131089,1,393227,1,1280527431,1685353262,808793134,0,196622,0,1,327695,
    5,4,1953719636,21315,393232,4,17,8,8,1,196611,5,500,262149,4,1953719636,21315,327685,6,1936020544,
    676545396,0,327685,10,1919906931,1835357285,112,262149,13,846751060,68,262215,13,34,0,262215,13,
    33,0,131091,2,196641,3,2,196630,8,32,262176,9,7,8,589849,11,8,1,0,0,0,2,3,262176,12,0,11,262203,12,
    13,0,262165,15,32,1,262167,16,15,2,262187,15,17,0,327724,16,18,17,17,262187,8,20,1065353216,327734,
    2,4,0,3,131320,5,262201,2,24,6,65789,65592,327734,2,6,0,3,131320,7,262203,9,10,7,262205,11,14,13,327778,
    8,19,14,18,327809,8,21,19,20,196670,10,21,262205,11,22,13,262205,8,23,10,262243,22,18,23,65789,65592
};

And here is the one that was compiled with Load and works OK:
const uint32_t spirv_ok[] = {
    119734787,65536,524298,27,0,131089,1,393227,1,1280527431,1685353262,808793134,0,196622,0,
    1,327695,5,4,1953719636,21315,393232,4,17,8,8,1,196611,5,500,262149,4,1953719636,21315,327685,
    6,1936020544,676545396,0,327685,10,1919906931,1835357285,112,262149,13,846751060,68,262215,13,34,
    0,262215,13,33,0,131091,2,196641,3,2,196630,8,32,262176,9,7,8,589849,11,8,1,0,0,0,2,3,262176,12,0,
    11,262203,12,13,0,262165,15,32,1,262167,16,15,2,262187,15,17,0,327724,16,18,17,17,262167,19,8,4,262187,
    8,22,1065353216,327734,2,4,0,3,131320,5,262201,2,26,6,65789,65592,327734,2,6,0,3,131320,7,262203,9,10,7,
    262205,11,14,13,327778,19,20,14,18,327761,8,21,20,0,327809,8,23,21,22,196670,10,23,262205,11,24,13,
    262205,8,25,10,262243,24,18,25,65789,65592
};

@TheMostDiligent Could you please provide the spir-v assembly instead of an array of integers?  In order to reproduce, I'll have to write an application instead of just using the command line tools.  It also makes it harder for others looking at the issue to know if they might be hitting the same problem.

@s-perron I was thinking it would actually be easier for you to reproduce, because you can pass this data directly to SpirvOptimizer.Run.
Here is the assembly for the Tex2D[int2(0,0)] version:
// Module Version 10000
// Generated by (magic number): 8000a
// Id's are bound by 31

                              Capability Shader
               1:             ExtInstImport  "GLSL.std.450"
                              MemoryModel Logical GLSL450
                              EntryPoint GLCompute 4  "main"
                              ExecutionMode 4 LocalSize 8 8 1
                              Source HLSL 500
                              Name 4  "main"
                              Name 13  "Tex2D"
                              Decorate 13(Tex2D) DescriptorSet 0
                              Decorate 13(Tex2D) Binding 0
               2:             TypeVoid
               3:             TypeFunction 2
               8:             TypeFloat 32
              11:             TypeImage 8(float) 2D nonsampled format:R32f
              12:             TypePointer UniformConstant 11
       13(Tex2D):     12(ptr) Variable UniformConstant
              15:             TypeInt 32 1
              16:             TypeVector 15(int) 2
              17:     15(int) Constant 0
              18:   16(ivec2) ConstantComposite 17 17
              20:    8(float) Constant<PHONE_NUMBER>
         4(main):           2 Function None 3
               5:             Label
              27:          11 Load 13(Tex2D)
              28:    8(float) ImageRead 27 18
              29:    8(float) FAdd 28 20
              30:          11 Load 13(Tex2D)
                              ImageWrite 30 18 29
                              Return
                              FunctionEnd

Here is the assembly for the Tex2D.Load(int2(0,0)) version:
// Module Version 10000
// Generated by (magic number): 8000a
// Id's are bound by 34

                              Capability Shader
               1:             ExtInstImport  "GLSL.std.450"
                              MemoryModel Logical GLSL450
                              EntryPoint GLCompute 4  "main"
                              ExecutionMode 4 LocalSize 8 8 1
                              Source HLSL 500
                              Name 4  "main"
                              Name 13  "Tex2D"
                              Decorate 13(Tex2D) DescriptorSet 0
                              Decorate 13(Tex2D) Binding 0
               2:             TypeVoid
               3:             TypeFunction 2
               8:             TypeFloat 32
              11:             TypeImage 8(float) 2D nonsampled format:R32f
              12:             TypePointer UniformConstant 11
       13(Tex2D):     12(ptr) Variable UniformConstant
              15:             TypeInt 32 1
              16:             TypeVector 15(int) 2
              17:     15(int) Constant 0
              18:   16(ivec2) ConstantComposite 17 17
              19:             TypeVector 8(float) 4
              22:    8(float) Constant<PHONE_NUMBER>
         4(main):           2 Function None 3
               5:             Label
              29:          11 Load 13(Tex2D)
              30:   19(fvec4) ImageRead 29 18
              31:    8(float) CompositeExtract 30 0
              32:    8(float) FAdd 31 22
              33:          11 Load 13(Tex2D)
                              ImageWrite 33 18 32
                              Return
                              FunctionEnd

Like I mentioned, the SPIRV validation fails here for the [] version.

I was able to reproduce the issue using the command line tools.  Note that if the target environment is set to spv1.0 then the validation step passes.  This is a valid check for the Vulkan 1.0 environment.  See https://www.khronos.org/registry/vulkan/specs/1.0/html/vkspec.html#spirvenv-module-validation-runtime where it says "The Result Type operand of OpImageRead must be a vector of four components."
This is not the type of things that the legalization pass tries or can handle.  It does not try to change the type of images.
It passes validation when the Tex2D.Load is used because glslang set the result type of the OpImageRead to a vec4 instead of just a float.  I don't think there is anything that can be done in SPIR-V tools.  We should probably ask why glslang uses different result types for Tex2D.Load and Tex2D[]?
stevenperron@stevenperron:issue/4138 $ cat t.comp.hlsl
RWTexture2D<float>Tex2D;

[numthreads(8, 8, 1)]
void TestCS()
{
    Tex2D[int2(0,0)] = Tex2D[int2(0,0)] + 1.0; // Legalization fails
}
stevenperron@stevenperron:issue/4138 $ glslangValidator -e TestCS t.comp.hlsl -V -o t.spv --target-env vulkan1.0
t.comp.hlsl
stevenperron@stevenperron:issue/4138 $ spirv-dis t.spv
; SPIR-V
; Version: 1.0
; Generator: Khronos Glslang Reference Front End; 7
; Bound: 30
; Schema: 0
               OpCapability Shader
          %1 = OpExtInstImport "GLSL.std.450"
               OpMemoryModel Logical GLSL450
               OpEntryPoint GLCompute %TestCS "TestCS"
               OpExecutionMode %TestCS LocalSize 8 8 1
               OpSource HLSL 500
               OpName %TestCS "TestCS"
               OpName %Tex2D "Tex2D"
               OpDecorate %Tex2D DescriptorSet 0
               OpDecorate %Tex2D Binding 0
       %void = OpTypeVoid
          %3 = OpTypeFunction %void
      %float = OpTypeFloat 32
         %11 = OpTypeImage %float 2D 0 0 0 2 R32f
%_ptr_UniformConstant_11 = OpTypePointer UniformConstant %11
      %Tex2D = OpVariable %_ptr_UniformConstant_11 UniformConstant
        %int = OpTypeInt 32 1
      %v2int = OpTypeVector %int 2
      %int_0 = OpConstant %int 0
         %18 = OpConstantComposite %v2int %int_0 %int_0
    %float_1 = OpConstant %float 1
     %TestCS = OpFunction %void None %3
          %5 = OpLabel
         %26 = OpLoad %11 %Tex2D
         %27 = OpImageRead %float %26 %18
         %28 = OpFAdd %float %27 %float_1
         %29 = OpLoad %11 %Tex2D
               OpImageWrite %29 %18 %28
               OpReturn
               OpFunctionEnd
stevenperron@stevenperron:issue/4138 $ spirv-val t.spv --target-env vulkan1.0
error: line 25: Expected Result Type to have 4 components
  %27 = OpImageRead %float %26 %18

@s-perron Thanks for looking into this, everything is clear now.


Remove alamoRequest.cancel() from stubbing closure

Fixes #1833.
The request was never really resumed and so it most likely doesn't need to be cancelled. This was only making random noise about cancelled request in console.
Targeted master as I'd like to make a 13.0.1 release with this fix.

SwiftLint is having problems on newest Xcode/macOS. Wonder if we should try to build it from source or just disable it until it's released as a new version on brew. 🤔

          2 Warnings

:warning:
Any changes to library code should be reflected in the Changelog. Please consider adding a note there and adhere to the Changelog Guidelines.

:warning:
The library files were changed, but the tests remained unmodified. Consider updating or adding to the tests to match the library changes.

          3 Messages

:book:
iOS: Executed 278 tests, with 0 failures (0 unexpected) in 13.330 (13.585) seconds

:book:
tvOS: Executed 278 tests, with 0 failures (0 unexpected) in 16.597 (16.769) seconds

:book:
macOS: Executed 278 tests, with 0 failures (0 unexpected) in 13.414 (13.538) seconds

  Generated by :no_entry_sign: Danger

Codecov Report

Merging #1841 into master will decrease coverage by 0.08%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           master   #1841      +/-   ##
=========================================
- Coverage   93.48%   93.4%   -0.09%
=========================================
  Files          24      25       +1
  Lines         706     864     +158
=========================================
+ Hits          660     807     +147
- Misses         46      57      +11

Impacted Files
Coverage Δ

Sources/Moya/MoyaProvider+Internal.swift
97% <ø> (+0.7%)
:arrow_up:

Sources/Moya/Plugins/NetworkLoggerPlugin.swift
93.44% <0%> (-6.56%)
:arrow_down:

Sources/ReactiveMoya/SignalProducer+Response.swift
93.93% <0%> (-6.07%)
:arrow_down:

Sources/Moya/MoyaProvider.swift
91.66% <0%> (-2.88%)
:arrow_down:

Sources/Moya/MultiTarget.swift
100% <0%> (ø)
:arrow_up:

Sources/Moya/URLRequest+Encoding.swift
100% <0%> (ø)

Sources/Moya/MultipartFormData.swift
72.41% <0%> (+7.41%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 08fb6c3...955fd23. Read the comment docs.


Feature Request - Adding text overlay

I looked on the main website and checked the docs, but I didn't see a way to have a text box overlay the rotating images.
I'd like to use slick to rotate images, but have a static text box over the images to display information.  Bootstrap friendly would be nice too; e.g. after a certain width, the text box moves from overlay to underneath the images.
Thank you for your time and attention..

You can achieve that by creating a div inside your slick slide and using the CSS property position:absolute on it

FYI:  If there's anyone else which is looking for this, I got something working.
If you put the div (which you wish to overlay) within the <div id="slider" class="slider"> then it'll actually create a new slide with your overlay text,
Yet, if you add another div layer above your slider and put both our overlay text div and your slider div within this, it'll work properly.
E,g,:
`
Overlay Text

......

`

If i do so then, the original place of that overlay text div doesn't remove from the document flow a blank space remained in the document flow.


added yaml mode to codemirror

fixes #363
useful for writing elastic queries once we support elastic2

✨ 👾 ✨

ahhh, nice.


fix: put > on a new line if the tag or last prop has trailing comments

Closes https://github.com/rescript-lang/syntax/issues/675 again...
See https://github.com/rescript-lang/syntax/issues/675#issuecomment-1279882340

@cknitt Could you try again on your codebase and see if that formatting issue is still there

Yes! The problem is now resolved. Thanks a lot for your work! 👍


Inplace editing must start by both Key_Return and Key_Enter

QtnPropertyDelegateWithValueEditor::acceptKeyPressedForInplaceEditImpl

@niello Do you have a pull request for this improvement?

Not yet. I just wrote it down not to forget)

Does this pull request https://github.com/qtinuum/QtnProperty/pull/49 do that you need?

Yes, thank you, looks exactly like what I need

Closed by #49


SAM-2466 Attachments buttons not working in samigo .

Reverted some unnecessary redirects on faces-config

So where did these redirects come from?  SAM-2378 and @csev?

The  tags are to stop the tool generating HTML in response to at POST request. If you removed the redirect tags then you start getting message in the logs from:
https://jira.sakaiproject.org/browse/SAK-27768

Hi, removing those redirect from questions item I didn't appreciate any extra log.
When I was looking for the bug I noticed that debug from SamigoJsfTool was processing the request to the  .jsp  twice instead of doing question first and filepicker later. On 10.x (or if you remove the redirect on this path in trunk) the behaviour is the correct. I presume that only affects to those few paths I changed (question items) because attachment buttons works when editing parts. I feel that in those part some HTML processing was required.

I've checked the log and It's true I see this log line:
2015-02-24 09:41:36,892  WARN http-bio-8080-exec-8 org.sakaiproject.portal.charon.handlers.SiteHandler - It is tacky to return markup on a POST CTI=sakai.samigo URL=http://localhost:8080/portal/site/ad6130f4-9859-41c6-9267-2b4b0c663b48/tool/8d271cbf-c64c-4841-83ef-f6e4d3d08b8c/jsf/author/editAssessment
By the way, I add more information here in the case that the question type is a multiplechoice :
If you enable debug on SamigoJSFTool you'll see that when push to the attachment button that dispatch method is called twice. If you have the  both  calls targets are /jsf/author/item/multipleChoiceSurvey
Instead, if you remove the <redirect/ > to the question types definitions then first points to /jsf/author/item/fileUpload and the second points to /jsf/author/item/sakai.filepicker.helper/tool
It's like the processing prevented by  is needed at some point to launch filepicker. Isn't it?

It's like the processing prevented by is needed at some point to launch filepicker. Isn't it?

Yes, I agree. What is your recommendation @sinmsinm?  Do you believe this PR should be merged?

I would merge it.  This PR has been opened for some days and nobody suggested an alternative to get attachment buttons working again.


Binaries may not expect shared libraries in /lib{,64}

All shared libraries get copied into /lib or /lib64 in the "container".
This does not work with binaries from Ubuntu, as they expect shared libraries (have rpath set) in "/lib/x86_64-linux-gnu/", even for "/bin/sh". Explicitly setting LD_LIBRARY_PATH in src/sexec.c with setenv(2) helps.

Ahh, good find! I think I have a very portable way of fixing this and will implement as soon as possible and let you know for testing.
Thanks!

I updated the ldd dependency resolver to use the default library paths and use LD_LIBRARY_PATH to update the search path. Will you recreate your SAPPs and let me know what you find?
Thank you!

Looks good!

I closed to early. When an application itself requires LD_LIBRARY_PATH set, the libraries are fetched into the sapp, but they are not found on execution. Am I missing something?

Ahhh, that is a good catch. Some variables should be caught and replicated into the SAPP. I'll think about how to do it and implement something when I get to the computer. Let me know if you think of a fix too!
Thanks!!
Sent from my iPhone

On Nov 27, 2015, at 10:33 AM, weidendo<EMAIL_ADDRESS>wrote:
I closed to early. When an application itself requires LD_LIBRARY_PATH set, the libraries are fetched into the sapp, but they are not found on execution. Am I missing something?
—
Reply to this email directly or view it on GitHub.

I just committed a method which will find all libraries in the SAPP and include them in the LD_LIBRARY_PATH. Can you check to see if that is working now? Thanks!

Hmm, no.
I compiled something with ICC, which needs
/opt/intel/composer_xe_2015.5.223/compiler/lib/intel64
to be in LD_LIBRARY_PATH.
...> singularity build heat.spec
Building: heat
Checking paths...
Checking writable...
Evaluating: %packages
Creating template configs
/etc/nsswitch.conf
/etc/resolv.conf
/etc/hosts
Evaluating: hard requirements
Installing file: /bin/sh
Evaluating: %exec
Installing file: /home/weidendo/SW/heat/heat
Evaluating: %files
Invoking dependency check
Installing file: /lib/x86_64-linux-gnu/libm.so.6
Installing file: /opt/intel/composer_xe_2015.5.223/compiler/lib/intel64/libiomp5.so
Installing file: /lib/x86_64-linux-gnu/libgcc_s.so.1
Installing file: /lib/x86_64-linux-gnu/libpthread.so.0
Installing file: /lib/x86_64-linux-gnu/libc.so.6
Installing file: /lib/x86_64-linux-gnu/libdl.so.2
Installing file: /lib64/ld-linux-x86-64.so.2
Including SAPPFILE
Calculating container size
Calculating checksums
Writing metadata
Building primary singularity exec driver
Building singularity test runscript
Building singularity main runscript
Running %test...
Cleaning up unpackaged files
Creating output SAPP container
WROTE: heat.sapp
weidendo@lapbode161:~/SW/heat (master)> ./heat.sapp
/home/weidendo/SW/heat/heat: error while loading shared libraries: libiomp5.so: cannot open shared object file: No such file or directory
What is your way to debug this?

Looking into ~/.sapp-cache/heat/c/singularity:
...
LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/lib/x86_64-linux-gnu
/opt/intel/composer_xe_2015.5.223/compiler/lib/intel64"
...
Newline instead of colon?

Woops... Sorry about that! I just committed a fix that will work... Will test as soon as I get into work.

The fix seems to be working for me, but I don't have a use-case on this system with multiple libdirs. Can you confirm the latest commits are working better?
Thanks!

Works fine here.

Excellent, thanks!


Move deband to end of tonemapping.

https://github.com/godotengine/godot/issues/66316
This avoids artifacts when using adjustments and color correction
Dither was incorrectly added before the tonemapping step (and thus before the adjustments and color correction). Dither should be applied right before quantization (i.e. before writing to the LDR framebuffer). Otherwise the debanding effect becomes stronger or weaker depending on the scene brightness and the adjustments/color correction steps.
As a bonus this also increases the quality of the debanding (but only very slightly).
Before:

After:

Without the extreme adjustments:
Before:

After (if you zoom in you will notice a very small increase in quality here):

This change is not relevant for GLES3 in 4.0, but is relevant for 3.x. It won't be able to be cherrypicked but it is a simple fix

I have noticed an issue with Debanding in some lighting conditions that aren't present with Debanding Off:
I don't think much can be done, as this is an optical illusion. (there is seemingly movement with Debanding even on a still image)
Debanding On

Debanding Off

(there is seemingly movement with Debanding even on a still image)

Are you using a monitor on a VGA connection (with or without an adapter), or a monitor with a 6-bit + FRC panel (which can't display 8-bit color without dithering on its own)?

using DVI on a ViewSonic VX2433wm, which is 1080p 60Hz LCD

using DVI on a ViewSonic VX2433wm, which is 1080p 60Hz LCD (which is a cheap, nearly 15 years old TN panel)

This is indeed a monitor with 6-bit + FRC color:

Colour Support: | 16.7M (6-bits + Hi FRC)

Thanks!


ENV vars and cloud.yaml keys for AK/SK are not standardized

Hi,
currently the python-otcextensions requires

in clouds.yaml ak and sk keys
in env variables S3_ACCESS_KEY_ID and S3_SECRET_ACCESS_KEY
for interacting with various otc resources.

This is missleading from multiple perspectives:

the AK/SK pair is not limited to S3 or OBS
AWS uses AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY so OS_ACCESS_KEY_ID or OS_SECRET_ACCESS_KEY could be considered
terraforms opentelekomcloud resource provider uses OS_ACCESS_KEY and OS_SECRET_KEY from the env and access_key and secret_key inside the provider - imho the most elegant solution which could as well be used for the clouds.yaml

To avoid breaking backwards compatiblity I would recommend evaluating the terraform variables too and deprecate the S3_* env vars as well as the ak and sk inside clouds.yaml (issuing a warning), to remove it later.
What's your opinion on that?
EDIT: For some reason it looks like this statement already pulls OS_ACCESS_KEY from the env - no idea why.

This was implemented historically as "backward-compatibility" for the old otc.sh tool. We might consider now dropping it, but likely better to support both. Actually terraform is not an indicator for us. On the other hand AK/SK are currently implemented in OTCE only for OBS, and not for other services.

@gtema
You have already merged a PR. Is this topic still relevant?


Update Composer dependencies (2021-10-15-00-11)

Visual regression test passed!
View the visual regression test report


Coalesce adjacent stream.async.fill ops.

Seen commonly from concats with constants and padding:
    %1124 = stream.async.alloca : !stream.resource<*>{%c20}
    %1125 = stream.async.update %1120, %1124[%c0 to %c4] : !stream.resource<*>{%c4} -> %1124 as !stream.resource<*>{%c20}
    %1126 = stream.async.fill %c0_i8, %1125[%c4 to %c8 for %c4] : i8 -> %1125 as !stream.resource<*>{%c20}
    %1127 = stream.async.fill %c0_i8, %1126[%c8 to %c12 for %c4] : i8 -> %1126 as !stream.resource<*>{%c20}
    %1128 = stream.async.fill %c0_i8, %1127[%c12 to %c16 for %c4] : i8 -> %1127 as !stream.resource<*>{%c20}
    %1129 = stream.async.fill %c0_i8, %1128[%c16 to %c20 for %c4] : i8 -> %1128 as !stream.resource<*>{%c20}

The adjacent fills could be coalesced into a single fill:
    %1124 = stream.async.alloca : !stream.resource<*>{%c20}
    %1125 = stream.async.update %1120, %1124[%c0 to %c4] : !stream.resource<*>{%c4} -> %1124 as !stream.resource<*>{%c20}
    %1126 = stream.async.fill %c0_i8, %1125[%c4 to %c20 for %c16] : i8 -> %1125 as !stream.resource<*>{%c20}

The offset/end/lengths allow for this to be done even in dynamic shape cases. stream.async.copy would be the other case we could coalesce but those are rarer (most copies are introduced as stream.cmd.copy from transfer operations).
Progress on #13637.

Another simple one to fix:
    %1567 = stream.async.splat %c0_i8 : i8 -> !stream.resource<*>{%c33554432}
    %1568 = stream.async.fill %c0_i8, %1567[%c0 to %c16777216 for %c16777216] : i8 -> %1567 as !stream.resource<*>{%c33554432}

Even sillier:
    %1567 = stream.async.splat %c0_i8 : i8 -> !stream.resource<*>{%c33554432}
    %1568 = stream.async.fill %c0_i8, %1567[%c0 to %c16777216 for %c16777216] : i8 -> %1567 as !stream.resource<*>{%c33554432}
    %1569 = stream.async.slice %1568[%c16777216 to %c33554432] : !stream.resource<*>{%c33554432} -> !stream.resource<*>{%c16777216}

that slice is of a splat and should just be a smaller splat


[EDIT] I1.1 example's LaTeX imaginary unit is inconsistent

Node number
Node I.1, Exercise I.1.1
Current content
The imaginary part of |0> is using "i" as a symbol for an imaginary unit. But in |1>, it is a "j".

Expected content
Both |0> and |1> should use "i" to avoid confusion.

[X] I have searched existing GitHub issues to make sure the issue does not already exist.

@rootEnginear good catch, thank you! I've made the correction on our end, and it will be pushed to the live site in our next round of content updates :slightly_smiling_face:


CZI proposal on accessibility: steering council approval

Hey y'all
I am opening this issue as @isabela-pf @tonyfast and I are working on a CZI EOSS proposal on accessibility for Jupyter
See issue: https://github.com/jupyter/accessibility/issues/36
Submitted letter of intent: https://github.com/jupyter/accessibility/pull/44
I know there is no formal process (yet) to apply for funding but there are a couple of discussions happening in #96  and #97 hence I would like to request formal approval from the steering council to move forward with this work
Let me know if there is any other information I'd need to provide

I Approved both.
disclosure: I work at Quansight with @trallard.

Enthusiastic support from me!

I haven't been involved with the accessibility work for a while, but I'm delighted to see forward motion on this!
Just to note, as I looked around the relevant materials I saw you posted the proposal draft (thx for doing that, just linking it here as it's relevant for anyone giving feedback/input).
I'm enthusiastically in support of this work - I trust you've all coordinated on the accessibility repo so that all the relevant folks are in the loop.  Specifically, from an (admittedly quick) reading of the proposal draft, I see that in addition to documentation and overall assessment of accessibility issues, you have some technical aims on a few specific projects - the pydata sphinx theme, jupyterbook and jupyterlab. I think it would be valuable to have feedback from some of those teams that they're +1 on the proposed ideas/plan to ensure good overall coordination.
But assuming that such coordination is in place (at a high-level, obviously, given it's early stages and not even funded yet), a very enthusiastic +1 from me, combined with gratitude that you're tackling this critical area of the project!

ps - I also sent a quick reminder to the steering council list so we get as much feedback as possible. We're doing our best to not have private discussions in the SC list so everyone participates here, but a little "heads up" email to that list is often useful to remind folks to come back here to the public spaces :)

I'm enthusiastically in support of this work - I trust you've all coordinated on the accessibility repo so that all the relevant folks are in the loop

@fperez They certainly have :smile: https://github.com/jupyter/accessibility/pull/44
+1 from me.

I support this effort, thank you for coordinating it!

@fperez Can this be closed now that the proposal has been submitted.

Can this be closed now that the proposal has been submitted.

I am not @fperez but I think we can close it now :wink:


Create DCO.yml

Trying the workaround for docapp being down

The temp action fix for DCO check will be removed when doc app is fixed


Feature: Replay blocks

When the client has been offline, replaying all blocks mined since is necessary to handle events emitted in smart contracts the application is subscribed to.

Might only be relevant in combination with the web3j connector, since EthereumJ is ran internally, so might not have these issues (?)

It is a good point. But it needs to be configurable. It is interesting to imagine being able to say "I want from this block number" and everything is being replayed. Is this what you had in mind or something else? And yes ethj does it automatically

Yeah, programmatically start replaying blocks/events from a certain block number to prevent missed events in an external application, or just to rebuild/verify its local database.

I wouldn't implement that for ethj but more for web3j only.
The reason I don't want to do that as a generic solution is:

Ethereumj doesn't really need that because it plays every blocks when it syncs after being offline
Because it wouldn't work for Test blockchain

I propose to do that in the EthereumRpcConfig where you can say that you want to read from a particular block number
config.fromBlock(12345667) // by default, from latest
and this will start reading the block at startup

Tend to agree with that, only reason I can think of why you would want to replay blocks in EthJ is when you subscribe to events thrown from a contract and you want to build your current state database with events thrown from the start of the contract, before you were listening.

do you think this is still relevant ?

Currently, instead of replaying blocks, I am pulling states directly from the smart contract, which is probably a lot more efficient than querying and replaying tens/hundreds of thousands of blocks.
As for thrown events, they are stored in the smart contract itself after processing and can be queried using web3. Currently, we are testing an implementation doing this via Propeller, so we can synchronize our local state via events as well.
So as far as our application goes, no.

yes def. Ok So I'm closing that for now. we can always re-open if needed


ability to add additional parameters to the outbound syslog messages

Right now, a file is dumped /etc/rsyslog.d/25-syslog-release-forwarding-setup.conf which contains a custom template that conforms to the specific syslog format this package uses.. However, this makes things difficult to modify. We would like to be able to add our own fields (e.g. we have multiple different bosh clusters, and adding env="dev" or env="prod" makes our lives 1000x easier, than having to try and figure out which deployment is which, etc. We could try and create our own template, but that is very cumbersome to manage.
Can we add a field in the yaml to define some additional attributes which get tacked on via templating? You'd of course want to restrict the key to something like... [a-zA-Z_]{2,20}, and the value to alphanumeric, with spaces, maybe? Something that doesn't collide with the existing metacharacters.

There is a currently-configurable director field for this purpose. It can be set using the syslog.director property. Does this do what you need?
As for the larger question of self-service fields, the feature you're suggesting sets off a lot of complexity alarms to me; I'd want to accumulate a fair amount of evidence for use-cases before even inviting PRs for something like that. It's also the kind of feature that tends to mask missing features and reduce feedback, since it "can" "solve" so many "problems" and there's no way to close the loop on what people end up needing it for.

I'm not at my computer, so I'm sorry if my response is lacking. Adding
additional "fields" is a very common supported feature of any log-sending
utility. This includes rsyslog itself (e.g. json). Custom fields are
supported by things like the splunk forwarder, and filebeat (ELK stack). It
seems like a reasonable request to want to include metadata, especially
when you want to go cloud, where the source machines are generated and not
pre-setup. Knowing what datacenter, AZ, vlan (or some form of identifiable
zone), environment, what automated utility deployed the given config,
software version something is running with, using a different field than
"program" because sometimes that's not accurate, if software deploys are on
a per-team basis, know what team it's assigned to, etc. There are many
reasons why we would benefit from this across our 60 (and growing) bosh
deployments.
Putting in the bosh director hostname does not provide things like the
environent, the best thing it provides is a best-guess (and if you only
have an IP, it's useless) -- this is bad design in my opinion.
You may say, well, if you want those things, put them into the log messages
and parse them out yourself. Unfortunately, that's a flawed answer, because
you may want logs from something you didn't create, and having to
implemement this metadata into every single piece of software seems very
inefficient.
On Wed, Feb 27, 2019, 6:39 PM Jesse Thomas Alford<EMAIL_ADDRESS>wrote:

There is a currently-configurable director field for this purpose. It can
be set using the syslog.director
https://github.com/cloudfoundry/syslog-release/blob/1ad449483bd8d66b57f5ed1944b697d6daa9d62b/jobs/syslog_forwarder/spec#L27
property. Does this do what you need?
As for the larger question of self-service fields, the feature you're
suggesting sets off a lot of complexity alarms to me; I'd want to
accumulate a fair amount of evidence for use-cases before even inviting PRs
for something like that. It's also the kind of feature that tends to mask
missing features and reduce feedback, since it "can" "solve" so many
"problems" and there's no way to close the loop on what people end up
needing it for.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/cloudfoundry/syslog-release/issues/54#issuecomment-468075235,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABwwRVbpWKgX_U8HUl9ut8NCR5ke0M4zks5vRxcogaJpZM4bDQup
.


Update JAVA and Gradle Plugin Versions for TIKI SDK Android Build Process

The JAVA and Gradle plugin versions used in the TIKI SDK Android build process may not be up-to-date. It is important to ensure that the TIKI SDK Android is built using the latest compatible JAVA version and Gradle plugin version to take advantage of performance improvements, security updates, and bug fixes. This includes updating the JAVA and Gradle plugin versions not only in the projects, but also in the GitHub Actions workflow for continuous integration and deployment.
Additionally, backward compatibility checks should be performed to ensure that the TIKI SDK Android remains compatible with older versions of JAVA and Gradle plugins. Flutter build aar process recommends Java 11, for example, and other libs might enforce specific versions.

[x] Review the current JAVA and Gradle plugin versions used in the TIKI SDK Android build process.
[x] Research and determine the latest compatible JAVA and Gradle plugin versions for the build process.
[x] Perform backward compatibility checks to ensure that the updated versions are compatible with older versions of JAVA and Gradle plugins.
[x] Update the JAVA and Gradle plugin versions in the TIKI SDK Android projects, including the build.gradle files and other relevant configuration files.
[x] Update the GitHub Actions workflow to reflect the changes in the JAVA and Gradle plugin versions for continuous integration and deployment.

Gradle Plugin 7.3.3 supports Java 17. Updated the actions to use Java 17 in #168


fix(package.json): updated

packages

:tada: This PR is included in version 2.0.19 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


feat(PaginationV2): introduce itemsPerPageFollowsText prop

It works as itemsPerPageText and colon combined, that allows consumer to translate the combination.
Fixes #835.
Changelog
New

itemsPerPageFollowsText  prop

Changed

PaginationV2-test.js that used to test older <Pagination> - Changed to actually test <PaginationV2>

@joshblack Sure, done! https://github.com/carbon-design-system/carbon-components-react/pull/842/commits/2ba7c77461ed71cdbc70e74fdf6c953adb8770d7


Razer Firefly not working

For some reason, after going to v0.6.0b I cannot get the Razer firefly to show up in Aurora. Not sure if I did something, or it is an issue with this version.

Have you set a "Preferred mouse" in the settings? That should make it work.

Thanks! Didn't think that would work as there was no designated option for the Firefly


Update SCE contents and processing with cell type information

Closes #160
This PR makes some small adjustments to docs to sync up what we say about cell typing here to where the cell types actually live in the object.

I added a link for both SingleR and CellAssign papers to the processing information section on cell type annotation.
In that same section, I added a note about submitter annotations.
I included the submitter_annotation column in the cell metrics section. When adding this in, I chose to add this in the table that mentions items that are in both filtered and processed objects. I think this is the right place since if they are available they will be in all of the objects.
I standardized how we refer to the cellassign_reference and singler_reference. This matches changes made in https://github.com/AlexsLemonade/scpca-nf/pull/510 and https://github.com/AlexsLemonade/scpca-nf/pull/503.

Please let me know if you think I missed any other areas here. I will update the download section separately as part of #159 and the getting started section as part of #165.

@allyhawkins FYI I'm going to get to this today but realize I forgot it in my scrum!

Thanks for taking a look at this @sjspielman. I incorporated your comments and then also filed https://github.com/AlexsLemonade/scpca-nf/pull/522 to return the label name. I then updated the docs here to reflect that.

for posterity noting that singler_reference_label metadata item referred to here matches the name used in https://github.com/AlexsLemonade/scpca-nf/pull/522 👍


Arbitrary LocalDateTimes

First draft for #1776
This pull requests adds a new generator for LocalDateTimes which is used by Arbitrary.

Current coverage is 97.10% (diff: 100%)

Merging #1777 into master will increase coverage by <.01%

@@             master      #1777   diff @@
==========================================
  Files            87         87
  Lines         11195      11207    +12
  Methods           0          0
  Messages          0          0
  Branches       1862       1863     +1
==========================================
+ Hits          10870      10882    +12
  Misses          203        203
  Partials        122        122

Powered by Codecov. Last update 6b646a7...687f78d

It was a bit late yesterday and I had some craft beers :smile:. Sorry for not cleaning before pushing. I hope this is fine now.

I had some craft beers 😄

beer only is ok, and always remember: wine after beer is fine ;-p
Looks great to me! Waiting now for the CI server, then will pull it in.

Don't push it yet. I accidently removed some empty lines in Arbitrary

Ok, will wait!

Looks awesome, thank you!


Fix abv and prefix_linked?

CC @tdsmith @MikeMcQuaid
Tim suggested we may want to realpath all pathnames before computing
abv, but in case that's not always universally desirable, this is a
slightly less invasive approach.
I also noticed prefix_linked? was returning lies, so there's a fix for
that as well.
Fixes https://github.com/Homebrew/brew/issues/1791.

Thanks for the PR @ilovezfs 🎉

Tim suggested we may want to realpath all pathnames before computing
abv, but in case that's not always universally desirable, this is a
slightly less invasive approach.

I think in this case I agree with @tdsmith given it's a mixin module we're providing we do want this functionality. I've made this and some other tweaks and pushed to this branch.

LGTM


[OP] Fix SequenceMask to support different datatypes for data / sequence_length

Currently, the SequenceMask operator does not support different dtypes for data and sequence_length. Usually, we could set data.dtype = np.float32 and sequence_length.dtype = np.int32. However, it's not allowed due to this line: https://github.com/apache/incubator-mxnet/blob/master/src/operator/sequence_mask-inl.h#L220
We should fix it.

Hi @sxjscience . I agree this is an annoying issue now, causing you to cast your sequence length tensor to be the same type of your data tensor. I fixed this back in January in this PR: https://github.com/apache/incubator-mxnet/pull/13880
It didn't get merged in time to make it into MXNet 1.4, but it will be in MXNet 1.5 when that release happens. So hang tight :)
Best,
Stephen


Added Multi-Language Support and User Authentication

Pull Request for PyVerse 💡
Requesting to submit a pull request to the PyVerse repository.

Issue Title
Enhance Encryption App with Multi-Language Support and User Authentication
Adding features for multi-language support using Google Translate and implementing user authentication using Streamlit Authenticator.

[x] I have provided the issue title.

Info about the Related Issue
What's the goal of the project?
The goal of this project is to build a user-friendly encryption and decryption app using basic cryptographic algorithms (Caesar and Vigenère Ciphers). The project aims to enhance the user experience by adding features such as multi-language support and secure user authentication.

[x] I have described the aim of the project.

Name
Please mention your name.
Lavanya

[x] I have provided my name.

GitHub ID
Please mention your GitHub ID.
lvb05

[x] I have provided my GitHub ID.

Email ID
Please mention your email ID for further communication.
<EMAIL_ADDRESS>
[x] I have provided my email ID.

Identify Yourself
Mention in which program you are contributing (e.g., WoB, GSSOC, SSOC, SWOC).
GSSOC/IEEE IGDTU open-source week 2024

[x] I have mentioned my participant role.

Closes
Enter the issue number that will be closed through this PR.
*Closes: #1132
*

[x] I have provided the issue number.

Describe the Add-ons or Changes You've Made
Give a clear description of what you have added or modified.
*Added multi-language support using Google Translate (googletrans) to enable users to translate messages before encryption and after decryption.
Integrated user authentication using the streamlit-authenticator library, allowing users to log in with predefined credentials to access the app.
The app now includes a language selector in the sidebar, where users can choose input and output languages.
Enhanced the app’s functionality to ensure encrypted and decrypted messages can be translated seamlessly.*

[x] I have described my changes.

Type of Change
Select the type of change:

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Code style update (formatting, local variables)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?
Describe how your changes have been tested.
*I tested the multi-language translation by selecting different input and output languages from the sidebar, ensuring the translation works both before and after encryption.
I verified the user authentication flow by logging in with predefined credentials and logging out.
I tested the Caesar Cipher and Vigenère Cipher encryption and decryption functionalities to ensure they work seamlessly with the translation feature.*

[x] I have described my testing process.

Checklist
Please confirm the following:

[x] My code follows the guidelines of this project.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly wherever it was hard to understand.
[x] I have made corresponding changes to the documentation.
[x] My changes generate no new warnings.
[x] I have added things that prove my fix is effective or that my feature works.
[x] Any dependent changes have been merged and published in downstream modules.

Can u provide me the evaluation form and steps
Discord is not working
On Sun, 17 Nov, 2024, 10:55 github-actions[bot], @.***>
wrote:

👋 Thank you for opening this pull request! We're excited to review your
contribution. Please give us a moment, and we'll get back to you shortly!
Feel free to join our community on Discord https://discord.gg/9mZnkTRFFe
to discuss more!
—
Reply to this email directly, view it on GitHub
https://github.com/UTSAVS26/PyVerse/pull/1132#issuecomment-2480931879,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/BGYQR2C6OHSNVLTHVKCAIUD2BASDDAVCNFSM6AAAAABR5RURVCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOBQHEZTCOBXHE
.
You are receiving this because you authored the thread.Message ID:
@.***>

Which evaluation form and steps you are talking about, and discord link is working fine.


[log_files] Major fixes for log download

Description
Fixes the log_files plugin. It's currently really broken and just barely works. I made a few posts in Discord if you want more information in rambling format https://discord.com/channels/1022170275984457759/1022185801615933460/1209702398650749030
Basically the current implementation continuously spams LOG_REQUEST_LIST and LOG_REQUEST_DATA and the logic for handling the incoming data is broken.
Before
This wireshark screenshot should give you a pretty good idea of the problem

After
I rewrote most of the code using the QGC implementation as a reference. It works reliably and much faster now.

Example
I'm working on an open source automatic log upload service for @alexklimaj and we will be regularly exercising this code. Check it out if you're interested! https://github.com/dakejahl/logloader

Initially I had updated ProgressData with rate_kbps but then I realized I would have to change the proto file. Figured that was more effort than it's worth?


Fix typo in WLS on VM Admin Offer: Remove extra server word from ELK section

In the admin offer, the sentence is mentioned as "Export logs to Elasticsearch server?", whereas in the cluster and dynamic cluster offer, it is mentioned as "Export logs to Elasticsearch?".

PR still failed validation check due to OCA. Closing the PR.


es6ify generator tools

Another step towards #503

I'm hoping the dataproc 404 will be fixed soon.

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@c08271e). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #737   +/-   ##
=========================================
  Coverage          ?   79.58%
=========================================
  Files             ?        6
  Lines             ?      240
  Branches          ?       53
=========================================
  Hits              ?      191
  Misses            ?       30
  Partials          ?       19

Impacted Files
Coverage Δ

scripts/generator_utils.ts
66.66% <ø> (ø)

lib/discovery.ts
67.64% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c08271e...6b192c1. Read the comment docs.

Alright - the dataproc issue is resolved, re-ran the generator, and picked up a few new APIs along the way.


Fix css and readme in opensrp-table package

Tasks:

cleanup: Readme has a few errors that need to be fixed:

the uesMemo should have a second argument for dependencies that should be watched to control cache
Import statement needs to be fixed.
Should change this line :  return <ReactTable {...{ data, tableColumns: useColumns() }} />;

copy the the index.css file into the dist folder next to index.js.

make a note in the Readme.md for the user to add import<EMAIL_ADDRESS>

@p-netm , what if i directly use the css file like this import<EMAIL_ADDRESS>

@p-netm , what if i directly use the css file like this import<EMAIL_ADDRESS>
@prodhan29 We need to include it in the dist folder. Otherwise it will not get downloaded when one does yarn add


Fix race conditions in kube database

Fixes issue: https://github.com/grafana/beyla/issues/759

@marctc I did it basically for performance reasons. The consistence of the database is not so important but just avoiding Go panicking for concurrent operations.
Having multiple mutexes would prevent that, for example, a Pod metadata read is not blocked because a containerID is blocked in another side.
But if you think the code is cumbersome, we might use only a single mutex and only consider using several if in the future we detect any performance issue.

But if you think the code is cumbersome, we might use only a single mutex and only consider using several if in the future we detect any performance issue.

No, I just wanted understand why. Thanks for clarifying!


CircularProgressIndicator overflows container

Steps to reproduce
I put a CircularProgressIndicator inside a SizedBox.square which is inside a Container. I set SizedBox.square.dimension to constraints.maxWidth, constraints.maxHeight (using a LayoutBuilder).
Expected results
The CircularProgressIndicator not overflowing its parent size.
Actual results
The CircularProgressIndicator overflow its parent size. I checked the Widget Inspector and apparently it is taking the correct size but... the visuals shows the overflow.
I also put Container.clipBehavior to Clip.antiAlias and then the CircularProgressIndicator overflow was cutted off
Screenshots or Video
https://github.com/flutter/flutter/assets/73592852/0cf7caff-ef51-473d-bf66-76eb5163f47e
https://github.com/flutter/flutter/assets/73592852/902d15e5-7af3-4e4b-b74d-7cb8c7ea0e53

Hi @busslina, this issue was fixed with https://github.com/flutter/flutter/pull/125945. By default the strokeAlign of CircularProgressIndicator is set to CircularProgressIndicator.strokeAlignCenter but you can set it to CircularProgressIndicator.strokeAlignInside and it will be contained within the container.
I'll be closing this since it has already been fixed.
If you still face this issue after setting  CircularProgressIndicator.strokeAlignInside kindly leave a comment below with a minimal complete reproducible code sample along with the output of flutter doctor -v so that we can investigate this issue.
Thank you

Ok, I don't have thes options yet. My Flutter version is:
Flutter 3.13.1 • channel stable • https://github.com/flutter/flutter.git
Framework • revision e1e47221e8 (3 months ago) • 2023-08-22 21:43:18 -0700
Engine • revision b20183e040
Tools • Dart 3.1.0 • DevTools 2.25.0

I'm upgrading flutter...

@danagbemava-nc It worked. Thank you very much


[@urql/svelte] Svelte subscription example from docs doesn't work

urql version & exchanges:
  "devDependencies": {
    "@urql/svelte": "^1.2.0",
    "graphql": "^15.5.0",
    "sapper": "^0.28.0",
    "svelte": "^3.17.3"
  }

Steps to reproduce

Install sapper (app framework, using svelte) by docs:

npx degit "sveltejs/sapper-template#rollup" my-app
npm install

npm install --save @urql/svelte graphql

Use example from @urql/svelte docs: https://formidable.com/open-source/urql/docs/advanced/subscriptions/#svelte

npm run dev

Expected behavior

Realtime rendering data with graphql subscription

Actual behavior

Error:

src/components/Test.svelte changed. rebuilding...
• server
src/components/Test.svelte
'result' is not defined
108:
109:
110: {#if !$result.data}
           ^
111:
No new messages

112: {:else}
• client
src/components/Test.svelte
'result' is not defined
108:
109:
110: {#if !$result.data}
           ^
111:
No new messages

112: {:else}

Questions

Where should $result come from if we have not declared it anywhere?
Where i can find repo with worked example with @urql/svelte + graphql subscriptions? I found two examples:

urql/tree/main/packages/svelte-urql/example example was missing
https://github.com/dgraph-io/svelte-urql-example example was missing

Note

If i replace $result to $messages, then the error is fixed, but the data is still not loaded, although the example using the query works on same server and same query sctructure

Can you check please whether the events are coming in for your WebSocket? Alternatively, you can use the debugExchange and add it to the list of exchanges of the Client like so and give us the output:
import { debugExchange, defaultExchanges, createClient, setClient } from '@urql/svelte';

const client = createClient({
  // ...
  exchanges: [debugExchange, ...defaultExchanges, /* subscriptionExchange */
});

If you then show us the console output we can have a look together at whether the subscription is started.
It looks like the reference to $result is indeed a typo and meant to be $messages

Output after adding debugExchange:
[Exchange debug]: Incoming operation:  {
  key:<PHONE_NUMBER>,
  query: {
    kind: 'Document',
    definitions: [ [Object] ],
    loc: { start: 0, end: 57, source: [Object] },
    __key:<PHONE_NUMBER>
  },
  variables: {},
  kind: 'subscription',
  context: {
    url: 'https://graphql-engine-heroku-test.herokuapp.com/v1/graphql',
    fetchOptions: [Function: fetchOptions],
    fetch: undefined,
    preferGetMethod: false,
    suspense: false,
    requestPolicy: 'cache-first'
  }
}
No exchange has handled operations of kind "subscription". Check whether you've added an exchange responsible for these operations.

Ok, this warning afterwards means that you haven't actually set up an exchange yet to handle your subscriptions:

No exchange has handled operations of kind "subscription". Check whether you've added an exchange responsible for these operations.

Subscriptions are obviously bi-directional, meaning that for the standard GraphQL spec and HTTP server/API a RESTful endpoint is not enough to set up subscriptions. Furthermore, since the GraphQL spec doesn't specify what this transport should look like, there are several libraries you'll have to choose from to add subscriptions to your API, probably using Websockets.
I'd recommend graphql-ws myself, unless you already have an API that supports subscriptions with a specific protocol: https://formidable.com/open-source/urql/docs/advanced/subscriptions/#setting-up-graphql-ws
The main thing to watch out for is to set up a subscriptionExchange for urql's Client in your client-side code.
I'll close this issue for now, since it's not a bug but a usage question, but feel free to keep posting and I'll try my best to help! 🙌

Ok, but this is mean that docs for Svelte doesn't correct / full. Example code should work out-of-the-box, i mean.

I have new error after adding subscriptions-transport-ws by doc
Unable to find native implementation, or alternative implementation for WebSocket!

React app with subscriptions worked with subscriptions-transport-ws with the same server.

@kitten  Ok, but this is mean that docs for Svelte doesn't correct / full. Example code should work out-of-the-box, i mean. So, this issue is not done.

I see! 👍 What we assumed was that this paragraph at the top would be enough:

To add support for subscriptions we need to add the subscriptionExchange to our Client.

That's because introducing each section for React/Preact/Svelte/Vue with the same paragraph linking to the first anchor/section would be repetitive and tedious, but I can look at that.
I'm not sure why Sapper would complain and I'm not sure whether graphql-ws would work better, but that package is better maintained than Apollo's older one so you're more likely to get help with that package

@timeshift92 has kindly provided an alternative solution, if it is useful to someone, I will leave it here:
https://github.com/timeshift92/sapper-hasura


Failed to create local file on some iOS devices

The reason is unknown currently.

Fixed in v4.3.2


can not extract table

KG-DWN-98-2 Audited EOY-2013-14.pdf
Kindly look at page 7 table. It is not able to exract table info. I am using tesseract. Kindly advice.

This library simply uses the results of OCR (Optical Character Recognition) you specify, so I bet the issue lies in the OCR configuration. Have you tried other OCR tools, such as DocTR?

  BLOCK KG DWN 98/2 VENTURE\n(Operator Oil & Nat...
1                                        PARTICULARS                                      AMOUNT IN INR
2  TOTAL CONTRIBUTION RECEIVED TILL 31.03.2014 & ...<PHONE_NUMBER>
3          TOTAL PENDING CASH CALLS AS ON 31.03.2014                                                NIL
                                                   0                                                1                                                2
0    Schedule 7(a) : Cash Call Contribution - PIB-BV  Schedule 7(a) : Cash Call Contribution - PIB-BV  Schedule 7(a) : Cash Call Contribution - PIB-BV
1                                        PARTICULARS                                    AMOUNT IN INR                                    AMOUNT IN USD
2  TOTAL CASH CALL CONTRIBUTION RECEIVED TILL 30....                                        602260812                                         14942722
3                                  CASH CALL PENDING                                             None                                             None
4                                                NIL                                                0                                                0
5           TOTAL CASH CALL PENDING AS ON 30.03.2014                                                0                                                0
                                                   0                                              1                                              2
0      Schedule 8(a) : Cash Call Contribution - HOEI  Schedule 8(a) : Cash Call Contribution - HOEI  Schedule 8(a) : Cash Call Contribution - HOEI
1                                        PARTICULARS                                  AMOUNT IN INR                                  AMOUNT IN USD
2  TOTAL CASH CALL CONTRIBUTION RECEIVED TILL 31....                                      367418583                                        9110606
3                                  CASH CALL PENDING                                           None                                           None
4                                                NIL                                           None                                           None
5           TOTAL CASH CALL PENDING AS ON 31.03.2014                                              0                                              0


virt-launcher, converter: Set SRIOV device as unmanaged

What this PR does / why we need it:
The SRIOV devices are defined in the domain spec as hostdevice.
They are handled in an unmanaged manner as libvirt is not allowed to
touch the device on the host/pod side.
This change explicitly sets the managed property in the domain to "no"
to reflect the actual operation.
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #
Special notes for your reviewer:
I do not really understand how it silently worked.
I would expect libvirt to try and re-attach the device to the host when shutting down and failing.
Release note:
NONE

/cc @vladikr

from what I know, I think libvirt will not deallocate any VFs if it didn't allocate them in the first place. However, I didn't see any harm in setting this parameter to no explicitly.
/approve
I'll let some from the network team a chance to look at this.

/cc @AlonaKaplan

Adding some ref.
Quoting from https://libvirt.org/formatdomain.html#host-device-assignment:

pci
For PCI devices, when managed is "yes" it is detached from the host before being passed on to the guest and reattached to the host after the guest exits.
If managed is omitted or "no", the user is responsible to call virNodeDeviceDetachFlags (or virsh nodedev-detach before starting the guest or hot-plugging the device and virNodeDeviceReAttach (or virsh nodedev-reattach) after hot-unplug or stopping the guest.

/lgtm
/hold
I can't understand this worked until now, hence setting the hold.
You're free to remove the hold whenever you want.

/lgtm

/lgtm
/hold
I can't understand this worked until now, hence setting the hold.
You're free to remove the hold whenever you want.

Indeed a mystery how it worked. But since it is clear the hostdev should be unmanaged giving lgtm.

/hold cancel


add host templates to dynamic hosts

Tested on local vagrant.

Changes look fine to me but I'd be happier if this was tested in a multi node cluster as well as vagrant as we've been bitten by this in the past

Tested with Paul on nwr1.


Web is hacky to transpile in a non-trivial project

I had quite a few problems getting web to work in a way that isn't hacky.
Long story short, my project requires multiple versions of lodash due entirely to dependencies. Therefore, yarn puts the version that react native vector icons requires nested under its own node_modules folder. Webpack looks at the top level when compiling, not that nested level, and therefore it errors and fails to build. To begin with, the only reason this project needs to be compiled with webpack is because there's no ES5 version distributed.
Details: I believe the problem stems from this: some package in my app requires lodash 3.10.1, so that's located at /node_modules/lodash. React Native Vector Icons requires lodash v4.0.0, so that one's located at /node_modules/react-native-vector-icons/node_modules/lodash. The problem is that when Webpack tries to compile the project it looks in the top level of node_modules, therefore getting the wrong versions AND not finding some newer lodash methods, causing the build to error and fail. I use yarn if that makes a difference.
The hacky solution I've come up with for now was to change react-native-vector-icons to require lodash-es v4.17.4. That compiles correctly for my project, and since that version of lodash is exported as ES modules, I get the bonus of tree shaking via webpack 2. I'd still rather consume an ES5 version of this project rather than have to change my build script and mess with all this just to use it.
Any ideas on how to get this to compile without hacky nonsense?
Either exclude and include work the same here. My webpack config:
{
        test: /\.js$/,
        // exclude: /node_modules\/(?!(react-native-vector-icons)\/).*/,
        include: [
          path.resolve(process.cwd(), 'node_modules', 'react-native-vector-icons'),
          path.resolve(process.cwd(), 'js'),
          path.resolve(process.cwd(), 'index.web.js'),
        ],
        loader: 'babel-loader',
        query: {
          babelrc: false,
          cacheDirectory: true,
          plugins: [
            'transform-react-inline-elements',
          ],
          presets: [
            'es2015',
            'stage-0',
            'react-native',
          ],
        },
      },

package.json
    "react-native-vector-icons": "^4.0.0",
    "webpack": "^2.2.1",

Seems like yarn is the problem here - deleting node_modules and installing via npm i fixes all the above issues. Installing via yarn I get the same issues.

Facing the same issue which is a problem seeing as the react-native cli is pushing the use of yarn these days :(


Updated to 1.18.2

If you're reading this, please also publish the new version on CurseForge ASAP. Someone is in need with it. Thanks!

I'd recommend publishing the port yourself since I have no real interest in maintaining this project—as long as I have a single line of credit somewhere in a README or something I'll be more than fine with it.

Thanks for the quick response! I will for sure credit you.

going to close this since you're able to handle the fork.


Newer JDK version

lein eastwood can report usages of deprecated APIs with the :deprecations linter.
src/cljam/io/util/bgzf.clj:38:31: deprecations {:kind :new}: Constructor 'public java.net.URL(java.lang.String) throws java.net.MalformedURLException' is deprecated.

The newer versions of JDK contain more deprecated APIs. For example, the API shown above is deprecated since JDK 20.
I think it is useful to have clj-lint-action to report more deprecation warnings to be able to notice such usages earlier.

(memo)
There is a clojure Docker image that includes JDK20.
For example, clojure:temurin-20-tools-deps-jammy.


Support MacOS via Vulkan Portability

Vulkan isn't natively available on MacOS, but it's becoming more accessible as a user-space layer. It would be great to have the corresponding Meson configuration setup, build instructions, and eventually CI coverage.

@cdavis5e - would you have something to share in this regard? Given your continuous work to chop through the dxvk/vkd3d blockers on MoltenVK, you probably have some sort of a playground setup.

I cannot support that platform. If you need Mac support, someone who actually uses one has to provide support.

FYI , texture swizzling resolved on gfx layer.
https://github.com/RPCS3/rpcs3/pull/4996
At least worked around for RPCS3 on MacOS.
But , i rather his focus on DXVK only. Not on Mac. So Philip shouldn't make adjustments , people should adjust their layers to DXVK.

They work around the lack of swizzle support by doing that manually on the cpu and in shaders,
While that could be done for DXVK, the missing shader stages are the biggest problem.

I agree , that's why i said Philip shouldn't make workarounds.
It is an open source project , people can make adjustments on their forks etc.
If Philip involves with MacOS that would also mean he would have to consider MacOS situation on every step.

@kvark yes, the VKx discord is good place where DXVK development is discussed
https://discord.gg/wHBTezH

Texture swizzles are crucial to implement a number of texture formats (the most common one being A8_UNORM). That's something MoltenVK should try to emulate on their end if possible.
DXVK already emulates swizzles for render targets since Vulkan doesn't support that, but that's effectively a zero-cost emulation since the render target formats are known at the time shaders are compiled. For sampled textures this would be much harder to support.
Another thing is extensions - does MoltenVK implement all the extensions that DXVK currently depends on? If not, there are going to be more issues. And the upcoming transform feedback EXT will probably make things worse. I just don't see full D3D11 support happen on Mac OS X, as nice as it would be.

Looking through MoltenVK source, here's the list of supported extensions:
VK_MVK_moltenvk;
VK_MVK_macos_surface;
VK_MVK_ios_surface;
VK_KHR_surface;
VK_KHR_swapchain;
VK_KHR_maintenance1;
VK_IMG_format_pvrtc;
VK_AMD_negative_viewport_height;
VK_KHR_shader_draw_parameters;
VK_KHR_get_physical_device_properties2;
VK_KHR_push_descriptor;
VK_KHR_descriptor_update_template;

More information in here: https://github.com/KhronosGroup/MoltenVK/issues/203

@doitsujin Vulkan Portability != MoltenVK. I represent gfx-portability. I'd appreciate if we discuss this in a more general sense of Metal limitations instead of MoltenVK implementation  details.

Texture swizzles are crucial to implement a number of texture formats (the most common one being A8_UNORM).

That got me curious as well, since D3D11 doesn't have swizzling. You must be requiring a limited set of swizzles to cover formats that are in D3D11 but missing from Vulkan, and we might be able to provide them. For example, A8Unorm is already a supported swizzle. Is there a list of formats+swizzle combinations for us to work on?
In terms of extensions, there is indeed still work to do, but it's not the end of the world.

And the upcoming transform feedback EXT will probably make things worse.

I don't think it's worse. TF via vertex stage should be fairly trivial in Metal. TF via geometry shader / tessellation is more of a problem for those shader stages not supported natively by Metal, not as much of a problem for TF specifically.

Is there a list of formats+swizzle combinations for us to work on?

(R,G,B,1) might be useful for some RGBX formats where the 'X' component is ignored.

TF via geometry shader / tessellation is more of a problem for those shader stages not supported natively by Metal

Transform feedback via geometry shaders is the only thing that is supported in D3D11. It's annoying to emulate because it may require a compaction pass when vertices are conditionally emitted, but if Metal doesn't support geometry shaders, you'll probably have to do that anyway.

Your GPU is too old, it doesn't support D3D11.

In fact it does. You can see at https://en.wikipedia.org/wiki/List_of_AMD_graphics_processing_units#Features_Overview that Evergreen and Northern Islands class cards support D3D feature level 11_0. Also, TW3 runs fine in Windows (well, renders properly. can't turn the settings up beyond medium otherwise the fps goes to shit). I want to run it under Linux because the AMD drivers are unmaintained and don't even properly support dual monitor.

Well, you only said "TeraScale", which also includes pre-D3D11 GPUs, I couldn't know that you actually meant something else. Why not just name your GPU to avoid confusion?
Anyway, I don't see how gfx-rs is supposed to help you when there is no Vulkan support for your GPU, or what this has to do with Mac OS X.

gfx-rs apparently supports rendering to opengl, although I'm not totally sure how that works.


Update versions for 1.6.0 release

Test name
Status

haproxy
passed

big-data
passed

storage
failed

recreate
passed

scaling
passed

init-deploy
passed

scaling-proxysql
failed

restore-to-encrypted-cluster
passed

limits
passed

monitoring
failed

demand-backup
failed

scheduled-backup
failed

self-healing
passed

upgrade
passed

self-healing-advanced
passed

upgrade-consistency
passed

demand-backup-encrypted-with-tls
passed

security-context
failed

monitoring-2-0
failed

operator-self-healing
passed

affinity
failed

one-pod
passed

auto-tuning
passed

proxysql-sidecar-res-limits
passed

users
passed

tls-issue-self
passed

tls-issue-cert-manager
passed

tls-issue-cert-manager-ref
passed


mobile/ci: Fix compile-time-options check completion

Commit Message:
Additional Description:
Risk Level:
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Fixes commit #PR or SHA]
[Optional Deprecated:]
[Optional API Considerations:]

pushing through to fix CI


Report airdrop hunter #2 by yaoydong

Related Safe Addresses
0x024be472582b66a00bc828469dfe6e0086e25270
0x1997cc1e34725aafc1d7288b44516176c9067532
0x1b3da570f81e0b1f5840f31d5efd18cf032ebfd9
0x33bd3d5b50ed7a119086cbdf4119fbec49398580
0x3479272d04d91059c688c94375f0ccae1fdfbc9b
0x3522d0574883fb5d1446f045c5ffb6e84d4e8465
0x3881edc1a03407f304bf2919980c5b64852ede91
0x3ade95f633be178ad918dc1137b45beda0371a93
0x50000662ef7a2bbfd14146dbb60add32ad5c4ccd
0x7ac286b009f9b8a65c52d7c17190689df399820c
0x8d2609749b574d72438851e829bd79ae986b4882
0x9b520f5e9a57b686bfcf02958024894a5e615265
0xa87e0786c514254e68669f56f2bfc49954ede840
0xe7bf23e0e9a81ebce1466e5ac74022a6e6fb5401
0xeb1998e2f892304e4007792f897abd13d7090e9f
Reasoning
same Safe airdrop<PHONE_NUMBER>)
created by these two wallets:0x66a0ff664f8509370c5d718a0f69ac1dc01f5c3d and 0x5769770f5efe8fb017fb09b6de3b2d096668377d
same exec transaction txs: 1
same execed by: 0xdf21b894a490cbdf49d9de7b9d780b79f7cb0ac7
same execed time :Jul-18-2021 04:54:37 PM +UTC  [timestamp:1626627277]
same event logs: AddedOwner and RemovedOwner
for example:
https://etherscan.io/address/0x024be472582b66a00bc828469dfe6e0086e25270
https://etherscan.io/address/0x1997cc1e34725aafc1d7288b44516176c9067532
https://etherscan.io/address/0x1b3da570f81e0b1f5840f31d5efd18cf032ebfd9
https://etherscan.io/address/0x3479272d04d91059c688c94375f0ccae1fdfbc9b
Methodology
I checked the winner xls, filter according to the air drop value, and then obtain safe's creator, caller, and the txs  of safe, and then manually check the safes with the same air drop value, creator, caller, and number of calls
checked with code:https://github.com/yaoydong/safe-checker
Safe Address
0xfdf11fa8ae68ddb3ff2b9138a517e6518f2e8796

Thanks for the report. All of these Safes have been found by a previous report, unfortunately.


Batching UI elements should easier to do correctly

Related forum thread: https://forum.playcanvas.com/t/batching-of-ui-elements/15946/9
Other issues:
https://github.com/playcanvas/engine/issues/2579
https://github.com/playcanvas/engine/issues/2578
Currently, batching UI elements requires using layers to enforce the render order for the batcher and ensure that elements are rendered correctly. If we can make this easier so there's less work on the user to have to construct their UI screens, that could be a big win.
The example in the forum was pretty straight forward to do as there wasn't any real overlapping elements and UI order is clearly defined. (Board, red tiles then text). I can imagine larger projects being more complex and harder to manage globally.

are there any updated on this? batching ui elements is still very hard to do, especially with complicated UI systems.

There was no update on this yet unfortunately.


Replace @elastic/datemath with @opensearch/datemath

Signed-off-by: Matt Provost<EMAIL_ADDRESS>Description
Replaces @elastic/datemath with @opensearch/datemath from OSD. This helps with our legal compliance and will help make it easier to fix security vulnerabilities in the future.
Issues Resolved
#129
Check List

[x] New functionality includes testing.
[ ] New functionality has been documented.
[x] All tests pass

[x] yarn lint
[x] yarn test-unit

[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

backported in https://github.com/opensearch-project/oui/pull/872


[WIP] Nydus: make manifest compatible with docker

Signed-off-by: Yan Song<EMAIL_ADDRESS>

This feature is quite useful, please consider converting it as an new option.

Since this is a temporary workaround, close the PR first before considering the formal solution.


Final edits before release of 0.7.0

Description

Added a few lines in the README to inform about the renaming of the folders.
Related Issues

Checklist:

[x] I have followed the contribution guidelines and code style for this project.
[ ] I have added tests covering my contributions.
[x] I have updated the documentation accordingly.
[x] This PR is being made to staging branch and not to main branch.

Codecov Report

Merging #1500 (1cab0e2) into staging (a8c81fe) will decrease coverage by 12.20%.
The diff coverage is 82.75%.

@@             Coverage Diff              @@
##           staging    #1500       +/-   ##
============================================
- Coverage    74.23%   62.03%   -12.21%
============================================
  Files           84       84
  Lines         8369     8397       +28
============================================
- Hits          6213     5209     -1004
- Misses        2156     3188     +1032

Impacted Files
Coverage Δ

recommenders/evaluation/spark_evaluation.py
86.66% <81.48%> (-0.78%)
:arrow_down:

recommenders/utils/constants.py
100.00% <100.00%> (ø)

...ecommenders/models/newsrec/io/mind_all_iterator.py
12.21% <0.00%> (-86.65%)
:arrow_down:

recommenders/models/newsrec/io/mind_iterator.py
15.67% <0.00%> (-82.71%)
:arrow_down:

...ommenders/models/deeprec/io/sequential_iterator.py
15.85% <0.00%> (-81.94%)
:arrow_down:

recommenders/models/newsrec/models/base_model.py
30.90% <0.00%> (-59.40%)
:arrow_down:

...deeprec/models/sequential/sequential_base_model.py
46.97% <0.00%> (-47.66%)
:arrow_down:

recommenders/models/geoimc/geoimc_data.py
41.66% <0.00%> (-44.80%)
:arrow_down:

...enders/models/deeprec/io/dkn_item2item_iterator.py
45.61% <0.00%> (-42.11%)
:arrow_down:

...menders/models/deeprec/models/graphrec/lightgcn.py
51.47% <0.00%> (-40.24%)
:arrow_down:

... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c806d55...1cab0e2. Read the comment docs.

Codecov Report

Merging #1500 (1cab0e2) into staging (a8c81fe) will decrease coverage by 12.20%.
The diff coverage is 82.75%.

@@             Coverage Diff              @@
##           staging    #1500       +/-   ##
============================================
- Coverage    74.23%   62.03%   -12.21%
============================================
  Files           84       84
  Lines         8369     8397       +28
============================================
- Hits          6213     5209     -1004
- Misses        2156     3188     +1032

Impacted Files
Coverage Δ

recommenders/evaluation/spark_evaluation.py
86.66% <81.48%> (-0.78%)
:arrow_down:

recommenders/utils/constants.py
100.00% <100.00%> (ø)

...ecommenders/models/newsrec/io/mind_all_iterator.py
12.21% <0.00%> (-86.65%)
:arrow_down:

recommenders/models/newsrec/io/mind_iterator.py
15.67% <0.00%> (-82.71%)
:arrow_down:

...ommenders/models/deeprec/io/sequential_iterator.py
15.85% <0.00%> (-81.94%)
:arrow_down:

recommenders/models/newsrec/models/base_model.py
30.90% <0.00%> (-59.40%)
:arrow_down:

...deeprec/models/sequential/sequential_base_model.py
46.97% <0.00%> (-47.66%)
:arrow_down:

recommenders/models/geoimc/geoimc_data.py
41.66% <0.00%> (-44.80%)
:arrow_down:

...enders/models/deeprec/io/dkn_item2item_iterator.py
45.61% <0.00%> (-42.11%)
:arrow_down:

...menders/models/deeprec/models/graphrec/lightgcn.py
51.47% <0.00%> (-40.24%)
:arrow_down:

... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c806d55...1cab0e2. Read the comment docs.

Codecov Report

Merging #1500 (1cab0e2) into staging (a8c81fe) will decrease coverage by 12.20%.
The diff coverage is 82.75%.

@@             Coverage Diff              @@
##           staging    #1500       +/-   ##
============================================
- Coverage    74.23%   62.03%   -12.21%
============================================
  Files           84       84
  Lines         8369     8397       +28
============================================
- Hits          6213     5209     -1004
- Misses        2156     3188     +1032

Impacted Files
Coverage Δ

recommenders/evaluation/spark_evaluation.py
86.66% <81.48%> (-0.78%)
:arrow_down:

recommenders/utils/constants.py
100.00% <100.00%> (ø)

...ecommenders/models/newsrec/io/mind_all_iterator.py
12.21% <0.00%> (-86.65%)
:arrow_down:

recommenders/models/newsrec/io/mind_iterator.py
15.67% <0.00%> (-82.71%)
:arrow_down:

...ommenders/models/deeprec/io/sequential_iterator.py
15.85% <0.00%> (-81.94%)
:arrow_down:

recommenders/models/newsrec/models/base_model.py
30.90% <0.00%> (-59.40%)
:arrow_down:

...deeprec/models/sequential/sequential_base_model.py
46.97% <0.00%> (-47.66%)
:arrow_down:

recommenders/models/geoimc/geoimc_data.py
41.66% <0.00%> (-44.80%)
:arrow_down:

...enders/models/deeprec/io/dkn_item2item_iterator.py
45.61% <0.00%> (-42.11%)
:arrow_down:

...menders/models/deeprec/models/graphrec/lightgcn.py
51.47% <0.00%> (-40.24%)
:arrow_down:

... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c806d55...1cab0e2. Read the comment docs.

Codecov Report

Merging #1500 (1cab0e2) into staging (a8c81fe) will decrease coverage by 12.20%.
The diff coverage is 82.75%.

@@             Coverage Diff              @@
##           staging    #1500       +/-   ##
============================================
- Coverage    74.23%   62.03%   -12.21%
============================================
  Files           84       84
  Lines         8369     8397       +28
============================================
- Hits          6213     5209     -1004
- Misses        2156     3188     +1032

Impacted Files
Coverage Δ

recommenders/evaluation/spark_evaluation.py
86.66% <81.48%> (-0.78%)
:arrow_down:

recommenders/utils/constants.py
100.00% <100.00%> (ø)

...ecommenders/models/newsrec/io/mind_all_iterator.py
12.21% <0.00%> (-86.65%)
:arrow_down:

recommenders/models/newsrec/io/mind_iterator.py
15.67% <0.00%> (-82.71%)
:arrow_down:

...ommenders/models/deeprec/io/sequential_iterator.py
15.85% <0.00%> (-81.94%)
:arrow_down:

recommenders/models/newsrec/models/base_model.py
30.90% <0.00%> (-59.40%)
:arrow_down:

...deeprec/models/sequential/sequential_base_model.py
46.97% <0.00%> (-47.66%)
:arrow_down:

recommenders/models/geoimc/geoimc_data.py
41.66% <0.00%> (-44.80%)
:arrow_down:

...enders/models/deeprec/io/dkn_item2item_iterator.py
45.61% <0.00%> (-42.11%)
:arrow_down:

...menders/models/deeprec/models/graphrec/lightgcn.py
51.47% <0.00%> (-40.24%)
:arrow_down:

... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c806d55...1cab0e2. Read the comment docs.

codecov went bananas XDDD

Codecov Report

Merging #1500 (1cab0e2) into staging (a8c81fe) will decrease coverage by 12.20%.
The diff coverage is 82.75%.

@@             Coverage Diff              @@
##           staging    #1500       +/-   ##
============================================
- Coverage    74.23%   62.03%   -12.21%
============================================
  Files           84       84
  Lines         8369     8397       +28
============================================
- Hits          6213     5209     -1004
- Misses        2156     3188     +1032

Impacted Files
Coverage Δ

recommenders/evaluation/spark_evaluation.py
86.66% <81.48%> (-0.78%)
:arrow_down:

recommenders/utils/constants.py
100.00% <100.00%> (ø)

...ecommenders/models/newsrec/io/mind_all_iterator.py
12.21% <0.00%> (-86.65%)
:arrow_down:

recommenders/models/newsrec/io/mind_iterator.py
15.67% <0.00%> (-82.71%)
:arrow_down:

...ommenders/models/deeprec/io/sequential_iterator.py
15.85% <0.00%> (-81.94%)
:arrow_down:

recommenders/models/newsrec/models/base_model.py
30.90% <0.00%> (-59.40%)
:arrow_down:

...deeprec/models/sequential/sequential_base_model.py
46.97% <0.00%> (-47.66%)
:arrow_down:

recommenders/models/geoimc/geoimc_data.py
41.66% <0.00%> (-44.80%)
:arrow_down:

...enders/models/deeprec/io/dkn_item2item_iterator.py
45.61% <0.00%> (-42.11%)
:arrow_down:

...menders/models/deeprec/models/graphrec/lightgcn.py
51.47% <0.00%> (-40.24%)
:arrow_down:

... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c806d55...1cab0e2. Read the comment docs.

@anargyri we got a max time in the test of notebook spark, I increased the time

looks good, are you planning to add in this one the new version in the init?

Sorry, I just changed it.

@anargyri we got a max time error in the test of notebook spark, I increased the time

Thanks. Is this related to the recurring failure we see in the build pipelines with ALS fit()? It frequently fails here.
It looks like a Java connection error when calling the ALS fit() method.

Now the build fails to see the GPU:
=================================== FAILURES ===================================
_________________________________ test_gpu_vm __________________________________
@pytest.mark.notebooks
@pytest.mark.gpu
def test_gpu_vm():

  assert get_number_gpus() >= 1

E       assert 0 >= 1
E        +  where 0 = get_number_gpus()
tests/unit/examples/test_notebooks_gpu.py:18: AssertionError

Now the build fails to see the GPU:

I think this one also went bananas, maybe when @laserprec comes he can review. I think this one is ok to merge


add freebsd support

Add support for FreeBSD

@ashleyj I have recently created an improved PR for FreeBSD, see #26.

@chirino You can close this PR. It has been implicitly resolved by #26.


replace old make intro by data processing pipeline

Changed the way we motivate and introduce Make - with a data processing pipeline. This worked well at the last workshop (apart from Python 2/3 hiccups which are now fixed) with the big bonus that it motivates Makefiles for Python/R developers. I can either walk you through it @blindij or alternatively teach this part. Of course feel free to disagree with this change.

Another motivation for rather significant changes was to let people learn about Makefiles more passively. In this exercise I leave out all details and focus on understanding. The goal is that after this exercise everybody "gets" make and my impression from last time is that this was the case.


Add checksum verification in ollama.py

Adds checksum check after model blobs pulls in ollama.py
Reference: #293
Example run:
(venv) spande-mac:ramalama (checksum-verification*) $ ./bin/ramalama pull tiny
Warning: podman needs to be configured to use krunkit for AI Workloads,
running without containers

Pulling tinyllama/latest: 100% ▕####################▏ 852/852 2.62MB/s 00:00
File https://registry.ollama.ai/v2/library/tinyllama/blobs/sha256:6331358be52a6ebc2fd0755a51ad1175734fd17a628ab5ea6897109396245362 already fully downloaded.
Pulling 83e42c043d6c7816: 100% ▕####################▏ 608M/608M 3.54MB/s 00:00
Checksum verified for blob /Users/spande/.local/share/ramalama/repos/ollama/blobs/sha256:2af3b81862c6be03c769683af18efdadb2c33f60ff32ab6f83e42c043d6c7816.

@ericcurtin If you want checksum verification to be silent (i.e. not printing the "Checksum verified for blob" string), I can amend the commit.

Please be silent... But we can have these messages under a debug or verbose option if needs be... Also we only need one progress bar... The model one... The other http request/responses can be silent...
But of course print on errors...

Please be silent... But we can have these messages under a debug or verbose option if needs be... Also we only need one progress bar... The model one... The other http request/responses can be silent...
But of course print on errors...

@ericcurtin I've observed that 3 separate files are downloaded whenever a model is pulled, two of them are generally really small and the final one is the large one ( which is the model file I assume :) ).
Shall I create a separate issue for this, such that it can be resolved under a separate PR ?

Please be silent... But we can have these messages under a debug or verbose option if needs be... Also we only need one progress bar... The model one... The other http request/responses can be silent...
But of course print on errors...

@ericcurtin I've observed that 3 separate files are downloaded whenever a model is pulled, two of them are generally really small and the final one is large ( which is the model file I assume :) ). Shall I create a separate issue for this, such that it can be resolved under a separate PR ?

Yeah sure it could be handled in another PR... The second of those files is unnecessary to be downloaded (I had this change in a branch somewhere but maybe I lost that branch)... The first one downloaded doesn't need to be stored in a file, it can just be stored in a variable in python... The only one we need to store is the money

@ericcurtin Got it!, I'll start to implement this in a separate PR. Thanks for the suggestions.


Create a pipeline for training data ingestion, merging, cleaning, validation and transformation

https://github.com/nasa-petal/data-collection-and-prep/issues/93#issuecomment-883660235

subtasks/related to: #93, #94.

dagshub.com may be useful. or DVC

https://greatexpectations.io/ - framework that can help with automation of data quality validation

[ ] Define the golden.json fields that are required for our ML model to function properly. @elkong

(cross-posted in the Discord)
I mentioned I'd have some ideas for golden.json quality checks. Partly based on what problems have come up in the past, here are a few I propose I'd like to see. Besides checks on datatype (e.g., "is this a string?", "is this an array of numbers?"):
In general:
- Are text fields free of any non-ASCII characters?
- Are arrays free of duplicates?

DOI
- Is it well-formed?
- Does it resolve?

Level1, Level2, Level3 labels
- Can each label be found in the taxonomy file?
- Conditional on isBiomimicry being true, is level1 non-empty?
- Are these not strings, but lists of strings?

Title
- Is it non-empty?

Abstract
- Is it non-empty?

URL
- Does it resolve?

fullDocLink
- Conditional on isOpenAccess being true, does it resolve?

And if I can ask for the moon on a silver platter:
venue_mag and venue
- Does each venue have a distinct name? (e.g., we don't have both "PNAS" and "Proceedings of the National Academy of Sciences of the United States of America" in the dataset)

(that one's actually not too important, but it might help)
Notes:

It would be nice for metadata (venue, authors, references, and MAG fields of study) to be non-empty, but for the sake of not throwing out too much good data, I'll only require that the text fields (title and abstract) are non-empty.
Sometimes I find labels in golden.json that cannot be found in the taxonomy file, and vice versa. Currently, the labels in golden.json which are not in the taxonomy file are:

['sense_chemicals',
 'manage_environmental_disturbances',
 'cooperate_within/between_species',
 'move_through/on_solids',
 'manage_environmental_disturbances_in_a_community',
 'send_chemical_signals',
 'move_through/on_liquids',
 'sense_send_or_process_information',
 'compete_within/between_species',
 'sense_disease',
 'move_through_gases',
 'store_soilds']

And the labels in taxonomy.txt which are not in the golden dataset are
['send_chemical_signals_',
 'sense_disease_in_a_living_system',
 'self-replicate',
 'chemically_assemble_inorganic_compounds',
 'sense_chemicals_',
 'send_tactile_signals',
 'compete_within_or_between_species',
 'manage_environmental_disturbance_in_a_community',
 'store_solids',
 'cooperate_within_or_between_species',
 'sense,_send,_or_process_information']

I know the PeTaL taxonomy's gone through some overhauling, so I hope I don't have an outdated taxonomy.txt file yet.
What are your thoughts on how feasible it is to implement these checks, and how important they are? Am I missing anything obvious? All of this is open to revision.

Issue resolved with new repository:
https://github.com/nasa-petal/petal-labeler-data-pipeline


Add Huawei LTE device registry support

Description:
Support device registry in Huawei LTE.
Checklist:

[x] The code change is tested and works locally.
[x] Local tests pass with tox. Your PR cannot be merged unless tests pass
[x] There is no commented out code in this PR.
[x] I have followed the development checklist

If user exposed functionality or configuration variables are added/changed:

[ ] Documentation added/updated in home-assistant.io

If the code communicates with devices, web services, or third-party tools:

[ ] The manifest file has all fields filled out correctly. Update and include derived files by running python3 -m script.hassfest.
[ ] New or updated dependencies have been added to requirements_all.txt by running python3 -m script.gen_requirements_all.
[ ] Untested files have been added to .coveragerc.

If the code does not interact with devices:

[ ] Tests have been added to verify that the new code works.

Would be nice to get this in in the same release as the large modernization :)