Type nativeptr isn't usable at all

Framework code which uses the type nativeptr<unit> isn't usable in F# because we cannot cast any nativeptr to nativeptr<unit>. Exact error is that unit isn't an unmanaged type.
I noticed this after trying to compile simple examples with the Span type from the System.Memory package. Because the C# equivalent of nativeptr<unit> is void* and nothing is wrong with a void pointer I thought reporting this here would be correct.
To have a senseful usage example look at this code (requires the System.Memory package!):
let someOperationWithSpan () =
    let data = NativePtr.stackalloc<byte>(256)
    let voidPtr = data |> NativePtr.toNativeInt |> NativePtr.ofNativeInt<unit> // This line doesn't compile
    let stackMemory = new Span<byte>(voidPtr, 256)
    // Do some stuff with the span!
    ()

Sorry I posted the wrong conversion code 😄
Corrected it to the right example.
I see some problems with allowing unit to be an unmanaged type in general. I would suggest to add a function to FSharp.Core which allows a conversion from any nativeptr kind to nativeptr<unit>.

Already created a Hacky fix for this in the compiler. It just changes in the type checker that the unit type passes the unmanaged constrain and the initial sample with stackalloc using unit works without runtime error.
I assume that size of the type should be calculated as 1 byte?

What are you trying to do? Casting to unit doesn’t really make any sense.

@Saul the constructor requires an void pointer :)
I'm going to correct the example later, having Christmas right now.

@saul I updated the example to be correct.

Note that a workaround is to use IntPtr.ToPointer:
let someOperationWithSpan () =
    let data = NativePtr.stackalloc<byte>(256)
    let voidPtr = (data |> NativePtr.toNativeInt).ToPointer()
    let stackMemory = new Span<byte>(voidPtr, 256)
    // Do some stuff with the span!
    ()

Now the claimed inferred type of voidPtr is nativeptr, but attempting to explicitly declare it as such results in the same error message. Overloaded meaning of types strikes again!
There is also IntPtr.op_Explicit for IntPtr -> void*, but there is no way of selecting the correct overload in F#.

@poizan42 after I applied a fix in the compiler I ran into the same issue. This then is a bug I guess?

@realvictorprm, yeah - there are actually several problems in F# like this happening because the same F# type can mean different .NET types. I think there might be more to fixing this than just marking unit as unmanaged, because unit can refer to both Microsoft.FSharp.Core.Unit and System.Void.

Must be a bug.
cc @dsyme

I am afraid it may need deeper changes in the system, than a few lines of change in the code. Without supporting the core low level stuff eg.: Reference semantics with value types constructs (all of them) F# may not be able to use the new constructs in the base libraries. The current unit type aliasing and the byref constructs are far from perfect in F#. I had to move for sevaral performance critical code to C# , because F# cannot express it properly for example structs types with indexed O(1) self element access.:
https://github.com/Microsoft/visualfsharp/issues/843
https://github.com/Microsoft/visualfsharp/pull/3525
"Another related language feature is the ability to declare a value type that must be stack allocated. In other words, these types can never be created on the heap as a member of another class. The primary motivation for this feature was Span and related structures. Span may contain a managed pointer as one of its members, the other being the length of the span. It's actually implemented a bit differently because C# doesn't support pointers to managed memory outside of an unsafe context. Any write that changes the pointer and the length is not atomic. That means a Span would be subject to out of range errors or other type safety violations were it not constrained to a single stack frame. In addition, putting a managed pointer on the GC heap typically crashes at JIT time."
https://docs.microsoft.com/en-us/dotnet/csharp/reference-semantics-with-value-types

Hm, I don't like this.
@zpodlovics did you ever open a language suggestion etc. ?

@realvictorprm Thanks for your help! I don't like it either. I would like to keep all the development in F# as much as possible, because I could retarget the high level parts later to different runtime eg.: fable, fez, fshlvm, etc.
I ran into some corner cases and issues in the past with F# that was related with these low level stuff (unmanaged atomics, byref restrictions, struct units of measure types, struct du, , etc). Most of the issues was fixed, but some of them was remained unaddressed (which is not unreasonable, as the netstandard and sdk integration was lot more important).
Please do not take this as offense, I would like to use Span as much as you, in fact I already tried to use it in development (with some C# helpers, because I am not yet figured out how to do it fully or partially in F# or inline IL eg.: by loading load a void* type to F# with custom typeofvoidptr function - something like tobyref for loading void* types https://github.com/Microsoft/visualfsharp/issues/409). However reading the original design span design docs and ref like design docs (the .net core design reviews are also avaialble for span at https://www.youtube.com/playlist?list=PLRAdsfhKI4OWIV9TuZ0QmmlQzz41o80j0) I realized the missing parts are may larger than I earlier estimated (eg.: stack only types, readonly byref types, byref restrictions), and using it properly - especially in production - may require lot more. In production. I had to satisfy different constraints, and running into weird unsolvable corner cases is not fun. I would love to make a technically sound language change plans, but I am afraid I am not here yet.
Here is the decompiled C# helper. Unfortunately I was not able to call the op_explicit cast without expressing the void* type somehow.
    // method line 2
    .method public static hidebysig
           default void* ToVoidPtr (native int 'value')  cil managed
    {
        // Method begins at RVA 0x206c
	// Code size 12 (0xc)
	.maxstack 1
	.locals init (
		void*	V_0)
	IL_0000:  nop
	IL_0001:  ldarg.0
	IL_0002:  call void* native int::op_Explicit(native int)
	IL_0007:  stloc.0
	IL_0008:  br.s IL_000a

	IL_000a:  ldloc.0
	IL_000b:  ret
    } // end of method VoidPtr::ToVoidPtr

Span design docs:
https://github.com/dotnet/corefxlab/blob/master/docs/specs/span.md
https://github.com/dotnet/csharplang/blob/master/proposals/csharp-7.2/readonly-ref.md
https://github.com/dotnet/csharplang/blob/master/proposals/csharp-7.2/span-safety.md
https://youtube.com/watch?v=VKnHYFIQpOs
https://youtube.com/watch?v=jKQWMCPZ08Y
https://youtube.com/watch?v=10w3tS03W60

@realvictorprm Is this a subset of #4288?

Well this is the issue tracking specifically that a void pts isn't usable
in F# but fixing the referenced issue would require this issue to be fixed
aswell.
Am 14.02.2018 5:30 vorm. schrieb "Phillip Carter"<EMAIL_ADDRESS>
:

@realvictorprm https://github.com/realvictorprm Is this a subset of
#4288 https://github.com/Microsoft/visualfsharp/issues/4288?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Microsoft/visualfsharp/issues/4166#issuecomment-365493188,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AYPM8IOw2lZ-ikVWVCoRoWu9DEPv9xa-ks5tUmF-gaJpZM4RL8A-
.

It seems that the same code (with the .ToPointer() trick) no longer works with the .NET 2.1 Preview1 using <TargetFramework>netcoreapp2.1</TargetFramework> in the fsproj.
open System
open Microsoft.FSharp.NativeInterop

[<EntryPoint>]
let main argv =
    let data = NativePtr.stackalloc<byte>(256)
    let voidPtr = (data |> NativePtr.toNativeInt).ToPointer()
    let stackMemory = new Span<byte>(voidPtr, 256)
    printfn "Hello World from F#!"
    0 // return an integer exit code

error FS0101: This construct is deprecated. Types with embedded references are not supported in this version of your compiler.

Environment: Ubuntu 16.04 x86_64
Packages:
ii  dotnet-host                                                 2.1.0-preview1-26216-03-1                                   amd64        Microsoft .NET Core Host - 2.1.0 Preview 1
ii  dotnet-hostfxr-2.1.0-preview1-26216-03                      2.1.0-preview1-26216-03-1                                   amd64        Microsoft .NET Core Host FX Resolver - 2.1.0 Preview 1 2.1.0-preview1-26216-03
ii  dotnet-runtime-2.1.0-preview1-26216-03                      2.1.0-preview1-26216-03-1                                   amd64        Microsoft .NET Core Runtime - 2.1.0 Preview 1 Microsoft.NETCore.App 2.1.0-preview1-26216-03
ii  dotnet-runtime-deps-2.1.0-preview1-26216-03                 2.1.0-preview1-26216-03-1                                   amd64        dotnet-runtime-deps-2.1.0-preview1-26216-03 2.1.0-preview1-26216-03
ii  dotnet-sdk-2.1.300-preview1-008174                          2.1.300-preview1-008174-1                                   amd64        Microsoft .NET Core SDK 2.1.300 - Preview

error FS0101: This construct is deprecated. Types with embedded references are not supported in this version of your compiler.

I don't know where that error is coming from, it's not an F# error

Oh, it's a deprecation error....
  | ObsoleteError _ -> 101

@zpodlovics @dsyme All ref structs (or "ref-like structs") are marked with an [Obsolete] attribute and that message, to make sure a compiler that doesn't support them won't be able to use them.
See the proposal and this constant in Roslyn.

OK, I can see that raises the priority of Span work...

@zpodlovics language suggestion at https://github.com/fsharp/fslang-suggestions/issues/648

Note that while escape rules for ref structs in C# are rather complex, most of that comes from supporting wrapping stackalloc buffers in spans.
C# supports things like:
Method()
{
       // obviously this span or its slices cannot be allowed to be returned from the containing method
       Span<int> sp = stackalloc int[10];
}

This is a fairly advanced scenario.
In reality most of the ref-like structs are either transient to your code or wrap heap data (strings, arrays, etc..). In those cases escaping is not a concern.
I'd recommend to not care about stackalloc at first, or possibly ever. Then you only need to guarantee the "no heap" part, which is simpler. - may not be boxed, may not be a generic type argument, may not be a field unless an instance field of another ref struct, and so on.
We had most of these rules already implemented for old "unboxable" types such as TypedReference. The only difference is that there are couple scenarios that are allowed for ref-struct, but for whatever reasons were not allowed for TypedReference.

And, yes, ReadOnlySpan has an indexer that is ref readonly. That will drag in the readonly references.
Good explanation is here:
https://github.com/dotnet/csharplang/blob/master/proposals/csharp-7.2/readonly-ref.md
Those features are useful on their own.
Conceptually they just allow to specify that a byref is readonly.
That could be used, for example, for passing around direct references to a large struct field or an array element just for reading while avoiding unnecessary copying.

@VSadov We already have similar code snippets in F# due the latency / GC / performance requirements, but with NativePtr.stackalloc / NativePtr.read / NativePtr.write / NativePtr.get / NativePtr.set.  [1] [2] [3]
Please do NOT drop stackalloc support in F# and especially for Span just because it would be easier to implement initially (but at the same time the some thing stackalloc/Span will be possible in C#), because doing latency / gc / performance critical code would be imppossible in F# otherwise. The F# language will become medocrie tool for latency / gc / performance critical code compared to C#. F# should never be a second class citizen in .NET or other runtime for latency / gc / performance critical code.
I intentionally used the Easy instead of Simple here. Using stackalloc and Span should be Simple in F#, but not necessary Easy to learn and do: https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/SimpleMadeEasy.md
[1] https://github.com/Microsoft/visualfsharp/issues/843
[2] https://github.com/Microsoft/visualfsharp/pull/3525
[3] https://github.com/Microsoft/visualfsharp/issues/409

@zpodlovics i dont want to muddle up your issues with irrelevant stuff, but I was wondering if the following escape hatch works for your situation:
let inline cast<'T when 'T : unmanaged>(p : byref<'T>) : nativeptr<'T> = (# "" p : nativeptr<'T> #);;

Was going to suggest a pinvoke escape hatch, which I believe would work, but I think a language level thing is much more appropriate, and testable !

The proposal is to address this by adding voidptr in this RFC: https://github.com/fsharp/fslang-design/blob/master/RFCs/FS-1053-span.md
Please take a look and review carefully, providing evil test cases where possible.

This code is now possible with F# 4.5:
open Microsoft.FSharp.NativeInterop

let someOperationWithSpan () =
    let data = NativePtr.stackalloc<byte>(256)
    let voidPtr = data |> NativePtr.toVoidPtr
    let stackMemory = new Span<byte>(voidPtr, 256)
    // Do some stuff with the span!
    ()

Closing, since voidptr is now a concept in the language.


.umap failed to maintain binary equality

Issue
This .umap failed to maintain binary equality when edited with uassetAPI
To Reproduce

Download the assets
Open with uassetGUI with the version of UE4 set to 4.25
see the failed to verify binary equality error

Desktop (please complete the following information):

OS: Windows 10
Version or Commit: this one

Context
If you are unable to or do not have enough time that is fine because my project does not solely rely on this one fil
Sealed_Land.zip

Thanks for sending this in, fixed in b65ecf2


SelectedBackgroundColor and SelectedTextColor in XAML

Using either option in XAML results in this runtime message:
Cannot assign property "SelectedBackgroundColor": Property does not exists, or is not assignable, or mismatching type between value and property.
or
Cannot assign property "SelectedTextColor": Property does not exists, or is not assignable, or mismatching type between value and property.
How can I set the selected background and text colour?

This was cause by the type being Color?.
In version 1.1.0 I change the type to Color  so this should be fixed in that version. Please reopen this issue if you still have problems in XAML.


FEATURE: Tree decay == rabbit decay

Resolves #19
Starting to feel like a real game now

Thanks for the review


Bug in E3SM related to mapping check

/compyfs/liao313/00raw/mingpan_runoff/susquehanna/
(shr_strdata_init_streams)  fileName =            1
/compyfs/liao313/00raw/mingpan_runoff/susquehanna/ming_daily_1980.nc
(shr_strdata_init_streams)  timeName =            1 time
(shr_strdata_init_streams)   lonName =            1 lon
(shr_strdata_init_streams)   latName =            1 lat
(shr_strdata_init_streams)   hgtName =            1 unknownname
(shr_strdata_init_streams)  maskName =            1 unknownname
(shr_strdata_init_streams)  areaName =            1 unknownname
(shr_strdata_init_mapping)  calling shr_dmodel_mapSet for fill
(shr_strdata_init_mapping)  calling shr_dmodel_mapSet for remap
('shr_map_getWts') ERROR: yd outside bounds    42.7750000000000
('shr_map_getWts')        cpole =    1.57079632679490
ERROR: ('shr_map_getWts')  ERROR yd outside 90 degree bounds

This bug is seemly caused by the function:
logical function shr_map_checkRad(Grid)
! check if grid is rad or degree

implicit none
real(SHR_KIND_R8),intent(in) :: Grid(:,:)
character(*),parameter :: subName = "('shr_map_checkRad') "
real(SHR_KIND_R8) :: rmin,rmax

!-------------------------------------------------------------------------------

shr_map_checkRad = .false.
rmin = minval(Grid)
rmax = maxval(Grid)
if (rmax.ne.rmin) then
   shr_map_checkRad = ((rmax - rmin) < 1.01_SHR_KIND_R8*c2*pi)
else
   shr_map_checkRad = .true.
end if

end function shr_map_checkRad
If the domain is small, then this function falsely think the units are radian because the lat diff is less than 2pi.


[3.2] Optimized ScriptEditor initialization when many scripts are loaded

3.2 backport of #33760
CC @Paulb23
This change avoids the editor to freeze for several seconds when a
project with lots of scripts is loaded in the editor.
It focuses on a few heavy operations previously executed on all
previously loaded scripts:

Initialize script resource (script validation/parsing) only
on focus
ScriptTextEditor: code editor and edit menu are added to the
scene only on focus
Add to recent scripts only when opening new scripts
(load/save scene metadata)

Thanks!


TypeError: Cannot read property 'type' of undefined

TypeScript Version:  2.5.3
Trying to import a primus.js file generated by Primus throws the following error from Typescript (inside awesome-typescript-loader):
TypeError: Cannot read property 'type' of undefined
  at parseTypedefTag (C:\example\node_modules\typescript\lib\typescript.js:17815:136)
  at parseTag (C:\example\node_modules\typescript\lib\typescript.js:17558:39)
...

Here's the code at that line:
typedefTag.typeExpression = childTypeTag && !isObjectOrObjectArrayTypeReference(childTypeTag.typeExpression.type) ?
    childTypeTag.typeExpression :
    finishNode(jsdocTypeLiteral);

Code
I've attached my primus.js:
primus.zip
I believe this is where it's dying:
/**
 * @typedef ProtocolExtract
 * @type Object
 * @property {String} protocol Protocol matched in the URL, in lowercase.
 * @property {Boolean} slashes `true` if protocol is followed by "//", else `false`.
 * @property {String} rest Rest of the URL that is not part of the protocol.
 */

Rolling back to Typescript 2.5.2 doesn't give me this error.

Fixed by #19013.


[5eOGL]Added Spell Attack Bonus Modifier.html

added a Spell Attack Bonus Modifier since there was one for the Spell Save DC but not the Spell Attack Bonus.
I don't know why that wasn't there already it was literally 5 lines and someone had make the same changes for the Spell Save DC earlier
I haven't done anything on GitHub before so I hope I did things correctly?

Thank you for your suggestion, but I'm going to reject these changes at this time. We have tried to avoid adding too many options, like this, to the sheet for simplicity sake, but we have had this suggestion many, many times now. We planning a sheet update in the near future, and will consider adding this when we do that. For now, you can just manually add the spell bonus to each spell attack, which usually won't be an issue because you won't have a whole lot of them. If you are looking for more finer control over everything right now, you should check out the 5e Shaped sheet.
As for the fix being only 5 lines, this is not correct. There are translations and sheetworker updates to be handled as well as these changes.


need support for creating releases with file groups

PKS Releng needs to release the PKS Tile with product files grouped in folders
Context
The PKS Releng team uses the pivnet-resource in order to upload a PKS release to Pivnet.
Currently, our PM has to manually go in and create folders, and place our CLI dependencies in each folder.
Our metadata looks something like
product_files:
- file: tile/content/pks-linux-amd64-VERSION
  upload_as: PKS CLI - Linux
  file_type: "Software"
- file: tile/content/pks-darwin-amd64-VERSION
  upload_as: PKS CLI - Mac
  file_type: "Software"
- file: tile/content/kubectl-linux-amd64-VERSION
  upload_as: Kubectl VERSION - Linux
  file_type: "Software"
- file: tile/content/kubectl-darwin-amd64-VERSION
  upload_as: Kubectl VERSION - Mac
  file_type: "Software"

Ask
We would like to be able to configure file_groups inside our pivnet metadata file and have our CLI dependencies show up as two folders, one that contains all PKS CLI files for all OSes, and one that has all Kubectl files for all OSes.
A proposed manifest could be something like
product_files:
  - file_groups:
      - name: PKS CLI VERSION
          - file: tile/content/pks-linux-amd64-VERSION
            upload_as: PKS CLI - Linux
            file_type: "Software"
          - file: tile/content/pks-darwin-amd64-VERSION
            upload_as: PKS CLI - Mac
            file_type: "Software"
      - name: Kubectl VERSION
          - file: tile/content/kubectl-linux-amd64-VERSION
            upload_as: Kubectl VERSION - Linux
            file_type: "Software"
          - file: tile/content/kubectl-darwin-amd64-VERSION
            upload_as: Kubectl VERSION - Mac
            file_type: "Software"

Ramya (@rumshenoy) and Maryam

Thanks! This sounds like a good feature to have. I'll create a story and bring it to my PM's attention.

@xtreme-debbie-chen can you provide a link to the story?

I'm not sure you'll be able to see the story, but here's the link to it: https://www.pivotaltracker.com/story/show/161893236

Does this documentation help you with file group support?
https://github.com/pivotal-cf/pivnet-resource/tree/master/metadata

Hi @maryamklabib .  I'm just following up with you to see if you still need help with this question.


it's possible? layer for utf-strings

keras.engine.input_layer.InputUTF(utf_string, max_length=max_length)

Hi @nex0ma
You can't directly pass strings or text into Keras models, or really any DL/ML models. Instead, look at tokenizing the string, so you're passing a vector into the model. Keras does have a text preprocessing module that may be of interest to you here.
Let me know if you have any further questions.
Cheers
James

i mean that any string auto-tokenized by this layer. this is more simplest. char-embedding with tokenizer in examples is a bad way, using same tokenizer for another text experiment.
for deep learning i need one char-dictionary for YEARS of works with many texts on any languages.
so, any utf8 char will be encoded to integer automatically and will be one for all trainings.
thanks.


Allow to join page with custom HTTP client

It was possible to create a new page with custom HTTP client using agouti.NewPage, but was unable to join a running webdriver session with custom HTTP client: agouti.JoinPage did not offer a way to do that.
In order not to break an existing API, this PR introduces a new method to join the running session using custom HTTP client.

I think we can make this change to JoinPage without breaking the API (for the vast majority of usages, at least). Just add options ...Option as the final argument in the method signature. This method isn't defined on a type, so we won't break any interfaces. Someone could have theoretically stored a pointer to the JoinPage method with the previous signature, but that seems rather unlikely.
See NewPage for an example.

Looks great! Thanks for contributing!


Add a test to audit tool: fail if volume has no images

Audit tool should trigger an error if a volume only contains scan request files and no images
This issue arose out of https://github.com/buda-base/library-issues/issues/315

Write use case for this scenario, and consider making tests configurable per user since this test case isn't necessary for all users

This test only applies to the "images" folder for a given work.  If an image group folder under the "images" directory contains only 2 scan request files, and no images, then the audit test should fail.  why?  because if we sync a work with a volume that only contains scan request files it shows up on the website as having images, which isn't the case.

This test only applies to the "images" folder for a given work.  If an image group folder under the "images" directory contains only 2 scan request files, and no images, then the audit test should fail.  why?  because if we sync a work with a volume that only contains scan request files it shows up on the website as having images, which isn't the case.

@TBRC-Travis How can a machine distinguish between a scan images and regular images? What if an imagegroup contains only two monochrome images?

@TBRC-Travis How can a machine distinguish between a scan images and regular images? What if an imagegroup contains only two monochrome images?

I suppose we can't unless we want to start embedding some kind of distinguishing metadata in our scan request files.
IMHO this is a highly improbable edge case, so my vote would be to go ahead and let it fail and if there really was a two-page volume with monochrome images then allow for the test to be disabled after the fact.

I suppose we can't unless we want to start embedding some kind of distinguishing metadata in our scan request files.
IMHO this is a highly improbable edge case, so my vote would be to go ahead and let it fail and if there really was a two-page volume with monochrome images then allow for the test to be disabled after the fact.

Resolved: create a test for a minimum number of files in ArchiveParent and DerivedImageGroup parent. Test property is in shell.properties. Error can also be a warning.

I had to sort scan request pages for thumbnail generation, one easy way to check is the width / height in pixels

That works when you have reference materials, (i.e. pages which have
different aspect ratios)  but might fail when scan request page size ==
image page size.
Since audit tool is a desktop product as well as a centralized website
(soon), it has to be simple and reliable, and can't contain too many
heuristics about its content.
On Thu, Feb 4, 2021 at 11:11 AM Elie Roux<EMAIL_ADDRESS>wrote:

I had to sort scan request pages for thumbnail generation, one easy way to
check is the width / height in pixels
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/buda-base/asset-manager/issues/84#issuecomment-773423692,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AH7LSRL3CPDBWOP4XWCZK43S5LBKTANCNFSM4UMFLFZQ
.

There is a very reliable heuristic which doesn't involve examining a file's internals - it's the eXists database: IG XML DOC.  See  archive-ops #256 The return has image groups:
<work:volumeMap>
<work:volume imagegroup="I1PD34157" num="1"/>
<work:volume imagegroup="I1PD34158" num="2"/>
</work:volumeMap>

Querying each image group with the same query https://www.tbrc.org/xmldoc?rid=I1PD34157 gives the files in the image group, as well as the "intro images"
<imagegroup:images tbrcintro="2" total="0"/>

It should be simple enough to capture this output and make decisions on what is there. (Note there's a variant result which could have <imagegroup:images tbrcintro="2" total="2"/> in which case there's a pipe delimited list of images, which could be taken to be the scan request images.

Can we use a BUDA equivalent?

See buda-base/lds-queries#70 for solution.


No output after clicking "submit" in the configurator UI

The problem
After clicking on submit I don't get any further. Seems like there is an issue from my schools side. Even though everyone got personal accounts I can't add it to HA. I'm just wondering what the Problem could be since I'm not even getting an error in the UI configurator.
If you need further details let me know :)
Version of Home Assistant?
Core 2024.2.5 Supervisor 2024.02.1 Operating System 11.4 Frontend 20240207.1
What type of installation are you running?
Home Assistant Core
Version of WebUntis
v1.06
Last working WebUntis version
No response
Traceback/Error logs
No response
Additional information
No response

Do you get anything in the home assistant logs?

Logger: webuntis
Source: /usr/local/lib/python3.12/site-packages/webuntis/utils/logger.py:19
First occurred: 21:49:11 (26 occurrences)
Last logged: 21:59:10
{'jsonrpc': '2.0', 'id': '2024-03-03 21:57:19.686264', 'error': {'message': 'bad credentials', 'code': -8504}}
{'jsonrpc': '2.0', 'id': '2024-03-03 21:59:08.231613', 'error': {'message': 'bad credentials', 'code': -8504}}
{'jsonrpc': '2.0', 'id': '2024-03-03 21:59:09.064979', 'error': {'message': 'bad credentials', 'code': -8504}}
{'jsonrpc': '2.0', 'id': '2024-03-03 21:59:09.645925', 'error': {'message': 'bad credentials', 'code': -8504}}
{'jsonrpc': '2.0', 'id': '2024-03-03 21:59:10.217675', 'error': {'message': 'bad credentials', 'code': -8504}}
and
This error originated from a custom integration.
Logger: custom_components.webuntis.config_flow
Source: custom_components/webuntis/config_flow.py:54
Integration: webuntis (documentation, issues)
First occurred: 21:57:13 (1 occurrences)
Last logged: 21:57:13
Cannot resolve hostname: [Errno -2] Name does not resolve

{'jsonrpc': '2.0', 'id': '2024-03-03 21:57:19.686264', 'error': {'message': 'bad credentials', 'code': -8504}}
this error suggests that sth is wrong with the password. Do you have a password with special special characters like "ÄÜß"?

Nope I don't have special characters in my pw.

Try again in a few hours, this sometimes fixes the "Bad Credentials" error

Could characters like space cause this issue?

Somehow fixed itself after a few retries. Thanks a lot for the quick response and for setting up this whole integration!
BR


Nix/HM: add instructions for upstream module

Adds instructions on how to use the upstream Home Manager module for Hyprland.
Will probably undergo some changes as the HM module needs a few changes.
Fixes #272

Greetings to all who come across this issue. I'm writing this to open a discussion about proceeding with migration of users to one of the new Hyprland Home-Manager modules.
To bring any readers up to speed, there are currently three HM modules for Hyprland:

The official flake's nix/hm-module.nix (by @fufexan)
The one in HM added recently via #3837 (by @fufexan)
And my own (by @spikespaz)

A tale of fragmentation
To summarize the differences between the three:

The one in the official flake does little more than basic configuration. It handles the package to use (Git master by default), systemd integration, package options (xwayland, patches, and HiDPI), and plugin dynamically-linked libraries. It has no mechanism for generation of the config file ($XDG_CONFIG_HOME/hypr/hyprland.conf) and that file must either be manually created or linked in via the HM option xdg.configFile."hypr/hyprland.config".source.
The module by @fufexan in Home Manager uses the package in nixpkgs-unstable by default unless overridden in the HM input's follows. It expands upon (and changes some) features of the original Flake's module. Specifically, it add a the options settings and extraConfig to allow you to declaratively define your Hyprland config; this uses a simple function to stringify the Nix config as hyprland.conf. The Nix config might look something like this and the generated hyprland.conf looks like this. This module started out with significant issues, especially with ordering. Some things like variables and beziers must be defined before they are used (fixed in #4263). There are more caveats that are handled in the following module.
My module, which is currently a subtree of my dotfiles. This is by far the most complete of the three modules, and provides (in my opinion) a very nice API for users to declare their Hyprland config entirely in Nix syntax without excessive interpolation and merging. The goal is to allow everything to be defined in Nix in a way that feels natural. More on this later.

Module #3's adoption as official
In the past, @fufexan reached out to me to ask if my module could be adopted into the Hyprland flake to publicize it as official. I responded that I did not yet consider it to be ready for public adoption, as I acknowledged it is incomplete. I would consider it a prototype, albeit a fully functional one. After describing these issues, @fufexan concluded that it was "quite complex", and implied that a project of this intricacy is out of scope for the purposes of a sanctioned, official, module (I expect to be corrected). For this reason, I encouraged him to wait, and to instead test the module and provide feedback. Regardless of these issues, my module is by far the most feature-complete of the three. Because the scope of my project is wide, and I am ready and willing to accept the responsibility of maintainership, it would be a worthwhile effort to seek compromise.
My project is designed to be robust, extensible, and therefore future-proof as long as the current config format exists.
Problems with my module:

Lacking documentation, only a few examples (config.nix, monitors.nix keybinds.nix, and windowrules.nix).

I say this, but I did a good job with most of the the option docs. If only the task below were easy--

No statically generated module option documentation (potentially use rycee/nmd).
Missing type definitions, specifically for option keyBinds (uses attrsOf anything, should be a recursive submodule instead).
Unreasonable error reporting (if you get something wrong, you get a nonsensical Nix trace) for anything that isn't strictly defined (would be mitigated by solving the above issue).
Some internal functions could be simpler/optimized/eliminated.

However: the problem is inherently complicated. Details later.

The internal intermediate representation (essentially a node tree) is ill-defined. Currently it feels like a prototype (which it is).

Perhaps nodes should contain certain tokens, instead of every noun being a node.

Lackluster formatting options (related to #3), currently requires understanding the internals to configure (especially sorting, which is delicate and necessary).
Superfluous options (this is a point of contention, I want to keep--but possibly hide--the formatting options since otherwise they would just need to be hidden variables).
Opinionated. The file configRenames.nix is entirely pedantic; it serves little purpose other than to make some of the variable names (such as col.-prefixed ones) more compatible with the intended semantics to be expressed in the Nix config. Without this, some things would need to be quoted as attribute names, and this bothered me.
Config-reloading is buggy. Changes to keybinds defined with the custom syntax are not reloaded despite reloadConfig being true. This has to do with a limitation in HM's DAG, but I do have a nearly-complete work-in-progress solution in a Git stash.
A significant amount of time has passed since I originally drafted this module, and some important changes (namely plugins option) have been made to the current official module and the one in Home-Manager.
Biggest blocker: This maybe a point of contention, but I want to keep ownership and be the BDFL (coined by Linus). This entails that:

I approve the PRs (or Vaxry merges with mine or @fufexan's go-ahead).
The license is CC-BY which is chosen to allow derivative works, even under a different license, but require that attribution is given to me and @fufexan (who I presume will be a core contributor).
I would be amenable to adjustments as long as attribution is preserved.
This module is not to be merged into Home-Manager in any official capacity. This would still be technically allowed, but unsanctioned.

Considering #5 and #8: I disagree with the opinion that this module ought to be simple. The problem is fundamentally complex. The problem-objective is this: to convert a structured Nix expression--consisting of attribute sets and weak dynamic types--into a node-based document language that has no specification, a weak schema, and ambiguous literal syntax. This is inherently a complicated problem that requires a complicated solution. For those who are familiar with Serde, that project is complicated for similar reasons. In fact, this project was inspired by that one (also see xml-rs, knuffel, and kdl).
Why so much code?
To expand upon the above paragraph:

My Hyprland module for Home Manager consists of a library for serializing and formatting an attribute set into Vaxry's config language. This functionality is contained within configFormat.nix.
This design is heavily modeled after pkgs.formats (doc) and I believe is the most correct way of implementing a language serializer in Nix. Currently the formatter is not exposed as any lib attributes, though I wish to change that to allow greater extensibility (including custom-generated files as sources.
I do take come creative liberties to make the whole process conceptually simpler, otherwise it would reminiscent of a bowl of noodles. Because Vaxry's config language is essentially a node document with some additional semantics, I have chosen to model it as such. Currently (and temporarily) every section, indent, value, and variable assignment is a node of sorts, with formatting options for each type of node.
The target language is full of edge-cases: for example, key binds have submaps, and submaps are not a {}-encapsulated block, whereas in Nix they are. This decision was made to both make the code you write prettier, and to preserve variable/section ordering. This is because features were added to the config language ad-hoc, and (presumably) not entirely planned (no shade, Vaxry has more important things to do, like #2415).

Thoughts on this PR
Now, for my review of this PR and to summarize my position. I do not think that this should be merged without further significant revision. I admit to being biased, I think that my module is best.
I do not think it wise to encourage users to migrate their configs to the technically inferior format exposed by the Home-Manager module at present, it would be premature. Proceeding would would mean that, once a consensus is reached regarding integration of my module, users would need to migrate a second time; users would be less inclined to do so.
As it stands, it is fairly easy to migrate a Hyprland config to a Nix config using my module. This can be done with a few replacements using regular expressions, whereas if someone were to first move to the current HM-module, and then some time later to my format, they may have written some complicated intermediate functions that conflict with the design of my module. Rectifying this would entail reversing those changes--which are probably complicated Nix expressions--and then reformatting to meet the specifications defined by my module.
This would decrease users' willingness to change their own configs, which means less people using my module, which means less feedback, which means less improvement for those who do decide to migrate. As my module is specifically designed to be extensible, I think that patching features and fixes into an inferior module would be a mistake, as over time it would become convoluted and unmaintainable.
Instead, I think it would be best to discuss all of the issues illustrated above and proceed with moving users over once the entire adoption process has been wholly officiated.
I look forward to further discussion and am excited at the prospect of developing this more, thank you for your patience and tolerance.

For now, I encourage users to test out my module and provide feedback.
Here are some links:

Module: https://github.com/spikespaz/dotfiles/tree/master/hm-modules/hyprland
Examples: https://github.com/spikespaz/dotfiles/tree/master/users/jacob/desktops/hyprland
Flake instructions: Will update once in README.md.

@fufexan Do you have any comments/suggestions about the module I made? What are your requirements for using it?

@fufexan This is an entirely separate codebase, so not what you want to comment on. Open an issue on https://github.com/spikespaz/hyprland-flake and put your suggestions there. I'll have time to work on it between plane flights.

Between the hm-module in this repo and the one on HM repo which one should is preferred to use? It seems that the one in home-manager has been updated more recently.

@andresilva That's up to you. I'm trying to get people to try out the one I made, as Fufexan has made room for it in his heart. If I know it works for people and can call it stable, I will move it to this hyprwm organization.
Until someone besides me is using it though, I cannot do this in good conscience.

Hey all! Came across this so thought I would drop my two cents:
I think the best approach would be to adopt @fufexan's upstreamed hm module.

Keep hm-module.nix exactly the same as HM's upstream module
Only diverge (preferrably never diverge on interface, only in implementation) if a hyprland package absolutely requires a change in the module

Breaking changes should be avoiding by making sure the module follows RFC42 (which fufexan's HM one already does)
In this case, also upstream the change to HM and merge it once nixos-unstable gets the updated package. Then eventually the modules will be in sync again.

This means that people following master should usually use hyprland's provided modules, and changing between master pkg+master module and nixpkgs pkg+hm module shouldn't break their setups.

@spikespaz's module looks nice and should absolutely be an alternative for those who are interested in a more tightly integrated solution (we should link to it in the docs). But, in my opinion, it goes against the general (nixos and hm) trend of using RFC42 to make modules easier to maintain and closer to the actual config generated. Following the nixos and hm trend is, in my opinion, the way to go, as it lowers the overall complexity of setting up it (as it feels more similar to other stuff, and less of an additional DSL you have to learn).

I think that #4004 supercedes this issue.


Typehint mapping and setter injection

I have a question/issue about/with Typehint mapping and setter injection.
I implement an interface on all of my handler classes:
    _instanceof:
        AppBundle\Service\HandlerInterface:
            tags:
                - { name: tactician.handler, typehints: false }
            autowire: true
            public: true

All goes fine by this.
But one handler requires a setter injection, for example.
    /**
     * @param EntityManagerInterface $em
     *
     * @required
     */
    public function setEm(EntityManagerInterface $em): void
    {
        $this->em = $em;
    }

The problem here is that the findCommandsForService in TypeHintMapping will find this function and tries to map it.
This will trigger the following error:
Can not route Doctrine\ORM\EntityManagerInterface to AppBundle\Service\Membership\MembershipRegistrationHandler, class Doctrine\ORM\EntityManagerInterface does not exist!

Because of
if (!class_exists($commandClassName)) {
    throw new InvalidArgumentException("Can not route $commandClassName to $serviceId, class $commandClassName does not exist!");
}

I could create a custom mapping, which only checks for a handle function instead of all other function in my class. But is this the expected behaviour? Or should we filter out functions with interface typehinting?
Thanks!

Hi @jverdeyen, thanks for opening such a detailed issue, much appreciated! :)
This is definitely not the intended behavior, we tried to exclude a lot of edge cases for this but interface typehints slipped through, my apologies!
If you (or anyone else) is interested, this seems like a fairly straightforward one to fix, we need an extra check here: https://github.com/thephpleague/tactician-bundle/blob/master/src/DependencyInjection/HandlerMapping/TypeHintMapping.php#L56 Probably something like $parameter->getClass()->isInterface() plus an extra test.
If you'd like to give it a go, let me know. Otherwise, I'll pick it up when I can but that may be a little while 😅
Also, purely from curiosity and no judgment intended, I'm curious why any handler services need setter injection as opposed to constructor injection? I'm a big fan of all required deps being passed in there and would like to make it easy, so if there's any shortcomings or edge cases, I'd love to see if we can improve them.
Thanks again for such a nicely researched issue and for using this bundle. :)

@rosstuck thanks for your feedback!
I'll try to create a PR with this solution and a test.
I'm using a EntityManagerAwareTrait in my project. The only was to properly inject the em by a trait is using a setter injection. This way it doesn't change the constructor of the classes using this trait. Might not be a good practice, but in rapid development it's a win-win.


Minor improvements on Angular snippets

Hey there !
I reached you out on Discord about some issues that I found on the Angular templates
I took the liberty to fix the other I noticed in the other Angular snippets, I also added a snippet for the Slot Fallback section.
My changes are:

Formatting (one liners in .html has been moved to template: '...'
Improvements (removed unnecessary Angular code when there was some, like setting the component change strategy)
Missing snippets (the Slot Fallback one)

Please feel free to tell me if you want me to update anything else on the changes I made !

Amazing ! You saw so many improvements !
Good thing about one-liners, the purpose was to show that you can declare HTML markup in a separate file.

Understood, I let the templates containing more than one line into their dedicated .html files and also fixed the Angular example for the content projection fallback to match the other ones
Let me know if anything else need an update !

Just perfect 👌
Thank you !

@pBouillon cc @matschik the Slot Fallback example is not correct.
<button #container>
  <ng-content></ng-content>
</button>

<button *ngIf="!container.hasChildNodes()">
  No content found
</button>

The first button would still show up as an empty button

Indeed it does, I missed that one, sorry about that
@nartc's suggestion would surely be better
Would you like to create the new PR with your code? Otherwise I can create it if you prefer!

@pBouillon I can send a PR later today. If you're available and want to PR, don't let me stop you :D

@nartc Had some time to spare, I just submitted it, hope I get it right this time 🤞


Patch AsyncTask errors on BukkitSchedulerMock (v1.16)

Description
This PR adds patches applied on v1.18 branch to v1.16 branch concerning BukkitSchedulerMock and its failures about AsyncTasks' stopping
Checklist
The following items should be checked before the pull request can be merged.

[x] No unit tests added but updated older one
[x] Code follows existing code format.

Closing as this doesn't actually fix the issue. #536 should do the trick.


Class based embed finders and new embed finder configuration

Based on #1877
This pull request implements class based embed finders and a new embed finder configuration. I hope this will make embed finders more configurable solving the following problems:

Allow additional oembed providers to be used (currently all are hardcoded)
Make per-provider configuration possible

Choose which finders get used for which provider (eg, force use of oembed for instagram but use embedly for everything else)
Allow specifying additional parameters for certian providers such as scheme=https to YouTube

For example, the below configuration would use oembed for fetching youtube videos and embedly for everything else. All youtube videos would be fetched with scheme=https. It also adds a custom oembed provider too.
from wagtail.wagtailembeds.oembed_providers import youtube, all_providers

ISSUU = {
    'endpoint': 'http://issuu.com/oembed',
    'urls': [
        '^http(?:s)?://(?:www\\.)?issuu\\.com/[^#?/]+/docs/.+$',
    ]
}

WAGTAILEMBEDS_FINDERS = [
    {  # Override Youtube oEmbed provider to pass through an extra option
        'class': 'wagtail.wagtailembeds.finders.oembed',
        'providers': [youtube],
        'options': {'scheme': 'https'}
    },
    {  # Then try all other builtin oEmbed providers
        'class': 'wagtail.wagtailembeds.finders.oembed'
    },
    {  # Custom oEmbed provider
        'class': 'wagtail.wagtailembeds.finders.oembed',
        'providers': [ISSUU]
    },
    {  # Fallback to Embedly
        'class': 'wagtail.wagtailembeds.finders.embedly',
        'key': 'mykey'
    },
]

How would we embed for example an iframe widget from a specific site that has no oembed support? So, one would enter the url, and Wagtail would need to turn that into an iframe somehow. Could I do that with a custom embed finder?

@balazs-endresz yep, you just need to subclass EmbedFinder and implement the accept(url) (returns True if the finder can handle the passed URL but does not perform a request) and find_embed(url, max_width) (fetches the embed and returns its contets) methods.
You can then add it to WAGTAILEMBEDS_FINDERS like so:
WAGTAILEMBEDS_FINDERS = [
    {
        'finder': 'mysite.myapp.finders.CustomEmbedFinder',
        # Any extra arguments here will be passed as keyword arguments to the embed finder's constructor
    }
]

I'll write up some docs about that when I get a moment

Rebased but not had a chance to document this yet

Docs done, ready for review!

@tomdyson tweaks made, thanks!

Do this make deprecated packages functionality like wagtail-embed-videos?

The same question.

@gasman Rebased, made suggested changes and also tweaked the docs bit. Ready for another review!

@gasman I've switched to use your rebased version and fixed https://github.com/wagtail/wagtail/pull/2127#discussion_r124547737

Merged in 01e2d5f4a0e94fcb5fdaee0dc8ea93455b6a00ab - thanks @kaedroho!


Wrong output in multi-array set

This is the JSON:
[{
   "orderId": "111-8763976-4506632",
   "version3.0": {
      "customizationInfo": {
         "surfaces": [
            {
               "name": "Angled Above Handle",
               "areas": [
                  {
                     "name": "Choose Knife",
                     "optionImage": ""
                  }
               ]
            },
            {
               "name": "Right Face",
               "areas": [
                  {
                     "name": "Right Face Engraving",
                     "optionImage": "https://m.media-amazon.com/images/S/pc-vendor-gallery-prod/A17JMGFJQ7S29T/options/2018-08-24/ef64d10d-7668-42f1-a0ea-6c9971c13a0f.png"
                  }
               ]
            }
         ]
      }
   }
}]

and the output we are getting is

See the column "customizationInfo.surfaces.areas.optionImage". The value should be there on the 2 row instead of first
Thanks

@anuj1405 this is a feature. please check https://github.com/kauegimenes/jsonexport/issues/22
You should use fillGaps: true to get the optionImage filled in both rows

Oh now i see whats happening, indeed this is a bug. and its prob caused by #22

We are always looking for new contributors, you want to help us fix this issue? feel free to start a PR on this =)

I've got similar problem.  I've set FillGaps = true, but subscriber1 CustomFields values not shifting down in CSV. The JSON data is from Campaign Monitor API.  Simple input and output files attached to demonstrate problem.  What support do you need to fix the bug?
Archive_CM.zip

This is still broken. Renders jsonexport unusable for my usecase.

@anuj1405 @markwynter @toxuin I introduced a new option to jsonexport called fillTopRow that should be desabled by default.
After this PR gets merged you guys should be able to get the expected results.


"Register" Button on Login page Does not Show in Chrome, Win 10

Bug
A clear and concise description of what the bug is.
To Reproduce
Go to login page. Register button does not appear
Expected behavior
Register button should appear in the bottom-left.
Screenshots

Desktop (please complete the following information):

Win 10, Chrome Version 80.0.3987.149 (Official Build) (64-bit)

Tobi pushed an update for this. Marking closed. Will reopen if we receive more field reports.


Search Feature Doesn't Return Pages

Details
When compiling the documents locally, the search feature works as expected, however when compiled by ReadTheDocs, the search feature doesn't return any pages.

Read the Docs project URL: https://readthedocs.org/projects/nathaniel-reference/
Build URL (if applicable): https://nathaniel-reference.readthedocs.io/en/latest/
Read the Docs username (if applicable): Nathaniel Hoefer

Expected Result
When utilizing the search feature found in the top left corner of my webpage, the pages with the searched keyword to be listed
Actual Result
No associated pages are listed, no matter the search.

I just search something on your project and it works as expected. Maybe a cache issue?

That's exactly what it was. Thanks for your help!


Does GitRepositoryCloner need the target directory to exist in advance?

Not strictly, no. But removing and then creating catches any permissions issues cleanly.


doesnt let me copy

If I open a file in view-tail-large-files it will not let me copy selected text

Copying is added in 1.0.8.  From the readme...
"When you click anywhere in the text and drag the mouse, text will be selected and immediately copied to the clipboard.  Only lines can be selected at this time".


As a user, I want changes to Opportunities to immediately refresh rollups for all household members, so I don't see incorrect data

Currently, when opps are inserted one at a time, we correctly update rollups for all household members.  But this code does not fire if the opp is just being updated.  Ideally it would also cause rollups to refire, at least in the case where only 1 opp is being updated.

**lurch: attach W-007360

Tracking W-007360

This has been marked as "Never" internally.


fix docker compose env variable modification

Fix in the docker compose file has the env variable for the list of authorized district has been modified from IDEFIX_BANID_DISTRICTS to IDEFIX_BANID_DISTRICTS_COG

closed : env variable deleted


document clock sync

Clock synchronization and related cross-origin issues should be documented.

Clock sync for live streams uses the Date header from the manifest response.  If this is served cross-origin, there is extra configuration required on the server in order for the client to be able to read this header.


Create python-package-conda.yml

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

[x] Make sure to open an issue as a bug/issue before writing your code!  That way we can discuss the change, evaluate designs, and agree on the general idea
[x] Ensure the tests and linter pass
[x] Code coverage does not decrease (if any source code was changed)
[x] Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> 🦕

@Rockyspade PR update workflow merge My-branch head patch1


Turbo doesn't respect package names

What version of Turborepo are you using?
1.3.1
What package manager are you using / does the bug impact?
npm
What operating system are you using?
Mac
Describe the Bug
Taking the default installation, I changed the name of the ui package to @web/ui because I want my packages namespaced.
In the default web/pages/index.tsx I changed the package name, e.g. import { Button } from "@web/ui";. I then restarted the TS server.
The result is a TS error:
Cannot find module '@web/ui' or its corresponding type declarations.

Expected Behavior
Turbo should understand the names of the packages from the package.json.
To Reproduce

Take a default installation
Change the name of the ui package to @web/ui in the package.json
In web/pages/index.tsx change the import name, e.g. import { Button } from "@web/ui";.

@andyjessop I ran into exactly issue, may I ask how did you solve it? I have nothing fancy, mostly copy from the example.

I'm really sorry, I should have posted my answer here before closing. I'm afraid I don't remember what the answer was. Maybe you can re-open to get an answer from the team.

I've been testing namespaced packages, but installation appears to favour NPM vs local package names.
Some additional documentation around this would be useful @mattpocock

Don’t you need to reinstall the packages, e.g. pnpm install when you rename packages?

FWIW I found this post, the last answer may be helpful with regards to scoped packages (I've not tested myself yet):
https://stackoverflow.com/questions/71293685/npm-workspaces-warn-some-package-name-in-filter-set-but-no-workspace-folder-p

@weyert is correct.


Unique assembly per identity per context

I think the idea is to load a unique assembly per identity per context, so it's only relevant if the same version is already loaded in the load-from context, not the load context.

@GrabYourPitchforks and @steveharter Can you please comment on this suggested docs change? Thanks!

Sent email to @steveharter

Thanks for the contribution @lbargaoanu


win,async: refactor internal implementation

While trying to remove no longer needed code I noticed some edge cases not being handled in uv_async_t, so I sat to fix them. I tried a couple of approaches and settled for this one, which uses the same model we use on Unix (sort of). I find the code is easier to follow now, seems to perform on the same ballpark and we can now run all async benchmarks without running into assertions.
I also suspect some weird threadpool tests failing could be because of this.
R= @libuv/collaborators and maybe @orangemocha or @joaocgreis ?

LGTM

Oh, thanks a lot for the reviews on this fine Tuesday afternoon, Ben!


load h5py error

Thank you for your code! When I run test.py, it works well. However, when I use pretrained model, it raises the following errors:  (I use pytorch in Windows, I don't known if this will affect something)
python main.py --cuda --pretrained model/model_epoch_50.pth
Namespace(batchSize=128, clip=0.4, cuda=True, lr=0.1, momentum=0.9, nEpochs=50, pretrained='model/model_epoch_50.pth', resume='', start_epoch=1, step=10, threads=1, weight_decay=0.0001)
Random Seed:  9559
===> Loading datasets
===> Building model
===> Setting GPU
=> loading model 'model/model_epoch_50.pth'
Y:\Software\Anaconda3\lib\site-packages\torch\serialization.py:286: SourceChangeWarning: source code of class 'torch.nn.parallel.data_parallel.DataParallel' has changed. you can retrieve the original source code by accessing the object's source attribute or set torch.nn.Module.dump_patches = True and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
Y:\Software\Anaconda3\lib\site-packages\torch\serialization.py:286: SourceChangeWarning: source code of class 'vdsr.Net' has changed. you can retrieve the original source code by accessing the object's source attribute or set torch.nn.Module.dump_patches = True and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
Y:\Software\Anaconda3\lib\site-packages\torch\serialization.py:286: SourceChangeWarning: source code of class 'torch.nn.modules.container.Sequential' has changed. you can retrieve the original source code by accessing the object's source attribute or set torch.nn.Module.dump_patches = True and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
Y:\Software\Anaconda3\lib\site-packages\torch\serialization.py:286: SourceChangeWarning: source code of class 'torch.nn.modules.conv.Conv2d' has changed. you can retrieve the original source code by accessing the object's source attribute or set torch.nn.Module.dump_patches = True and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
Y:\Software\Anaconda3\lib\site-packages\torch\serialization.py:286: SourceChangeWarning: source code of class 'torch.nn.modules.activation.ReLU' has changed. you can retrieve the original source code by accessing the object's source attribute or set torch.nn.Module.dump_patches = True and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
===> Setting Optimizer
===> Training
Epoch=1, lr=0.1
Traceback (most recent call last):
File "main.py", line 128, in
main()
File "main.py", line 83, in main
train(training_data_loader, optimizer, model, criterion, epoch)
File "main.py", line 101, in train
for iteration, batch in enumerate(training_data_loader, 1):
File "Y:\Software\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 301, in iter
return DataLoaderIter(self)
File "Y:\Software\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 158, in init
w.start()
File "Y:\Software\Anaconda3\lib\multiprocessing\process.py", line 105, in start
self._popen = self._Popen(self)
File "Y:\Software\Anaconda3\lib\multiprocessing\context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "Y:\Software\Anaconda3\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "Y:\Software\Anaconda3\lib\multiprocessing\popen_spawn_win32.py", line 65, in init
reduction.dump(process_obj, to_child)
File "Y:\Software\Anaconda3\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
TypeError: can't pickle h5py.h5d.DatasetID objects
Y:\Pytorch_project\from_github\VDSR>Traceback (most recent call last):
File "", line 1, in
File "Y:\Software\Anaconda3\lib\multiprocessing\spawn.py", line 105, in spawn_main
exitcode = _main(fd)
File "Y:\Software\Anaconda3\lib\multiprocessing\spawn.py", line 115, in _main
self = reduction.pickle.load(from_parent)
EOFError: Ran out of input

@ZhaoJinHA I haven't tried the script on windows, but it should work on linux,
check your data and can change model = torch.nn.DataParallel(model).cuda()  to  model = model.cuda(), see if it works

@twtygqyy Thank you for your advice. I did what you sayed, but still get error. I think this error is mainly related to data loading part, because test.py can work well. Now I only run main.py without pretrained, the error raises like follow. Maybe it's result from the difference between Window and Linux. Or I made other mistakes? How do you think of this?
Y:\Pytorch_project\from_github\VDSR>python main.py --cuda
Namespace(batchSize=128, clip=0.4, cuda=True, lr=0.1, momentum=0.9, nEpochs=50, pretrained='', resume='', start_epoch=1, step=10, threads=1, weight_decay=0.0001)
Random Seed:  5261
===> Loading datasets
===> Building model
===> Setting GPU
===> Setting Optimizer
===> Training
Epoch=1, lr=0.1
Traceback (most recent call last):
File "main.py", line 135, in
main()
File "main.py", line 88, in main
train(training_data_loader, optimizer, model, criterion, epoch)
File "main.py", line 108, in train
for iteration, batch in enumerate(training_data_loader, 1):
File "Y:\Software\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 301, in iter
return DataLoaderIter(self)
File "Y:\Software\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 158, in init
w.start()
File "Y:\Software\Anaconda3\lib\multiprocessing\process.py", line 105, in start
self._popen = self._Popen(self)
File "Y:\Software\Anaconda3\lib\multiprocessing\context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "Y:\Software\Anaconda3\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "Y:\Software\Anaconda3\lib\multiprocessing\popen_spawn_win32.py", line 65, in init
reduction.dump(process_obj, to_child)
File "Y:\Software\Anaconda3\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
TypeError: can't pickle h5py.h5d.DatasetID objects
Y:\Pytorch_project\from_github\VDSR>Traceback (most recent call last):
File "", line 1, in
File "Y:\Software\Anaconda3\lib\multiprocessing\spawn.py", line 105, in spawn_main
exitcode = _main(fd)
File "Y:\Software\Anaconda3\lib\multiprocessing\spawn.py", line 115, in _main
self = reduction.pickle.load(from_parent)
EOFError: Ran out of input

@twtygqyy Finally, I find the solution is very simple: just change "training_data_loader = DataLoader(dataset=train_set, num_workers=opt.threads , batch_size=opt.batchSize, shuffle=True)" to
"training_data_loader = DataLoader(dataset=train_set, batch_size=opt.batchSize, shuffle=True)". After this change, it can work on windows. Thank you again for your code : )

@ZhaoJinHA Good to hear that you solved the issue!


Recreate of 'fix an error in the regex urlRegEx' PR

I recreated PR, with changes from https://github.com/guybedford/require-css/pull/97 PR.
Decided to choose @dataway variant, but I agree with @fearphage there is for sure simplified version. And we will work it later.
Here is screenshot of RegExr.com, how pattern works:

During this PR, I added 2 new test cases:
assert(
    '@import statements with url()',
    normalize('@import url("test.css")', '/first/', '/second/'),
    '@import url("../first/test.css")'
);
// so that to test combination of @import and url()

assert(
    'multiple url includes on the same line (mixed)',
    normalize('src: url("../fonts/font.eot") format("embedded-opentype"), url("http://server.com/opensans.woff") format("woff");', '/base/', '/'),
    'src: url("fonts/font.eot") format("embedded-opentype"), url("http://server.com/opensans.woff") format("woff");'
);
// so that to test combination of absolute and protocol-based urls.

Her is test results:
--- Starting Require CSS Tests ---

Testing URL Base Conversions
  Changing subfolder... passed.
  Changing subfolder with backtrack... passed.
  Changing two subfolders with a folder... passed.
  Double forward slashes in relative URI... passed.
  protocol base urls work... passed.
  absolute protocol paths work with base conversion... passed.

--- Require CSS Tests Complete: 15 passed, 0 failed. ---

In scope of this PR, I also added requirejs to devDependencies, because it's needed by command node test/test.js aliased as npm test (from previous commits). Yes, I know, there is require-css/test/maxStylesTest/lib/require.js and I know it's the same version 2.1.10. And I did it on purpose. So far, I can't dramatically change lot of things I see in require-css module, but I will do it step-by-step.

PR was not merged in fact. I rebased my branch with latest master, and git decided it's now the same. I will create new branch and new PR.


fix(nodebuilder/state): overwrite state flags only if they were passed explicitly

Closes #3661

Codecov Report
Attention: Patch coverage is 0% with 4 lines in your changes missing coverage. Please review.

Project coverage is 45.73%. Comparing base (2469e7a) to head (61b71cd).
Report is 165 commits behind head on main.

Files
Patch %
Lines

nodebuilder/state/flags.go
0.00%
2 Missing and 2 partials :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3663      +/-   ##
==========================================
+ Coverage   44.83%   45.73%   +0.90%
==========================================
  Files         265      281      +16
  Lines       14620    16042    +1422
==========================================
+ Hits         6555     7337     +782
- Misses       7313     7864     +551
- Partials      752      841      +89

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


No module named 'ftx.medical' in spacy/spacy_processors.py #92

This PR fixes #92.
Description of changes
Describe what are the changes, and how they solve the problem.
Possible influences of this PR.
Describe what are the possible side-effects of the code change.
Test Conducted
Describe what test cases are included for the PR.

This will be a redundant change, since PR #89 and issue #87 already address this problem and the PR is in review.

#89

Hi, Piyush, thanks for this. I just check I have a review draft sitting there but wasn't submitted for some reason. Just submitted the review.

close since duplicated with https://github.com/asyml/forte-wrappers/pull/89


ARROW-5651: [Python] Fix Incorrect conversion from strided Numpy array

Fix the issue described in https://issues.apache.org/jira/browse/ARROW-5651
Problem
pa.array gives wrong results for numpy arrays with a non-trivial stride when the type is different from the original one.
>>> np_arr = np.arange(0, 10, dtype=np.float32)[1:-1:2]
>>> print(np_arr)
[1. 3. 5. 7.]
>>> pa_arr = pa.array(np_arr, type=pa.float64())
>>> print(pa_arr)
[
  1,
  2,
  3,
  4
]

Cause
The problem arises due to the function CopyStridedArray, which is supposed to copy the elements of a given numpy array with a non-trivial stride to a contiguous memory buffer. The current implementation does not deliver the expected result.
Tracing the execution of the above example with gdb shows that the function just copies all elements of the given numpy array.
$ vim sample.py
$ cat sample.py
import numpy as np
import pyarrow as pa
np_arr = np.arange(0, 10, dtype=np.float32)[1:-1:2]
pa_arr = pa.array(np_arr, type=pa.float64())
$ gdb --args python sample.py
... several commands to reach the invocation of `CopyStridedArray`
(gdb) bt 1
#0  arrow::py::NumPyConverter::PrepareInputData<arrow::DoubleType> (this=0x7fffffffdc50, data=0x7fffffffda40)
    at /vagrant/arrow/cpp/src/arrow/python/numpy_to_arrow.cc:429
(gdb) li 429
424	    // TODO
425	    return Status::NotImplemented("Byte-swapped arrays not supported");
426	  }
427
428	  if (is_strided()) {
429	    RETURN_NOT_OK(CopyStridedArray<ArrowType>(arr_, length_, pool_, data));
430	  } else if (dtype_->type_num == NPY_BOOL) {
431	    int64_t nbytes = BitUtil::BytesForBits(length_);
432	    std::shared_ptr<Buffer> buffer;
433	    RETURN_NOT_OK(AllocateBuffer(pool_, nbytes, &buffer));
(gdb) print PyArray_DATA(arr_)
$7 = (void *) 0x5555560a1834
(gdb) x/32bx  0x5555560a1834
0x5555560a1834:	0x00	0x00	0x80	0x3f	0x00	0x00	0x00	0x40
0x5555560a183c:	0x00	0x00	0x40	0x40	0x00	0x00	0x80	0x40
0x5555560a1844:	0x00	0x00	0xa0	0x40	0x00	0x00	0xc0	0x40
0x5555560a184c:	0x00	0x00	0xe0	0x40	0x00	0x00	0x00	0x41
(gdb) n
(gdb) print (*data)->size_
$12 = 32
x/32bx (*data)->mutable_data_
0x7fffee409000:	0x00	0x00	0x80	0x3f	0x00	0x00	0x00	0x40
0x7fffee409008:	0x00	0x00	0x40	0x40	0x00	0x00	0x80	0x40
0x7fffee409010:	0x00	0x00	0xa0	0x40	0x00	0x00	0xc0	0x40
0x7fffee409018:	0x00	0x00	0xe0	0x40	0x00	0x00	0x00	0x41

However, the expected result is a contigous array of 1, 3, 5, 7 encoded in float32 type.
(gdb) print (*data)->size_
$12 = 16
(gdb) x/16bx (*data)->mutable_data_
0x7fffee409000:	0x00	0x00	0x80	0x3f	0x00	0x00	0x40	0x40
0x7fffee409008:	0x00	0x00	0xa0	0x40	0x00	0x00	0xe0	0x40

Note that float32 encoding of 1, 2, 3, 4, ... are
1.0: 0x00	0x00	0x80	0x3f
2.0: 0x00	0x00	0x00	0x40
3.0: 0x00	0x00	0x40	0x40
4.0: 0x00	0x00	0x80	0x40
...

Solution
As described in the ticket, the wrong behavior of the function CopyStridedArray could be fixed by changing the type T declared in the function not to take the type of the output type (float64 in the above example) but to take the input type (float32 in the above example). However, I could not change the behavior of the function in this way because the function does not take the type of the input array explicitly. Instead, I used the itemisize_ retrieved from the input array to get the type (size) of each element of the input and passed it to the CopyStridedArray.
Use of itemsize instead of type T forces me to remove function CopyStridedNatural because reinterpret_cast<T*> cannot not be used because the type T is necessary.
Note
AMD64 Conda C++ Test fails now, but it seems unrelated to this PR.

@pitrou Thank you for your review!
Here are some benchmarks on Ubuntu virtual environment running on MacbookPro.
Current master
In [4]: a = np.arange(100000, dtype=np.float32)[1:-1:2]
In [5]: strided = a[-1:1:-1]
In [6]: %timeit b = pa.array(a)
The slowest run took 8663.68 times longer than the fastest. This could mean that an intermediate result is being cached.
1 loop, best of 3: 129 µs per loop
In [7]: %timeit b = pa.array(a, type=pa.float64())
The slowest run took 88.79 times longer than the fastest. This could mean that an intermediate result is being cached.
1000 loops, best of 3: 299 µs per loop
In [8]: %timeit b = pa.array(strided)
10000 loops, best of 3: 134 µs per loop
In [9]: %timeit b = pa.array(strided, type=pa.float64())
1000 loops, best of 3: 287 µs per loop

This PR
In [4]: a = np.arange(100000, dtype=np.float32)[1:-1:2]
In [5]: strided = a[-1:1:-1]
In [6]: %timeit b = pa.array(a)
The slowest run took 962.69 times longer than the fastest. This could mean that an intermediate result is being cached.
10000 loops, best of 3: 155 µs per loop
In [7]: %timeit b = pa.array(a, type=pa.float64())
The slowest run took 67.47 times longer than the fastest. This could mean that an intermediate result is being cached.
1000 loops, best of 3: 304 µs per loop
In [8]: %timeit b = pa.array(strided)
10000 loops, best of 3: 157 µs per loop
In [9]: %timeit b = pa.array(strided, type=pa.float64())
1000 loops, best of 3: 306 µs per loop

The warning The slowest run took ... times longer than the fastest may be caused by the environment (VirtualBox) running the benchmarks. To determine the actual causes, I plan to try the benchmarks on another environment in a few days.
Above benchmarks show that this PR probably introduces performance regression. If the regression is real and unacceptable, I think it would be better to close this PR and try another. What would you think?

The warning The slowest run took ... times longer than the fastest may be caused by the unstableness? of the environment (VirtualBox) running the benchmarks.

Right, I think that's probably the explanation. I will try to run them here.

Here are some benchmarks on Ubuntu 18.04. On master:
>>> a = np.arange(100000, dtype=np.float32)
>>> strided = a[-1:1:-1]
>>> %timeit b = pa.array(a)
9.07 µs ± 29 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
>>> %timeit b = pa.array(strided)
52.5 µs ± 94.3 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
>>> %timeit b = pa.array(a, type=pa.float64())
35.2 µs ± 66 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
>>> %timeit b = pa.array(strided, type=pa.float64())
77 µs ± 101 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)

On this PR:
>>> a = np.arange(100000, dtype=np.float32)
>>> strided = a[-1:1:-1]
>>> %timeit b = pa.array(a)
9.11 µs ± 47.3 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
>>> %timeit b = pa.array(strided)
288 µs ± 203 ns per loop (mean ± std. dev. of 7 runs, 1000 loops each)
>>> %timeit b = pa.array(a, type=pa.float64())
36 µs ± 64.9 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
>>> %timeit b = pa.array(strided, type=pa.float64())
312 µs ± 128 ns per loop (mean ± std. dev. of 7 runs, 1000 loops each)

So there's a very large slowdown for strided inputs, even when no type conversion is requested. It would be nice if you could try to improve the situation.

(side note: benchmarks should be run on a release build. Your numbers suggests that you may have run them on a debug build. I'm guilty of the same error from time to time :-))

Codecov Report

Merging #4958 into master will increase coverage by 1.61%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #4958      +/-   ##
==========================================
+ Coverage    87.5%   89.11%   +1.61%
==========================================
  Files         998      721     -277
  Lines      141784   101727   -40057
  Branches     1418        0    -1418
==========================================
- Hits       124065    90658   -33407
+ Misses      17357    11069    -6288
+ Partials      362        0     -362

Impacted Files
Coverage Δ

python/pyarrow/tests/test_pandas.py
95.15% <100%> (+0.01%)
:arrow_up:

cpp/src/arrow/python/numpy_to_arrow.cc
95.26% <100%> (-0.17%)
:arrow_down:

cpp/src/arrow/array/builder_adaptive.h
93.75% <0%> (-3.13%)
:arrow_down:

cpp/src/arrow/ipc/reader.cc
89.09% <0%> (-0.14%)
:arrow_down:

cpp/src/plasma/client.cc
96.5% <0%> (ø)
:arrow_up:

cpp/src/parquet/arrow/arrow-reader-writer-test.cc
93.71% <0%> (ø)
:arrow_up:

python/pyarrow/serialization.py
84.76% <0%> (ø)
:arrow_up:

cpp/src/plasma/test/client_tests.cc
99.09% <0%> (ø)
:arrow_up:

go/arrow/ipc/writer.go

js/src/util/fn.ts

... and 281 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 38b0176...486729f. Read the comment docs.

@pitrou Thank you for benchmarking my PR and for your kind advice. Because I assume a patch that can mitigate the deceleration would be very different from this patch, I close this PR for now.
If I can find a way to fix the main issue and the deceleration problem, then I will propose that with another PR.


pwm specification should comment on the supported range of clocks

The PWM specification should comment on what range of clocks are supported.
it should also comment on the relationship between TL <-> core clock.
this is needed to know what range the clocks should be used to verify the CDC

estimate 1

(Coming back to this after the V2 review)
The CDC is now managed by the reg-tool interface generator.  That said, do we need any special documentation for this?

@msfschaffner @tjaychen do you have any insight on this?

@martin-lueker I think this issue can be closed- the responsibility of verifying the CDC is now somewhere else

Triaged for pwm. Assigning to M2.5 with https://github.com/lowRISC/opentitan/labels/Priority%3AP3 because as per the last comment:

"There's no clock frequency requirement anymore that we know of." So documentation needs to be updated but that could happen post M2.5.
Block-level DV should check this: The clocks are being randomized, and we collect coverage for it in clock_cg, which is currently at 100%. However, the TL clocks are mapped to a single bin, so that bin and the cross with the core clock is not very meaningful. This should be improved to verify this more thoroughly.

This is low-prio and not needed for M2.5 hence deferring.

Moving to M5 based on priority.


Update CyberArkEPMSentinelConn.zip

Required items, please complete
Change(s):

Repackage distribution of CyberArkEPMSentinelConn.zip as it had wrong structure

Reason for Change(s):

The function CyberArkEPM was not being recognized properly

Approved

Hi @cyberark-bizdev
Sorry for the delay. We are getting module not found error while testing the latest zip.

Can you please add all the dependencies to the python package folder and confirm.
Thanks

Added lxml dependency to requirements.txt
Review

Approved

Hi @cyberark-bizdev
Can you please update the zip folder with updated requirement.txt and also add relevant lxml packages under .python_packages folder.
Can you please share the working screenshot from function app, just to ensure the function app is working fine without any errors.
I have tried adding these things locally and tested, but I'm running into below error. So please help me with the success screenshot to approve the changes.

Thanks

Review

Hi @cyberark-bizdev
I have tried testing the updated zip file, still getting the same issue.
Thanks

Hi @cyberark-bizdev
Can you please verify the function app and share the success screenshot.
Thanks


Squashed commit of the following:

This change is to improve the Documentation of Properties and other small changes to improve the usability of the Operators.

Please replace the text above with a proper commit message.

Sandesh can you change the commit description so it better represents its contents


Update tslib and uuid dependency in communication-email

Packages impacted by this PR
communication-email
Issues associated with this PR
None, but https://github.com/Azure/azure-sdk-for-js/issues/15697 is related
Describe the problem that is addressed by this PR
Follow-up from https://github.com/Azure/azure-sdk-for-js/pull/15702 that upgraded the tslib dependency for other packages.
What are the possible designs available to address the problem? If there are more than one possible design, why was the one in this PR chosen?
Are there test cases added in this PR? (If not, why?)
Provide a list of related PRs (if any)
https://github.com/Azure/azure-sdk-for-js/pull/15702
Command used to generate this PR:**(Applicable only to SDK release request PRs)
Checklists

[x] Added impacted package name to the issue description
[ ] Does this PR needs any fixes in the SDK Generator?** (If so, create an Issue in the Autorest/typescript repository and link it here)
[ ] Added a changelog (if necessary)

From my side its ready to go and awaiting review. @yogeshmo, @joheredi could you maybe have a look, or assign someone more suitable? Thanks!


python310Packages.whois: 0.9.15 -> 0.9.16

Description of changes
https://github.com/DannyCork/python-whois/releases/tag/0.9.16

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 177262 run on x86_64-linux 1

4 packages built:

dnstwist
home-assistant-component-tests.whois
python310Packages.whois
python39Packages.whois


Add health stats aggregator

https://github.com/eleme/banshee/issues/215
@nrzipher

[ ] Record health stats in detector.. etc.
[ ] Add an api to query health data


Snowflake Doubles interpreted at Integers

Tables with column types of 'Double' are interpreted by Datagrok as type 'int'.
I can work with the developer to show the error if that will help.

https://reddata.atlassian.net/browse/GROK-12510

Please tell me which version of the Grok Connect you are using?

Hi Pavlo, The column in Snowflake is defined as a FLOAT. Datagrok server was recently upgraded to 1.13.7. I am not sure what version of Grok Connect I have. Please let me know how to find this information and I will provide it.

Could you please provide creational script of this table? SELECT GET_DDL('TABLE', 'schema.TABLE_NAME');

Could you please provide creational script of this table? SELECT GET_DDL('TABLE', 'schema.TABLE_NAME');

create or replace TABLE SCREENS (
NAME VARCHAR(16777216),
SMILES VARCHAR(16777216),
MOLECULAR_WEIGHT FLOAT,
LOG_P NUMBER(20,3),
NUM_H_BOND_DONORS NUMBER(38,0),
NUM_RULE_OF_5_VIOLATIONS NUMBER(38,0),
HEAVY_ATOM_COUNT NUMBER(38,0),
NUM_ROTATABLE_BONDS NUMBER(38,0),
IUPAC_NAME VARCHAR(16777216),
INCHI_KEY VARCHAR(16777216),
OWNER VARCHAR(16777216),
CREATED_AT TIMESTAMP_NTZ(9),
BATCHES ARRAY
);

And please, try to clear cache of connection if possible and run again query.

I cleared the cache for the connection, but get the same result.

Hi, Tom. I have made some investigations and it turned out that you use the older version of Grok Conne - 1.3.1. Support for Snowflake was corrected in newer versions of GC. Please contact Sofia Podolskaya to update your Grok Connect.

Thank you Pavlo for investigating. I will have Grok Connect upgraded.
On Mon, Mar 20, 2023 at 1:17 AM Pavlo Polovyi @.***>
wrote:

Hi, Tom. I have made some investigations and it turned out that you use
the older version of Grok Conne - 1.3.1. Support for Snowflake was
corrected in newer versions of GC. Please contact Sofia Podolskaya to
update your Grok Connect.
—
Reply to this email directly, view it on GitHub
https://github.com/datagrok-ai/public/issues/1678#issuecomment-1475795841,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABOXTNCKSZXD5HLHKH4LDT3W5AHCBANCNFSM6AAAAAAV4IOZ6I
.
You are receiving this because you authored the thread.Message ID:
@.***>

Hi Pavlo, Confirming that the upgrade to the latest Grok Connect fixed this issue. Thanks!


[BUG] Provide commands reference sorted by data type

Describe the bug Several places in the documentation reference URLs like: /commands/#hyperloglog
Expected behavior It goes to a page with commands related to specific datatypes
This could be solved either by providing hash links on /commands/ or create pages for each datatype and update hash links to true links (may be better for SEO)

closed by #152


How does conflict resolution work with model-drive apps?

We can now include model-driven apps in a solution, but how does the conflict resolution work with those? Is it Merge or Top wins? What happens when choosing to overwrite unmanaged changes, or not?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 8c25c71c-c1d0-10be-5904-f72ad490faf8
Version Independent ID: 03c97917-e4ee-f8ba-ae88-8aa39951307e
Content: Introduction to solutions (Developer Guide for Dynamics 365 for Customer Engagement apps)
Content Source: ce/developer/introduction-solutions.md
Service: crm-online
GitHub Login: @JimDaly
Microsoft Alias: jdaly

@KumarVivek
Please retire/redirect to the CDS topic.
Also remove me as author. @shmcarth is better fit


bug: Fix timezone setting in the UP

Describe the bug
The incorrect timezone is being used when converting date times in the UP. The convertDateTimeToHL7 function in CustomTranslationFunctions.kt needs to use the timezone from the receiver's timezone setting instead of getting the timezone from the date.
Impact on ReportStream
Receivers will not get dates in the format they're expecting.
Steps to reproduce
Steps to reproduce the behavior:

Update the DEV_FULL_ELR receiver by adding timeZone: PACIFIC and convertDateTimesToReceiverLocalTime: true
Send a report to RS using this postman collection:Simple Report.notes test.txt
Inspect the generated HL7 file and notice that dates didn't get converted to the receiver's local timezone.

https://github.com/CDCgov/prime-reportstream/pull/11445

We should follow this logic:


Visual display tweaks

Patch adding changes as discussed at length in #127

For some reason using underscores in the values of "Display" caused the bug that I reported in #131 I think? Really strange stuff, keeping bug report in backlog in case it does rear its head again, but seems to be resolved at least with the smoketest datasets.

Additional tweaks:

limit precision of recipient performance in bar and line charts to zero decimal points
add case count values for numerator and denominator to recipient performance in bar and line charts (all months of recipient bars, but only last three months of line chart)
remove chart labels above bar chart and line chart
remove axis labels "Performance Level" and "Time" from charts
Rotate month labels on x axis to be horizontal
change legend to display items in one row, horizontally, rather than a column
remove border around legend
increase font sizes by 3 to 5 points or so


No longer able to load Valkyrie store

First of all, thank you so much for making this! It's been really helpful.
It seems to have broken for me sometime in the last day or two. When I load any PSN page via the plugin, I get the plugin's "now loading" page for about 30 seconds followed by a generic "Unable to Find This Page" message from PSN. I checked the JS console and it does look like it's throwing a few exceptions, but I'm having trouble parsing out anything useful from it. In particular, I'm seeing:
Uncaught ReferenceError: WidgetBrowserData is not defined

Uncaught ReferenceError: a is not defined

Unhandled exception occurred:  TypeError: Unable to add 'ember-application' class to root element (#sb-social-toolbar-root). Make sure you set rootElement to the body or an element in the body.

Unhandled exception occurred:
{…}

reason: "appFailedToLoad"

Along with a generic 400 auth error from Sony, though that might be expected.

they blocked the OAuth Client ID used by Valkyrie, i tried replacing it with one they use somewhere else, but it seems the kamaji: scopes are gone too, meaning you cant get permission to use the storefront API's. so even though they still exist you cant really use them.


Rollback for string annotate

Local rollback to undo annotations on SharedString. This adds the previous values of properties to localOpMetadata (same data already generated for annotation event) and then restores them on rollback.

Zamboni test was failing because properties comparison decided undefined and {} were not equivalent. Fixed now.


[pylint] - restrict iteration-over-set to only work on sets of literals (PLC0208)

Summary
Restricts iteration-over-set to only work on sets of literals. There are valid use cases for iterating over a set, for de-duplication purposes.
Fixes #11694
Test Plan
cargo test

I guess I'm not a massive fan of this rule in general :/

Iteration over a list or tuple might be slightly more efficient in theory, but in practice it's a micro-optimisation that I would expect to very rarely make a difference
Using a set rather than a tuple or list can make it clearer to readers of your code that the iteration order doesn't matter, and/or that you expect each element in the iterable to be unique
Using a set rather than a tuple or list means you have deduplication of elements guaranteed, and means that Ruff will check your code with useful lints such as B033. With a tuple or list you could easily make an accidental typo such as (1, 1, 2, 3) when you meant (1, 2, 3).

However, all of that's pretty irrelevant to this PR, which I think pretty clearly makes the rule better!


RequireJs Support

Hey All,
I am wondering if there are any examples of using this directive with RequireJs. I cannot seem to get it to work.
snippet of code that doesnt work:
define(['angular', 'jquery',  'angular-google-chart', './routes', './Chart/ChartCtrl'], function(angular) {
'use strict';
return angular.module('dashboard', ['ngRoute', 'googlechart', 'dashboard.routes', 'dashboard.chart.ctrls']);
});

I may be out of date on this one, but I do not believe that angular officially supports RequireJS.  I know there are workarounds that can help make it happen.  The main problem is that Angular's Dependency Injection requires that all the code files be loaded when angular is bootstrapped.
Is this something you are already using in your project, or is this the first module you are trying to load with RequireJS?

Thanks for the response. yes, I already have Require and Angular trying to play nicely. you can look at an example project here https://github.com/mariussoutier/play-angular-require-seed/tree/master/app/assets/javascripts . Theres a few more moving parts in that for JVM land you download javascript JARs from a CDN. Its difficult to reason about why the module isnt being loaded. for most 3rd party libraries its as simple as adding the dependency to both Require and Angular. So if you have a file named myThing.js with a angular module called 'mymodule', you just do as above: add the file to the require dependency and the module to the returned super module as a dependency

What is the result of what you are doing?  Is it throwing any errors?  Is it loading but not functioning?

loading but not functioning. i suppose this may be an unanswerable issue (environment related)

update: I got it to work with this missing piece - adding angular as dependency in RequireJs shim!
shim: {
'angular-google-chart': ['angular'],
}
glad i can use this library! thanks


Refactor permission objects into their own class

Should have methods such as

parsing
formatting
getting parents

think steering away from classes is actually the better idea


Allow PartialRadixSort for Partial Array

SUMMARY

The method PartialRadixSort sorted the complete array. But there could be scenarios where sort is needed for a part of the array.

Automated Checks

[x] I have run ./gradlew test and made sure that my PR does not break any unit test.
[x] I have run ./gradlew checkstyleMain or the equivalent and corrected the formatting warnings reported.

That's reasonable, but it would be best to have a use case. I am not sure that generalizing code that is only used in a specific manner is necessarily helpful.

That's reasonable, but it would be best to have a use case. I am not sure that generalizing code that is only used in a specific manner is necessarily helpful.

So, the use case that lead to this effort is as following:

RoaringBitMap (RBM) library has an API add(). This was initially used. When we profiled the library, add seemed to be taking the most amount of time. So, we explored other possible APIs - addN came to the forefront.
Now, to use addN we have two options - pre create a fixed size array. Keep filling it. Once filled - sort it and do addN, other option is to create a dynamically growing List but this results in a copy while growing, which we didn't want to.
So, we went with sort and do addN. While sorting, partialRadixSort came to our use case. But this always assumed that the sort needs to happen for the entire array. That is why we modified the routine here.

@kf-vikas Can you give me an example of code where you'd use this function?
My concern is that your PR adds functionality that is not used, not tested, not benchmarked. If we could at least illustrate the use case..

This method was never intended to be a public sorting utility, and I don’t object to copying it (I wrote it so I think I can grant permission to copy it.)
Can I ask about the use case though? Are you inserting values into the bitmap in ascending order? If so, you can use the bitmap writer API which avoids the binary searches which make the add method expensive.

@kf-vikas Let me elaborate on my concern. As @richardstartin stated, this is not part of the public API. This means, in particular, that it could change or disappear in a future release. Now, we can make it part of our public API, but we would want something like a test that illustrates the use case.
We also want to promote good usage of what is already available. To be clear, increasing the surface area of our public API is not a goal. In some instances, that can be a negative.

@kf-vikas Can you give me an example of code where you'd use this function?
My concern is that your PR adds functionality that is not used, not tested, not benchmarked. If we could at least illustrate the use case..

@lemire - we tried in it one of our internal implementation. I am not sure if I can share the code.
Let me explain what lead to this.
One of our use case happened to insert a lot of random values in the Roaring Bitmap. On profiling, it was found that add call was quite CPU intensive. So we started exploring alternatives. One of the option was to do batch insert (using addN) API. But then, based on previous knowledge from @ashishkf, it was suggested to sort the array before insertion.
However, the array (to sort) was pre-allocated and was not guaranteed that it will be completely filled. Hence we looked at the method partialRadixSort and thought to augment it.
Orthogonal to this, I would like to know your thoughts/recommendations on how to insert random numbers?
Thank you.

This method was never intended to be a public sorting utility, and I don’t object to copying it (I wrote it so I think I can grant permission to copy it.)
Can I ask about the use case though? Are you inserting values into the bitmap in ascending order? If so, you can use the bitmap writer API which avoids the binary searches which make the add method expensive.

@richardstartin - our use case is insertion of random numbers in the RoaringBitmap. Options:

add
addN - we haven't verified this yet
sort + addN

Your thoughts are welcome!

@kf-vikas Let me elaborate on my concern. As @richardstartin stated, this is not part of the public API. This means, in particular, that it could change or disappear in a future release. Now, we can make it part of our public API, but we would want something like a test that illustrates the use case.
We also want to promote good usage of what is already available. To be clear, increasing the surface area of our public API is not a goal. In some instances, that can be a negative.

@lemire - if the concern is of a missing unit test, I am happy to add it to the diff.
The use case is something I described earlier. Please let me know if the explanation is not clear.
Thank you.

Adding a test would help a lot.


Create metadata table

I'm not seeing the table out in Snowflake. Do you know if it will be written out or will this go somewhere else?

since there was not schedule info it had to be executed manually. I did, the table is there (metadata)


almost useless error messages

I realize this crate is legacy, but at the moment it's the best way to write async software if you depend on anything from crates.io, which is basically every useful program one might want to write. Also, this could be a much bigger issue with proc macros that surely needs reporting, so, sometimes when developing a new program the compiler spits out error messages like this,
This happens on rustc 1.30.0-nightly (73c78734b 2018-08-05) and ever other version of nightly I tried (which believe me is quite a few more than I want to list)
error[E0507]: cannot move out of borrowed content

error[E0597]: borrowed value does not live long enough
   --> src/protocol.rs:339:5
    |
339 |     #[async]
    |     ^^^^^^^-
    |     |      |
    |     |      temporary value only lives until here
    |     temporary value does not live long enough
    |     borrow later used here

error: aborting due to 2 previous errors

This is almost completely useless, and I have to say, intensely frustrating. The horrible thing about this problem is that sometimes the error message is completely reasonable, and I have yet to identify a small example that triggers the behaviour, it doesn't seem to happen in small projects, but once a project crosses some complexity threshold the error context is getting totally clobbered by either async await, or proc macro. Another example, even more frustrating,
error[E0308]: match arms have incompatible types
  |
  = note: expected type `std::result::Result<protocol::Client, _>`
             found type `std::result::Result<(), _>`

error[E0308]: mismatched types

error: aborting due to 2 previous errors

That was the entire error message. It could be anywhere in the crate. Or even worse,
error[E0308]: mismatched types

error: aborting due to previous error

For more information about this error, try `rustc --explain E0308`.

This one happened to be a u64 vs i64 type mismatch in a function call. The function expected i64 and was being passed a u64. The function being called wasn't async, but the caller was.
Barring a simple example, I can make the code available, it's only about 400 lines, just let me know.

I can confirm this issue is still present.
error[E0282]: type annotations needed

is about the worst Rust compiler error I have ever encountered.