[RFC] Simplify integers and floats

Right now Crystal has a variety of integer and float types:

Integers: Int8, Int16, Int32, Int64, UInt8, UInt16, UInt32, UInt64
Floats: Float32, Float64

The default integer type when you don't use a suffix is Int32 and the default float type is Float64.
This kind of works but I imagine something better.
Int32 and Float64
Given that Int32 and Float64 are the default types it feels a bit redundant to type those 32 and 64 numbers all the time.
So here's an initial idea: what if we name those types Int and Float? We would of course need to rename the existing base types Int and Float but that's not a problem, we can maybe call them IntBase and FloatBase or Integral and Floating, it doesn't matter much because those names won't be used a lot.
Then talking about ints and floats is so much simpler: just use Int and Float everywhere. In the case where you do need a specific limit, which is rare and usually only useful in low-level code such as interfacing with C or writing binary protocols, you can still use the names Int32, Int64, Float32 or whatever you need.
What to alias to
Now, we could make Int be an alias of Int32 and Float an alias of Float64, but maybe it's better if we make Int depend on the architecture. That means Int would be equivalent to Int64 in 64 bits architectures.
This is also how Go works. They recommend using int everywhere unless you have good reasons to use a specific size. It's probably the case that using Int64 by default instead of Int32 works equally fine (maybe even better because the range is bigger so overflow is less possible) without a real performance degradation.
Another nice thing is that if eventually 128 bit architectures appear all programs will automatically start using this bigger range (if we want to) without needing to change any code.
To alias or not
Now, we could make Int be an alias of the respective underlying type, but I don't think that's a good idea. The reason is that if you have a program that does:
x : Int = 1
y : Int32 = 2
x = y

that would compile in 32 bits but would stop compiling in 64 bits. Ideally we'd like our programs to always compile regardless of the architecture.
So, we could make Int and Float be different types. To assign Int32 or Int64 to them you would need to call to_i first. Then programs on 32 and 64 bits will go through that explicit conversion process.
Another benefit is that we could start making collections use Int as the size. This increases their amount a bit but I think it's fine: it's probably not a huge performance/memory penalty (most of the memory is in the actual data). But then their limit becomes the limit of the architecture's memory (well, half of it if we use signed integers, but it's still a lot more than we can do right now). And, like before, this limit will automatically increase when the architectures improve (well, if the Amazon burning doesn't mean our imminent doom 😞).
Friction?
If we need these conversions between Int and all other integer types, and same for Float, wouldn't it make it really hard to write programs, having to convert between integer types all the time?
No, I don't think so. Because Int will be the default type everywhere, except for the few cases I mentioned before (C bindings and binary protocols) there would be no reason to use another integer type.
More benefits
Right now when we parse JSON and YAML we use Int64 because it would be a shame to parse to Int32 because we might lose some precision.
With this change the type would be Int, as everywhere else, and this can be assigned to everything else too if we stick to Int as a default. I know in 32 bits the limit will be smaller, but 32 bits machines are starting to become obsolete (for example I think Mac is dropping support for 32 bit apps).
Breaking change?
This is probably a breaking change, but a good one.
Summary
In summary if we do this change we get:

Simpler names: Int and Float
Bigger ints by default on 64 bits architectures
More room for collections (slices, arrays, hashes, deques) and file sizes
No need to think about the architecture if you always use Int
Still be able to use a different size if you need it

A part of this RFC includes an older one https://github.com/crystal-lang/crystal/issues/6626, about making integer type depend on the platform.

I'm perfectly happy with the end result of this change, but I wonder how best to stage this change into the language.

Yeah... it's even hard to develop because Int and Float are baked in the compiler so we'll first have to change their meaning, then compile a compiler with the existing primitives.cr file, but then change that file to define the new hierarchy (and use Int everywhere), and then compile the final new compiler.
In any case I think this can be delayed to the future, after we get parallelism and windows. But it's something I would definitely like to have before 1.0 because it's a big change.

It's curious that I'm also repeating myself (#6626) but I'm glad what I wrote here is what we ended up concluding there (though I don't know why I said it's impossible to do so).

I'll happily welcome the change. I grew to really dislike Int being the union of all signed integers, and wished it just some integer (32 or 64-bit or arch-dependent). It will break some programs, thought maybe not that much, and it can be quickly fixed by temporarily using an AnyInt alias or something.
Swift also has distinct Int and UInt types that are architecture dependent, and are the recommended and default integer types (https://docs.swift.org/swift-book/LanguageGuide/TheBasics.html#ID317). Same for Nim with int and uint. Even C/C++ have long and ulong.
Yet, I can't find a language with architecture dependent floats. Swift has Float and Double, Go as float32 and float64. Nim has a float type that used to be platform dependent but now is merely an alias for float64 (https://nim-lang.org/docs/manual.html#types-preminusdefined-floating-point-types).

Yet, I can't find a language with architecture dependent floats

Good catch! Yeah, I think for float we should have Float be an alias of Float64, or even a distinct type, But I'd rather have something short like Float instead of having to type and read Float64 all the time.

If this is going to be a huge breaking change surely it makes sense to get this out the way as soon as possible, not delay it until the language has even more users.

I like that idea!
Just note that:

I didn't discuss this yet with bcardiff and waj
I didn't play with a possible implementation yet (it shouldn't be hard, but trying it might uncover some issues)

So I guess the first thing for me will be to try this out and see how it works.

One thing to think about: when you want to map an integer to a database you usually want Int32 or Int64 (or even other integer types). Using Int then is a bit confusing because the DB column type will need to change if we are in 32 bits or 64 bits. Making it Int32 in the DB but exposing it to the use as Int works for reading but not for writing (if you try to write something bigger than Int32::MAX it will fail), and making it Int64 in the DB works for writing but not for reading, in 32 bits.
Another problem: the literal 1 will have the type Int and that's fine. But what about<PHONE_NUMBER> (Int32::MAX + 1)? It could be Int but then it won't compile in 32 bits, effectively making some programs stop compiling depending on the architecture.  So maybe it's fine? 😅

Those are great counter-examples of having architecture-specific Int and UInt.
Literals
If I use something higher than Int32::MAX then I actually expect an Int64, not an Int, and it just happens to work on 64-bit targets. Having a compile time error for 32-bit targets seems appropriate?
It means Crystal can't infer<PHONE_NUMBER> as an Int64 or<PHONE_NUMBER>854775808 as an Int128, and we'll have to manually type them (oh no), but does it happen much? maybe some explicitness ain't that bad?
Database
I believe database columns should be explicit, that is either Int32 or Int64, but if integers are usually an Int it may create some friction and require some explicit casts (oh no)...

How will that work when math shard A uses Int (now fixed at Int32), math shard B uses Int64 and serialized formats (for example protobuf) are a mix of Int8|16|32|64, UInt8|16|32|64?  Will I need to manually convert between types every time a variable crosses a function boundary?  Where does over/underflow checking happen?  Do I have need to check manually with each conversion?

How will that work when math shard A uses Int (now fixed at Int32), math shard B uses Int64 and serialized formats (for example protobuf) are a mix of Int8|16|32|64, UInt8|16|32|64?

I think that's also a problem right now with Int32 being the default integer type.

Will I need to manually convert between types every time a variable crosses a function boundary? Where does over/underflow checking happen? Do I have need to check manually with each conversion?

The answer is yes because that's something you also need to do now with Int32 being the default integer type.

Looks like I can pass any type of Int with the type preserved.  With your proposal would this still work or would it convert to Int32?
def lib1_add(a : Int, b : Int)
  c = a + b
  lib2_func c
end

def lib2_func(x)
  p typeof(x)
end

x = 1_u64
lib1_add x, x

Output:
UInt64

@didactic-drunk Int is currently an union alias (not aliased to Int32 only):
alias Int = Int8 | Int16 | Int32 | Int64

We can rename the alias AnyInt and keep the same behavior.

@didactic-drunk Int is currently an union alias (not aliased to Int32 only):
alias Int = Int8 | Int16 | Int32 | Int64

We can rename the alias AnyInt and keep the same behavior.

Based on my example doesn't that mean math functions (or most functions) should use AnyInt and we're right back where we started?
A major complaint when working in physics with c++ is Int sizes.  Someone writes an algorithm using Int32 or Float32 for their problem and it's fine.  Someone else attempts to use it with physics data and it over/underflows.  SInce they're only half programmers they don't use things like version control.  Instead they email files back and forth so things like Int128 never make it upstream.  Each person who gets the file from the original programmer has to change Int32 to Int128.
They probably should have used a template but that's beyond them.  They tend to use the default.
If Int32/64 is the default it will be wrong some portion of the time.  They should use AnyInt?  No.  They'll copy and paste from an example they found on google which likely uses Int.  When it's too small they'll change it to Int128 manually.  When the first person refines the algorithm?  They email it to a few of the people who change the types again.
Why?  Int128 doesn't perform as well as Int32/64.  It also requires much more memory/storage space.  These run on huge clusters with > petabyte data sets.  Each person wants the Int type for their specific problem space but the algorithms are generic.
AnyInt solves the problem, which is why I think it should remain the default named as Int.

@didactic-drunk Names are exchangeable. I won't go into details about pro and contra of which name.
The problem isn't names. It's default behaviour. A union type can't be used as type of an instance variable. But some type must be specified everywhere you need to store integers. Currently, we advocate to use Int32 everywhere by default because that's safe and fits for most use cases. It is also the default type of untyped integer literals.
Even your non-programmer algorithm writers need to pick data types for their integers. And it can't always be a union type, no matter whether it's called Int or AnyInt.

+1, would be a good change.
I just asked this question how to hack crystal to use Int and Float everywhere and got link to that issue.
Clean, readable, compact code is one of the key feature of Ruby. Hard to justify 32 and 64 noise in codebase if they don't contribute or mean anything, at least in my projects as I use only those two everywhere.

+1 for making them the same on all platforms.  Less confusion porting (and debugging somebody else's code).  If they want to interface with C...maybe create a new type called "NativeInt" or something, that can be used as the parameter?

I attempted to ask here: https://forum.crystal-lang.org/t/int32-and-float64-why-the-defaults/1797 why are Int32 and float64 the defaults?  Curious, since one is "32" and the other "64", thanks :)

I think it makes sense to have this before 1.0
It's much safer to use Int64 by default when dealing with native numbers in JSON and database.

@cyangle This is not going to happen before 1.0. No other major changes are expected before 1.0

Really? I think this and #8872 are just as important as overflow checks. It changes everything about numbers in the language...

The thing is that @waj just showed me a couple of benchmarks. For example this:
require "benchmark"
puts 1
a = Array(Int32).new(50_000_000) { rand(Int32) }
puts 2
b = Array(Int64).new(50_000_000) { rand(Int64) }
sa = 0_i32
sb = 0_i64
Benchmark.ips do |ips|
  ips.report("Int32") { sa = a.reduce(0_i32) { |s, i| s &+ i } }
  ips.report("Int64") { sb = b.reduce(0_i64) { |s, i| s &+ i } }
end
puts sa
puts sb

It's slower for Int64. The reason is that even though math operations take probably the same time, the data that you can put on a cache line or bus is smaller, so there's that performance loss with Int64.
What we are considering, though, is adding a Size type that's a different type than Int32 and Int64, and that would be used as the type of size in collections. That way you can have bigger collections in 64 bits machines. But the default integer type still stays Int32 for performance reasons (same decision as, for example, Rust).

I'm not sold on that. When 32-bit vs 64-bit performance matters (and 32-bits are big enough) you can simply optimize your code by using Int32 explicitly. But that's actually an edge case for heavy math operations.
For the vast majority of use cases the performance difference is completely negligible. But usability would greatly improve if we just had a simple default integer data type that works for (almost) everything. You would only have to resort to explicit types for binary interfaces, optimizations and maybe some other special cases.

Does the Rust-way fits Crystal? I think Crystal is closer to Go and Swift: abstract details but give access to low-level when needed. In that benchmark, if Int32's are enough, then you can optimize (cool), thought we're talking of 190MB vs 380MB arrays. That's kinda big, and the performance hit ain't so bad (1.28× slower) given that the CPU caches are busted twice as many times.
Having a specific Size type for collection sizes introduces friction (or weird type changes/overflows) whenever we'll want to compute anything with them (not cool). It also requires to continue to type as Int32 instead of a simpler Int —using Size for integers is weird, and not the recommended way to interact with libraries.

Personally I think discussion new integer types right now is entirely missing the point of 1.0.
The original plan was to release 1.0-pre1 as 0.35.0+bugfixes and now we're discussing this? Even #9357 can be implemented after 1.0 by adding a long_size instead of changing size, which is originally why I stopped working on it.

I personally wouldn't mind having a default integer type that's Int64 in 64-bit machines. I think the same was you, @ysbaddaden . But not everyone thinks the same so we have to come to some consensus.
We've also been talking about making the @size of collections (maybe only Slice for now) be Int32 or Int64, exposed as Int32 with size and as Int64 with size64. That's similar to how it's done in C#, where arrays have a LongLength property. This way, if you really need big collections or slices you can still work with them, but for the general case collections with less than Int32::MAX elements are probably enough for most use cases.
However, nothing is set in stone yet, this is what we've been discussing so far.

Ask MIT
On Thu, Jul 9, 2 Reiwa at 3:52 PM Ary Borenszweig<EMAIL_ADDRESS>wrote:

I personally wouldn't mind having a default integer type that's Int64 in
64-bit machines. I think the same was you, @ysbaddaden
https://github.com/ysbaddaden . But not everyone thinks the same so we
have to come to some consensus.
We've also been talking about making the @size of collections (maybe only
Slice for now) be Int32 or Int64, exposed as Int32 with size and as Int64
with size64. That's similar to how it's done in C#, where arrays have a
LongLength
https://docs.microsoft.com/en-us/dotnet/api/system.array.longlength?view=netcore-3.1
property. This way, if you really need big collections or slices you can
still work with them, but for the general case collections with less than
Int32::MAX elements are probably enough for most use cases.
However, nothing is set in stone yet, this is what we've been discussing
so far.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/crystal-lang/crystal/issues/8111#issuecomment-656319108,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/APXLM3LF5H63FA3NDGGFXKLR2YNWJANCNFSM4IO7LCVQ
.

We've also been talking about making the @size of collections (maybe only Slice for now) be Int32 or Int64, exposed as Int32 with size and as Int64 with size64.

That's even worse :sob:

Given that Int32 and Float64 are the default types it feels a bit redundant to type those 32 and 64 numbers all the time.

I think being specific about the type in a statically typed language is a positive. It shouldn't feel redundant, it should feel good because it's explicit.  Not against an Int or Float alias that is platform-dependent though.

I agree with this proposal, but it may change too much and be difficult to implement.
Regarding simplifying numerical systems, I have referred to multiple programming languages and provided the following highly feasible proposal.
This proposal is gradual, some can be completed in 1. x, and some need to be completed in 2.0.
Definition

Abstract numerical types: Number, Float, Int.
Specific numerical types: Float32, Float64, Int8, Int32, UInt8, UInt32, and so on.
Default numerical type: The numerical type determined by the compiler when the user does not explicitly specify a specific numerical type. The default numerical type is a specific numerical type, depending on the compiler version and platform architecture, which meets regular cases. Although the current default types are Float64 and Int32, users cannot assume that they will not change in the future.

Principles

When not caring about specific numerical types, it is advisable to minimize type constraints.
When caring about specific numerical types, the type should be explicitly specified.
It should be made as concise as possible in regular cases.
The following design follows the above principles.

Step 1: Modify method signature and document description
Change the return value type of size, sizeof and instance_sizeof to Int, and the actual return value type is the default numerical type.
Change the return value type of to_i to Int, and the actual return value type is the default numerical type. When users care about specific numerical types, they should call a method that explicitly specifies the type, such as to_i32. One advantage of doing this is that to_i and to_i32 has different application scenarios instead of being mixed according to user preferences, which can better unify the coding style.
For floating-point numbers, the same applies.
What about to_u? We should deprecate it. When the user calls to_u, it means caring about specific numerical types and should call a method that explicitly specifies the type, such as to_u32.
At this step, we only modified the method signature and document description, unified the coding style, without breaking change. Therefore, the old code can compile normally without being affected. So this step can be completed in 1.x.
Step 2: Unify the coding style related to operating numerical values
Whether it is a standard library or a third-party library, follow the coding style determined in the step 1 and gradually unify the code related to numerical operations.
This step will go through a considerable amount of time, not only covering a wide range, but more importantly, it will take a considerable amount of time to cultivate user habits.
This step is only about better unifing the code related to numerical operations, without breaking change, so it can be completed in 1.x.
Step 3: Add practical methods
If possible, these two methods can be added under the Top Level namespace:
max (*numbers: Number) : Number
min (*numbers: Number) : Number

This is more convenient to use than Math.max and Math.min.
Add Number#one method and deprecate Number#additive_identity and Number#multiplicative_identity, because Number#zero and Number#one are simpler and clearer.
Add Float#% method, the % operation should not be limited to integers, and can make // operation implementation simpler.
There was no breaking change in this step, so it can be completed in 1.x.
Step 4: Simplify overflow handling strategies
Although there are not many cases of overflow, we cannot ignore it. There are two strategies for handling overflow:

Raise an exception.
Return a reasonable value. For Float: +Inf, -Inf and NaN. For Int: a wrapping is performed.

I suggest referring to rust-lang strategy here.
Deprecate all wrap operations and only retain regular operations. For which overflow handling strategy, specify it in the compilation options:

In debug mode, the default is raising an exception.
In release mode, the default is returning a reasonable value.
Of course, users can also explicitly specify strategy for handling overflow.

We can even deprecate methods to_i!, to_i32!, to_f!, to_f64!, etc.
For very few cases where users need to specify overflow handling strategies in their code, we can refer to approach in C# lang.
In short, we should make the code in regular cases as simple as possible.
There was no breaking change in this step, so it can be completed in 1.x.
Step 5: Improve non primitive numerical types
There is an undeniable fact here: Decimal, Rational, and Complex are "second class" relative to primitive numerical types, and they are only applicable to certain specific fields.
I personally strongly recommend lifting their inheritance relationships with Number, Float, and Int. From a practical perspective, I cannot see the benefits of inheriting these non primitive types from Number, Float and Int. Because special handling often occurs where these types are involved, this inheritance relationship does not effectively reuse code and can also make originally simple code more complex. Whenever we see Number, Int, or Float types, we always need to consider: can this code handle non primitive types correctly? Especially for user-defined types.
Limiting Number, Float, Int to abstract types of primitive numerical types and prohibiting user-defined types from inheriting them can make many codes much simpler.
In this way, Number::Primitive, Float::Primitive, Int::Primitive can also be deprecated.
In addition, the Decimal, Rational, and Complex in the current standard library are not complete. These numerical types are essentially composed of primitive numerical types. We should implement them through generics and give an alias to commonly used types, which looks like this:
Decimal
  Decimal::Generic(V, S) < Decimal
  alias Decimal64 = Decimal::Generic(Int64, UInt8)
  alias Decimal32 = Decimal::Generic(Int32, UInt8)
  alias BigDecimal = Decimal::Generic(BigInt, UInt64)

Rational
  Rational::Generic(T) < Rational
  alias Rational64 = Rational::Generic(Int64)
  alias Rational32 = Rational::Generic(Int32)
  alias BigRational = Rational::Generic(BigInt)

Complex
  Complex::Generic(T) < Complex
  alias Complex64 = Complex::Generic(Float64)
  alias Complex32 = Complex::Generic(Float32)
  alias BigComplex = Complex::Generic(BigFloat)

NOTE: BigRational has an implementation that is not generic, and the performance of generic may not be as high as before, but the code consistency is better.
There are some breaking changes in this step, but these are all related to specific application areas and the impact will not be significant.
Step 6: Change default numerical types
Change the default integer type to platform specific. On the 64-bit platform is Int64, and on the 32-bit platform is Int32.
The internal implementation of types such as Pointer, Slice, Array, String, etc. has also changed from Int32 to the default integer type.
The changes in this step are a bit significant, but considering that it has been quite some time since the Step 1 and the user's coding habits have formed, it is safe to make breaking changes at this time.
Step 7: Automatic numerical type conversion
What type of result is obtained when performing operations on different integer types? There are several different strategies:

The return type is self, such as the current version of Crystal.
Prohibit this operation and only support operations of the same integer type, such as golang.
Automatic elevation: Int8 + Int32 -> Int32, such as nim-lang.

I think the third strategy is more suitable for 'static typed languages that write like dynamically typed languages'.
When users mix different types of integers, it indicates that they do not care about specific numerical types, and we should automatically elevate them to appropriate numerical types.
Unsigned and signed operations promoted to signed: UInt32 + Int32 -> Int32
For different types of floating-point operations, the same applies.
Int and Float operations promoted to Float: Int64 + Float32 -> Float32
The signature for division is as follows: Number#/(other: Number) : Number, implemented by converting to the default float number and calculating it. Although the result is Float64, the method signature is Number, indicating that users should not assume that the return value type will not change in the future.
Assignment operations for class variables and instance variables, automatic type conversion.
The assignment operation of a local variable remains the same as the current implementation, that is, union type.
For overflow during automatic type conversion, refer to the overflow handling strategy section.
Bit arithmetic is not considered here, and the rules of bit arithmetic need to be considered separately.
The changes in this step are significant and have a wide range of impacts, and can only be implemented in 2.0.
Step 8: Clean up deprecated code
This step is done in 2.0, and we can freely clean up deprecated code in 1.x.
At this point, all the simplification of the numerical system has been completed.

Step 3 and 4 may be good ideas but they don't seem directly related to the simplification of integer and float types (either set of changes could be implemented independently).
Step 5 makes no sense. There are no Rational and Decimal types. Complex exists, but it does not even inherit Number.
Step 7 also seems a bit of a separate issue (ref #8872).

Step 5 makes no sense. There are no Rational and Decimal types. Complex exists, but it does not even inherit Number.

Yes, there are currently no Rational and Decimal, but there may be in the future.
What I mainly want to express is that Number, Float and Int are only represented as primitive numeric types, preventing other non primitive numeric types from inheriting them, such as BigRational, BigDecimal, BigFloat and BigInt. That is:
Number == Number::Primitive
Float == Float::Primitive
Int == Int::Primitive

And deprecate Number::Primitive, Float::Primitive, Int::Primitive

This will simplify the code in many places, we don't need to always consider: can this code handle non primitive types correctly? Especially for user-defined types.

Minor note: something like #14393 adds the possibility of an Int type smaller than Int32

To be honest the C types for AVR are challenging: the CPU registers are 8-bits, so native integers are also 8-bits, but pointers are 16-bits (up to 64KB or RAM) while some boards ave 128KB of flash memory (?!) while size_t and int are also 16-bits, but long is 32-bits and long long is 64-bits. Oh, and both float and double are 32-bits but starting from GCC 10 double  may be 64-bits :shrug:

ISO C mandates int and size_t to have at least 16 bits. The same goes for cc65 (targets 6502) too

Please solve this problem in some way because it's scary to use. I'm not kidding it's really scary how many errors there can be. My friend and I are very interested in your language, but these problems with numeric types quickly put us off in the beginning. We'll just keep watching for now. Good luck to you. You have made a very interesting programming language.

@nerzh Could you elaborate what's "scary" about using number types? This hasn't been brough up yet in the discussion, so it's really not clear what you're referring to.
Please be aware that there are no fundamental issues with number types. This discussion is merely talking about improving the developer exerience, not fixing a bug or anything like that.

This hasn't been brough up yet in the discussion, so it's really not clear what you're referring to.

This came up on the Discord and the gist of it, as I followed it, was:
a = 1
b =<PHONE_NUMBER>854775805
pp a + b # => Unhandled exception: Arithmetic overflow (OverflowError)

Whereas in other languages they're used to, this would be a compile time error vs runtime.

@straight-shoota sorry, I got my issues mixed up, this is in response to https://github.com/crystal-lang/crystal/issues/8872


numbering on legalcode is broken

The current legal code numbering is broken on 4.0 licenses.
Each article and paragraph shows double numbering both in blue and in black. I believe only the blue should remain, but please confirm that with the CC legal team.
See attached for what I see on the licenses:

@TimidRobot This should be a priority fix. cc: @kgodey

Fixed in https://github.com/creativecommons/new-www-theme/commit/0e476326bc36331de482a29b930ca0bfb1bcb87c


Telemetry message is not available on brew installation

The message
We are constantly looking to make Gauge better, and report usage statistics anonymously over time. If you do not want to
 participate please read instructions https://manpage.getgauge.io/gauge_telemetry_off.html on how to turn it off.

is not available during gauge installation with brew.

Homebrew does not allow this.


Users get stuck in the login screen waiting forever

Users do not seem to become a "welcome" message and get stuck loging in forever.
https://github.com/FAForever/server/issues/562
Possible ways to improve the situation:

[ ] Make a timeout after which we cancle log in
[x] Improve loging and catch and log potentially swallowed exceptions

User posting this on forum:
https://forums.faforever.com/viewtopic.php?f=3&t=18996&p=183021#p183021
Wanna have the bug fixed quickly?
Visit Issue hunt...

So in the server log I do see quite a few
Client <username> dropped. You have been signed out because you signed in elsewhere.

which might mean that old connections aren't being closed properly. Or maybe somehow people trigger two connections at the same time which kick each other out.


Update link

The link here is throwing.
@bradygaster I don't quite see where this one is at in https://docs.microsoft.com/javascript/api/%40aspnet/signalr/?view=signalr-js-latest ... if you point me to the right one, I'll take care of the link (I'll attempt to XREF link it in here).

I code fenced it for now. If the format is ok, we can fix this with a proper API link when it becomes available.


[docs] Improve about WithUniversalID()

I'm studying the Global Object Identification with Ent.
i found this in docs:

If you only have one deployment or all deployments have the exact same data in the ent_types table, you can use WithUniversalID instead. This will enable a file based type range store instead of a database-table as source of truth.

Is WithUniversalID() still a thing?
I cannot find anything for it in the entire repo:

How should I use it?

Hey! Please see https://entgo.io/docs/migrate#universal-ids.

Hey! Use the WithGlobalUniqueID option. ent_types file was started as an experimental solution and was deleted. Thanks and sorry for the inconvenience.

Ok. Should be update the docs?

Doc was updated. Thanks for reporting this <3


Shortpath for Public Networks feature does not seem to work with Palo Alto Firewalls in Azure

When leveraging Palo Alto Firewalls in Azure the UDP Shortpath for Public Networks feature does not seem to work.  I'm guessing this falls under the "Double NAT" recommendation in the docs.
The hard part to understand is the feature seems to work just fine with Azure Firewall which is also SNAT'ing to the public Internet.
Tests performed:
Palo Alto with Dynamic IP and Port SNAT - TCP only
Palo Alto with Dynamic IP SNAT - UDP (This is not a scalable solution)
Palo Alto with Dynamic IP and Port SNAT, Internal Load Balancer removed from path - TCP
Palo Alto with Dynamic IP and Port SNAT, PIP removed from Untrust, NATGW added - TCP
Azure Firewall inline - UDP
Since Palo Alto Firewalls are widely used by Azure customers, adding a note regarding the current compatibility might be helpful to customers looking to leverage this feature.
Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 485a0823-340a-3eeb-1ab9-f4eb96a28005
Version Independent ID: 651d5e74-4308-3f2a-e541-d741750f1d29
Content: Azure Virtual Desktop RDP Shortpath for public networks (preview) - Azure
Content Source: articles/virtual-desktop/shortpath-public.md
Service: virtual-desktop
GitHub Login: @femila
Microsoft Alias: femila

Thanks for the feedback! I have assigned the issue to the content author to investigate further and update the document as appropriate.

Thanks for the feedback. The use of Palo Alto specifically is not something we can advise on in this article, but rather on a more generic firewall basis.  Azure Firewall also can cause issues due to symmetric NAT. The article has had a few updates recently regarding this, which I hope are useful.
#please-close

@sercast actually shortpath works just fine on public networks with Palo Alto Firewalls if you enable persistent NAT for DIPP with "set system setting persistent-dipp enable yes". This feature is available at least on PANOS 10.1.6 or 10.1.7 depending on the model.

@sercast actually shortpath works just fine on public networks with Palo Alto Firewalls if you enable persistent NAT for DIPP with "set system setting persistent-dipp enable yes". This feature is available at least on PANOS 10.1.6 or 10.1.7 depending on the model.

Thanks for posting this. Unfortunately, on PANOS 10.2 and up, this feature is command is nowhere to be found in the CLI.
Not sure if the command has changed. I've reached to PAN for feedback on how to enable this.
If anyone knows, please post it here.
Cheers,
Neurobit


[auto-merge] branch-21.06 to branch-21.08 [skip ci] [bot]

auto-merge triggered by github actions on branch-21.06 to create a PR keeping branch-21.08 up-to-date. If this PR is unable to be merged due to conflicts, it will remain open until manually fix.

SUCCESS - auto-merge


Incorrect directions.

The following steps in the article do not follow the current layout of MS portal:
In the admin center, go to the Settings > Org Settings > Services page.
Select Reports.
Uncheck the statement Display concealed user, group, and site names in all reports, and then save your changes.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 1eaa6e98-66e9-aad3-2d67-543c22f4ce4b
Version Independent ID: 26bddf9e-f338-329f-76e5-302a1343f7f1
Content: Microsoft 365 admin center activity reports - Microsoft 365 admin
Content Source: microsoft-365/admin/activity-reports/activity-reports.md
Service: microsoft-365-business
GitHub Login: @camillepack
Microsoft Alias: camillepack

@ThompsonSolutions  Thank you for your feedback. the steps seem to be accurate

Hope this helps!
Thanks
Sri


cannot typing before switching between workspaces

System Info and Version
 System/Version info
Hyprland, built from branch  at commit cba1ade848feac44b2eda677503900639581c3f4  (props: bump version to 0.40.0).
Date: Sat May 4 15:42:32 2024
Tag: v0.40.0, commits: 4606

flags: (if any)

System Information:
System name: Linux
Node name: alienware
Release: 6.8.9-arch1-2
Version: #1 SMP PREEMPT_DYNAMIC Tue, 07 May 2024 21:35:54 +0000

GPU information:
00:02.0 VGA compatible controller [0300]: Intel Corporation HD Graphics 530 [8086:191b] (rev 06) (prog-if 00 [VGA controller])

os-release: NAME="Arch Linux"
PRETTY_NAME="Arch Linux"
ID=arch
BUILD_ID=rolling
ANSI_COLOR="38;2;23;147;209"
HOME_URL="https://archlinux.org/"
DOCUMENTATION_URL="https://wiki.archlinux.org/"
SUPPORT_URL="https://bbs.archlinux.org/"
BUG_REPORT_URL="https://gitlab.archlinux.org/groups/archlinux/-/issues"
PRIVACY_POLICY_URL="https://terms.archlinux.org/docs/privacy-policy/"
LOGO=archlinux-logo

plugins:

Bug or Regression?
Bug
Description
After the screen is locked with hyprlock (and sometimes after closing the eww), when you enter the password and log in, you cannot type anything until you switch the workspace once.
How to reproduce

Open an editor like VS Code.
Lock the screen with hyprlock shortcut.
Enter the password to get back to the editor.
Start typing in the editor and see if you can do it!

Crash reports, logs, images, videos
No response

ye dupe


TabNine 3.2.6 doesn't work on my installation

OS version: Ubuntu 18.04.5 LTS 5.4.72-microsoft-standard-WSL2
Editor version: 1.52.1
Programming language: all
TabNine extension version: 3.2.6

The TabNine configuration page is empty. There is nothing on the logs.
Everything works if I downgrade to version 3.2.4 from last week.

hi @fungiboletus, when running the 3.2.6 version, do you see tabnine in the status bar?
can you please share your tabnine_config.json file?
for Linux it should be located at  $XDG_CONFIG_HOME or $HOME/.config/TabNine /home/alice/.config/Tabnine
for windows {FOLDERID_RoamingAppData}\TabNine

Yes using tabnine 3.2.6, the tabnine button is in the status bar.
This is my tabnine_config.js file:
{
  "version": "3.1.14",
  "hide_promotional_message": false,
  "beta_enabled": "No",
  "ignore_all_lsp": false,
  "creation_time": "2020-11-10T12:45:39.723816Z",
  "guuid": null,
  "semantic_status": {},
  "enable_telemetry": false,
  "user_understands_that_enabling_tabnine_cloud_sends_code_to_tabnine_servers": false,
  "hosted_deep_completions_enabled": "Disabled",
  "tabnine_cloud_host": null,
  "tabnine_cloud_certificate_domain": null,
  "tabnine_cloud_port": null,
  "cloud_whitelist": [],
  "num_of_suggestions": null,
  "multiline_suggestions": null,
  "api_key": null,
  "api_base_url": null,
  "binary_update_interval_seconds": null,
  "local_enabled": "Unset",
  "disable_local_when_using_battery": false,
  "hide_deep_information_message": false,
  "enable_power_saving_mode": false,
  "rate_limit_interval_seconds": null,
  "rate_limit_amount": null,
  "generation": 30,
  "local_model_size": null,
  "model_hash_override": null,
  "local_indexing": null,
  "heartbeat_interval_seconds": null
}

Same issue here. Pop-os 20.10 (Ubuntu derivate).
config looks like this
{   "version": "3.2.34",   "hide_promotional_message": false,   "beta_enabled": "No",   "ignore_all_lsp": false,   "creation_time": "2020-12-02T19:28:15.869203190Z",   "guuid": "5d649545c591b57c27fee8787ab17964bcdd941aacf2104ba58624c294519b5c",   "semantic_status": {},   "enable_telemetry": true,   "user_understands_that_enabling_tabnine_cloud_sends_code_to_tabnine_servers": false,   "hosted_deep_completions_enabled": "Disabled",   "tabnine_cloud_host": null,   "tabnine_cloud_certificate_domain": null,   "tabnine_cloud_port": null,   "cloud_whitelist": [],   "num_of_suggestions": null,   "multiline_suggestions": null,   "api_key": null,   "api_base_url": null,   "binary_update_interval_seconds": null,   "local_enabled": "Unset",   "disable_local_when_using_battery": false,   "hide_deep_information_message": false,   "enable_power_saving_mode": false,   "rate_limit_interval_seconds": null,   "rate_limit_amount": null,   "generation": 32,   "local_model_size": null,   "model_hash_override": null,   "local_indexing": null,   "heartbeat_interval_seconds": null }⏎

Hi @MariusVB, can you please share a screen recording of the issue? Thanks

https://user-images.githubusercontent.com/24921111/107225513-2c9ac800-6a19-11eb-9065-77e91db96a2e.mp4

@MariusVB - can you please close all your tabnine instances and reload your ide (i suspect that you have some zombie process running)?

Restarted my computer but didn't work. Downgraded to 3.2.3 and it works now.

@MariusVB - thanks for the debug session! this is very helpful.  can you please reinstall 3.2.6 and send us the vscode extension log (Help -> Toggle developer tools)?

I'm also getting this issue, and I have reinstalled the extension 5 times.
Here is my config file:
{
  "version": "3.3.48",
  "hide_promotional_message": false,
  "beta_enabled": "No",
  "ignore_all_lsp": false,
  "creation_time": "2021-02-08T17:46:18.121359600Z",
  "guuid": "5176da141f974466049d2faeda4b4a4185b9a75b03d0b20fc50428dc24691a1f",
  "semantic_status": {},
  "enable_telemetry": true,
  "user_understands_that_enabling_tabnine_cloud_sends_code_to_tabnine_servers": true,
  "hosted_deep_completions_enabled": "Disabled",
  "tabnine_cloud_host": null,
  "tabnine_cloud_certificate_domain": null,
  "tabnine_cloud_port": null,
  "cloud_whitelist": [],
  "num_of_suggestions": 5,
  "multiline_suggestions": null,
  "api_key": null,
  "api_base_url": null,
  "binary_update_interval_seconds": null,
  "local_enabled": "Unset",
  "disable_local_when_using_battery": false,
  "hide_deep_information_message": false,
  "enable_power_saving_mode": false,
  "rate_limit_interval_seconds": null,
  "rate_limit_amount": null,
  "generation": 40,
  "local_model_size": null,
  "model_hash_override": null,
  "local_indexing": null,
  "heartbeat_interval_seconds": null
}

And here is a screenshot of using the developer tools and trying to access the Tabnine page which is empty:

@rapritchard Thanks, can you please paste the log?

Some messages have been moved to the Issues panel.
workbench.desktop.main.js:60 [Extension Host] (node:19312) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues.

Strange too. It works now but I can't really explain why.

I have released version 3.2.7, please update it and report if you have any issue. Thanks

I have released version 3.2.7, please update it and report if you have any issue. Thanks

@rapritchard

@dimacodota I'm still unable to use the extension after updating to 3.2.7. Same error in the console too.

@rapritchard - I see that you have an "UNRESPONSIVE extension host" warning, can you please share the .cpuprofile file?

@dimacodota
.cpuprofile file

@rapritchard, can you please check if the old (3.2.5) version of Tabnine works for you?

@rapritchard, can you please check if the old (3.2.5) version of Tabnine works for you?

one more thing, can you please share a screenshot of the binaries directory located under the tabnine extension installation directory?
vscode extensions lcation should be %USERPROFILE%\.vscode\extensions for windows

I get the same error in the developer console in version 3.2.5. I'm starting to think it may be my work machine messing up, so I need to try installing it on my personal machine to confirm.
And a screenshot of my binaries in the latest version.

@rapritchard - I've made some changes related to your CPU log (thanks a lot for that 🙏 ) can you please try to install the vsix attached to this alpha release ?

Tried to install that alpha release and ran into an error with the installer.

Here's the log:
11/02/2021 10:08:07 - Microsoft VSIX Installer
11/02/2021 10:08:07 - -------------------------------------------
11/02/2021 10:08:07 - vsixinstaller.exe version:
11/02/2021 10:08:07 - 16.7.3069
11/02/2021 10:08:07 - -------------------------------------------
11/02/2021 10:08:07 - Command line parameters:
11/02/2021 10:08:07 - C:\Program Files (x86)\Microsoft Visual Studio\Installer\resources\app\ServiceHub\Services\Microsoft.VisualStudio.Setup.Service\VSIXInstaller.exe,C:\Users\Reecep\Downloads\tabnine-vscode.vsix
11/02/2021 10:08:07 - -------------------------------------------
11/02/2021 10:08:07 - Microsoft VSIX Installer
11/02/2021 10:08:07 - -------------------------------------------
11/02/2021 10:08:08 - Skipping product Microsoft.VisualStudio.Product.BuildTools (ea408851) since it does not support extensions
11/02/2021 10:08:08 - Initializing Install...
11/02/2021 10:08:08 - Microsoft.VisualStudio.ExtensionManager.InvalidExtensionManifestException: Element 'Metadata' has attribute 'DisplayName' with value 'Tabnine Autocomplete AI: JavaScript, Python, TypeScript, PHP, Go, Java, Ruby, C/C++, HTML/CSS, C#, Rust, SQL, Bash, Kotlin, React' whose length is greater than maximum value of 100.
   at Microsoft.VisualStudio.ExtensionManager.ExtensionImpl.VerifyString(String value, ErrorInfo info, Boolean isRequired, Int32 maxLength, String defaultValue)
   at Microsoft.VisualStudio.ExtensionManager.ExtensionImpl.ExtractManifestData(PackageManifest manifest)
   at Microsoft.VisualStudio.ExtensionManager.ExtensionImpl.ReadPackageManifest(XmlReader reader)
   at Microsoft.VisualStudio.ExtensionManager.ExtensionImpl.ReadManifest(Stream manifestStream)
   at Microsoft.VisualStudio.ExtensionManager.InstallableExtensionImpl.ReadVSIXManifestFromPackage(Stream stream, CultureInfo preferredCulture)
   at Microsoft.VisualStudio.ExtensionManager.InstallableExtensionImpl..ctor(String path, CultureInfo preferredCulture)
   at Microsoft.VisualStudio.ExtensionManager.ExtensionEngineImpl.CreateInstallableExtension(String extensionPath, CultureInfo preferredCulture)
   at Microsoft.VisualStudio.ExtensionManager.ExtensionEngineImpl.CreateInstallableExtension(String extensionPath)
   at VSIXInstaller.ExtensionService.GetInstallableData(String vsixPath, String extensionPackParentName, Boolean isRepairSupported, IStateData stateData, IEnumerable`1& skuData)
   at VSIXInstaller.ExtensionPackService.IsExtensionPack(IStateData stateData, Boolean isRepairSupported)
   at VSIXInstaller.ExtensionPackService.ExpandExtensionPackToInstall(IStateData stateData, Boolean isRepairSupported)
   at VSIXInstaller.App.Initialize(Boolean isRepairSupported)
   at VSIXInstaller.App.Initialize()
   at System.Threading.Tasks.Task`1.InnerInvoke()
   at System.Threading.Tasks.Task.Execute()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.VisualStudio.Telemetry.WindowsErrorReporting.WatsonReport.GetClrWatsonExceptionInfo(Exception exceptionObject)

@rapritchard - can you please install it via the vscode extensions tab?

Thanks

Hi @dimacodota
Sorry for the late reply, the alpha version you asked me to install seems to work now, but this only seem to happen after my company updated my machine.
Thanks :)


Added DashboardLogic component

Components added:

DashboardLogic

This PR add the DashboardLogic component, whose purpose is to get the informations about the users tenant workspaces and pass them to the Dashboard component.

/deploy-staging

/deploy-staging

/rebase

/deploy-staging

/rebase

/undeploy-staging

/merge


Slate does not update the product.

Problem
Slate does not update the product price when changing the variant. This problem occurs if the prices in euros

@maxwell-movchan what class does your variant selector have?
Is it no-js maybe?
If is not, is there any error in console log on variation change?

@dejanstrancar There are no errors in the console. I just downloaded a clean slate theme and here is the same error. If the price in euros, if the price changes, the price is updated but only the euro's paw is shown

@dejanstrancar

The problem is solved. I took a function with the Timber framework and replaced old function on new function.
function formatMoney(cents, format) {
var value = '',
placeholderRegex = /{{\s*(\w+)\s*}}/,
formatString = (format || this.money_format);
if (typeof cents == 'string') {
  cents = cents.replace('.','');
}

function defaultOption(opt, def) {
  return (typeof opt == 'undefined' ? def : opt);
}

function formatWithDelimiters(number, precision, thousands, decimal) {
  precision = defaultOption(precision, 2);
  thousands = defaultOption(thousands, ',');
  decimal   = defaultOption(decimal, '.');

  if (isNaN(number) || number == null) {
    return 0;
  }

  number = (number/100.0).toFixed(precision);

  var parts   = number.split('.'),
      dollars = parts[0].replace(/(\d)(?=(\d\d\d)+(?!\d))/g, '$1' + thousands),
      cents   = parts[1] ? (decimal + parts[1]) : '';

  return dollars + cents;
}

switch(formatString.match(placeholderRegex)[1]) {
  case 'amount':
    value = formatWithDelimiters(cents, 2);
    break;
  case 'amount_no_decimals':
    value = formatWithDelimiters(cents, 0);
    break;
  case 'amount_with_comma_separator':
    value = formatWithDelimiters(cents, 2, '.', ',');
    break;
  case 'amount_no_decimals_with_comma_separator':
    value = formatWithDelimiters(cents, 0, '.', ',');
    break;
}

return formatString.replace(placeholderRegex, value);

}

Confirming a bug if you change currency formatting from {{amount}} to {{amount_with_comma_separator}} .. i believe currency it self does not make a difference, but devider is problematic. #


Add config option to disable first logon animation

The config option added will allow the disabling of the first login
animation dispalyed on client versions of Windows.

:white_check_mark: Build windows-openstack-imaging-tools 1.0.136 completed (commit https://github.com/cloudbase/windows-openstack-imaging-tools/commit/fe6964ba1c by @ilenghel)

:white_check_mark: Build windows-openstack-imaging-tools 1.0.137 completed (commit https://github.com/cloudbase/windows-openstack-imaging-tools/commit/843eb6901f by @ilenghel)


Improve code quality

I found a bug linked to the command line:
cat parcels.geojson | supermercado burn 18 | mercantile shapes | fio collect > parcels_z18_tiles.geojson
I reported this error here: https://github.com/swiss-territorial-data-lab/object-detector/pull/17#issuecomment-1719033188
I also made typo corrections in the branch ch/dl_dem (PR#17)
To avoid redundant work can we close this PR and focus on PR#14 and #17?

So... #17 is now closed.
We shall treat this one (merge & close), then #14...
@cleherny can you approve this?!


New version: JetBrains.DataSpell version 2023.3.3

This pull request is automatically generated by 🥟 Dumplings in #3259
Microsoft Reviewers: Open in CodeFlow

Duplicate of https://github.com/microsoft/winget-pkgs/pull/137147


python37Packages.flufl_i18n: 3.1.3 -> 3.1.4

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for python37Packages.flufl_i18n is: ""
meta.homepage for python37Packages.flufl_i18n is: ""
meta.changelog for python37Packages.flufl_i18n is: ""
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS

Warning: a test defined in passthru.tests did not pass

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 3.1.4 with grep in /nix/store/41y36v2rss9yiir80y54i1jkbkxmr995-python3.7-flufl.i18n-3.1.4

directory tree listing: https://gist.github.com/b75c3be9c365255b8f7bfb2126759c78

du listing: https://gist.github.com/82b8a5b05a896a0fd023a69689b620a7

Rebuild report (if merged into master) (click to expand)

40 total rebuild path(s)

10 package rebuild(s)

10 x86_64-linux rebuild(s)
10 i686-linux rebuild(s)
10 x86_64-darwin rebuild(s)
10 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
mailman
python37Packages.flufl_i18n
python37Packages.mailman
python37Packages.mailman-hyperkitty
python38Packages.flufl_i18n
python38Packages.mailman
python38Packages.mailman-hyperkitty
python39Packages.flufl_i18n
python39Packages.mailman
python39Packages.mailman-hyperkitty

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/41y36v2rss9yiir80y54i1jkbkxmr995-python3.7-flufl.i18n-3.1.4 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A python37Packages.flufl_i18n https://github.com/r-ryantm/nixpkgs/archive/2d9e62aeda9e4e8ae8215eff78af61f736288a7c.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/41y36v2rss9yiir80y54i1jkbkxmr995-python3.7-flufl.i18n-3.1.4
ls -la /nix/store/41y36v2rss9yiir80y54i1jkbkxmr995-python3.7-flufl.i18n-3.1.4/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

10 packages built:

mailman
python37Packages.flufl_i18n
python37Packages.mailman
python37Packages.mailman-hyperkitty
python38Packages.flufl_i18n
python38Packages.mailman
python38Packages.mailman-hyperkitty
python39Packages.flufl_i18n
python39Packages.mailman
python39Packages.mailman-hyperkitty

Maintainer pings

This is a semi-automatic executed nixpkgs-review which does not build all packages (e.g. lumo, tensorflow or pytorch)
If you find some bugs or got suggestions for further things to search or run please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 108315 run on x86_64-darwin 1

7 packages marked as broken and skipped:

mailman
python37Packages.mailman
python37Packages.mailman-hyperkitty
python38Packages.mailman
python38Packages.mailman-hyperkitty
python39Packages.mailman
python39Packages.mailman-hyperkitty

3 packages built:

python37Packages.flufl_i18n
python38Packages.flufl_i18n
python39Packages.flufl_i18n

This is a semi-automatic executed nixpkgs-review which does not build all packages (e.g. lumo, tensorflow or pytorch)
If you find some bugs or got suggestions for further things to search or run please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 108315 run on x86_64-darwin 1

7 packages marked as broken and skipped:

mailman
python37Packages.mailman
python37Packages.mailman-hyperkitty
python38Packages.mailman
python38Packages.mailman-hyperkitty
python39Packages.mailman
python39Packages.mailman-hyperkitty

3 packages built:

python37Packages.flufl_i18n
python38Packages.flufl_i18n
python39Packages.flufl_i18n


Salesforce custom lightning components not visible to testcafe

What is your Test Scenario?
Login to Salesforce and interact with any custom lightning component.
What is the Current behavior?
No custom lightning components appear to be visible to testcafe
What is the Expected behavior?
testcafe should be able to interact with custom lightning components just like a user in browser
What is your web application and your TestCafe test code?
Salesforce
Your website URL (or attach your complete example):

Your complete test code (or attach your test files):

import { Selector } from 'testcafe';

fixture`Custom Lightning Component issue`// declare the fixture
    .page`https://brave-bear-e42cd9-dev-ed.my.salesforce.com/`;  // specify the start page

test('Lightning Component', async t => {
    await t
<EMAIL_ADDRESS>       .typeText('#password','Password1234')
        .click('#Login')
        .click('.slds-icon-waffle')
        .click(Selector('ul li a span .label.slds-truncate.slds-text-link').withText('Accounts'))
        .click(Selector('.slds-truncate.outputLookupLink.slds-truncate.forceOutputLookup').withText('United Oil & Gas Corp.'))
        .expect(Selector('div.slds-card__body div').innerText).contains('Hello I\'m a custom lightning component')
});

</details>

<details>
<summary>

✖ Lightning Component

   1) Cannot obtain information about the node because the specified selector does not match any node in the DOM tree.

       > | Selector('div.slds-card__body div')

      Browser: Chrome 73.0.3683 / Mac OS X 10.14.4

          9 |        .typeText('#password','Password1234')
         10 |        .click('#Login')
         11 |        .click('.slds-icon-waffle')
         12 |        .click(Selector('ul li a span .label.slds-truncate.slds-text-link').withText('Accounts'))
         13 |        .click(Selector('.slds-truncate.outputLookupLink.slds-truncate.forceOutputLookup').withText('United Oil & Gas Corp.'))
       > 14 |        .expect(Selector('div.slds-card__body div').innerText).contains('Hello I\'m a custom lightning component')
         15 |        .debug()
         16 |});

         at Object.<anonymous> (/Users/brian.okeefe/Projects/DevExpress-TestCafe/e2e/tests/lightning_comp_test.ts:14:64)
         at <anonymous> (/Users/brian.okeefe/Projects/DevExpress-TestCafe/e2e/tests/lightning_comp_test.ts:7:71)
         at __awaiter (/Users/brian.okeefe/Projects/DevExpress-TestCafe/e2e/tests/lightning_comp_test.ts:3:12)
         at test (/Users/brian.okeefe/Projects/DevExpress-TestCafe/e2e/tests/lightning_comp_test.ts:6:39)
         at markeredfn (/usr/local/lib/node_modules/testcafe/src/api/wrap-test-function.js:17:28)
         at <anonymous> (/usr/local/lib/node_modules/testcafe/src/api/wrap-test-function.js:7:5)
         at fn (/usr/local/lib/node_modules/testcafe/src/test-run/index.js:242:19)
         at TestRun._executeTestFn (/usr/local/lib/node_modules/testcafe/src/test-run/index.js:238:38)
         at _executeTestFn (/usr/local/lib/node_modules/testcafe/src/test-run/index.js:291:24)

 1/1 failed (30s)

Screenshots:

attached

Steps to Reproduce:

Go to the salesforce scratch org https://brave-bear-e42cd9-dev-ed.my.salesforce.com
log in with credentials in test
Navigate to Accounts
Assert on custom lightning component with text "Hello I'm a custom lightning component"

Your Environment details:

testcafe version:  1.1.3
node.js version:  v11.14.0
command-line arguments:  testcafe chrome e2e/tests/lightning_comp_test.ts
browser name and version: Chrome 73.0.3683
platform and version: Mac OS X 10.14.4
other:  have tried across multiple browsers and multiple versions of testcafe.  bumping up to hammerhead v 14.6.0 resolved another issue for us where the navigation bar was not appearing in testcafe.
Noticed the issue in our production like QA environment.
Custom Lightning Component for this example was created through this trailhead https://trailhead.salesforce.com/en/content/learn/projects/quickstart-lightning-components/quickstart-lightning-components3

@blokeefe
Hello.
I didn't reproduce this issue (I used my own account) under Windows 10 / macOS Mojave 10.14.4 / Chrome 73 /<EMAIL_ADDRESS>(https://github.com/DevExpress/testcafe-hammerhead/commit/05d2788732c9b3e3549170f93a78cb52f0ac39d6).
testcafe-hammerhead result:

It is impossible to debug Salesforce with your credentials because of the "email verification" action.
There are two ways to reproduce the issue:

Disable the "email verification" on your account if it is possible. This way I can debug your test case.
Describe steps I can use to get your screenshot page with the "Contact list for ..." section.

@Farfurix i cannot disable email verification but we can white list your ip address to bypass that step.  Would that work for you?

@blokeefe, OK, our IP: <IP_ADDRESS>

@Farfurix Added.
when you navigate to this url as a user you will see the custom lightning component.
https://brave-bear-e42cd9-dev-ed.lightning.force.com/lightning/r/Account/0012E00001pSKxpQAG/view
Testcafe will not see it.
Thanks for your help.

@blokeefe
Thank you.
I've reproduced the issue under Window 10 / Chrome 73 /<EMAIL_ADDRESS>(DevExpress/testcafe-hammerhead@05d2788).
We need some time to investigate this issue using your credentials.


Unable to get font-awesome in production with Rails 5.1

I am using font-awesome-rails gem and everything works fine in development. In production, I am not able to get the file to work. It is looking for the http://forme-dev.us-west-2.elasticbeanstalk.com/assets/fontawesome-webfont-2adefcbc041e7d18fcf2d417879dc5a09997aa64d675b7a3c4b6ce33da13f3fe.ttf
I have few doubts! Will this file be compiled into public/assets? If yes, then it is not present there!
Here is how I am importing it in my application.scss:
@import "font-awesome";

Can you help me if I am missing something? Thanks.

I don't think our test suite covers Rails 5.1 yet (though we should add support for it more explicitly soon).
Based on your report, it sounds like you have config.action_controller.asset_host configured in production to point to a CDN.
http://edgeguides.rubyonrails.org/asset_pipeline.html#cdns
Assets only get compiled into public/assets if rake assets:precompile is first run during a deploy:
http://edgeguides.rubyonrails.org/asset_pipeline.html#precompiling-assets

I have the same issue with Font Awesome and Rails 5.1 (I'm running it on Heroku with default production.rb):

http://[hidden].herokuapp.com/assets/fontawesome-webfont-2adefcbc041e7d18fcf2d417879dc5a09997aa64d675b7a3c4b6ce33da13f3fe.woff2 - 404 Not Found
http://[hidden].herokuapp.com/assets/fontawesome-webfont-ba0c59deb5450f5cb41b3f93609ee2d0d995415877ddfa223e8a8a7533474f07.woff - 404 Not Found
http://[hidden].herokuapp.com/assets/fontawesome-webfont-aa58f33f239a0fb02f5c7a6c45c043d7a9ac9a093335806694ecd6d4edc0d6a8.ttf - 404 Not Found

I didn't change the default Heroku build process (everything is working as usual).

I haven't verified any of this myself as I don't have an app that I can take to Rails 5.1 at the moment, but it sounds like we might have some work to do to get things running as expected on Rails 5.1.
I'm currently traveling for family matters this week and next. If I find a hunk of down time, I might give this a go, but I can't make any promises as of yet.
A pull-request to remedy this, with new additions to the Travis-CI build matrix, would be very much welcome and super helpful!

Rails 5.1 is been released and I have this error if I want use last version of gem:
font-awesome-rails (~> 4.7) was resolved to <IP_ADDRESS>, which depends on
railties (< 5.1, >= 3.2) x64-mingw32

+1

Using it with Rails 5.1.rc2 and the require method ...
/*
 *= require font-awesome
 */

... everything works well.

A pull-request to remedy this, with new additions to the Travis-CI build matrix, would be very much welcome and super helpful!

I added a MR with rails 5.1. support and updated travis matrix (#185). But I don't have any problems on production, so cannot investigate it - for me it works.

Hi all. I just merged @doits additions to make the dependencies compatible with Rails 5.1. Because there's some uncertainty about whether things fully work in Rails 5.1, could  everyone  please first try their applications on the master branch (I'm in a bit of a bind this week and don't have much time to dig in on a real application just yet). Please update your Gemfile (temporarily) to read:
gem "font-awesome-rails", github: "bokmann/font-awesome-rails", branch: "master"

And let me know the results. If I get enough positive confirmations, I'll publish an official versioned release of font-awesome-rails with Rails 5.1 support.
Thanks!

The 5.1 compatible branch is causing zero issues in my application.

gem "font-awesome-rails", github: "bokmann/font-awesome-rails", branch: "master"
And let me know the results. If I get enough positive confirmations, I'll publish an official versioned release of font-awesome-rails with Rails 5.1 support.

Works for me, I'm running Rails 5.1.0

font-awesome-rails v<IP_ADDRESS> was just published.


AUTORUN parsing produces major delay compared to normal parsing, hitting Roll20 "possible infinite loop" timeout

After some more investigation, it looks to me as if the problem I've had goes back to something that is different between parsing the same code as part of an AUTORUN macro vs. through the chat.
This new code is the offending example. Without it, everything parses and runs fine. With it as part of the AUTORUN macro, startup takes about twice as long and Roll20 ends up killing the sandbox for hitting its "possible infinite loop" timeout. Here are the two logs in comparison in terms of the execution time:

"MMM [1.36.1/2024-02-07T19:35:15.329Z] (+359ms) started autorun macro: !MMM-AUTORUN (owner: phyllo, privileged: true)"
"MMM [1.36.1/2024-02-07T19:35:15.329Z] (+3243ms) completed autorun macro: !MMM-AUTORUN (owner: phyllo, privileged: true)"

vs

"MMM [1.36.1/2024-02-07T19:35:35.201Z] (+146ms) started autorun macro: !MMM-AUTORUN (owner: phyllo, privileged: true)"
"MMM [1.36.1/2024-02-07T19:35:35.201Z] (+1558ms) completed autorun macro: !MMM-AUTORUN (owner: phyllo, privileged: true)"

As soon as I disable the AUTORUN macro and just execute the same code (which means, in terms of the two new functions in the diff above, that they should only get parsed and publish to game gets executed for them), everything runs fine!
N.B. On the suspicion that the parser or publish to game might do some sort of mock execution of these functions with undefined or default arguments, I added a quick validation check on the arguments and cut the functions short before entering any of the for loops or reading any of the expected lists and structs. No change: works when called directly in game, crashes the sandbox if parsed through AUTORUN.

This is odd, especially the observation that it's taking longer when run as !MMM-AUTORUN than when the same thing is run interactively.
What's even more curious is that the Sandbox crashes after the autorun macro has completed running. I'm getting these logs in the test game:
"MMM [1.36.1/2024-02-13T19:32:31.550Z] starting up"
"MMM [1.36.1/2024-02-13T19:32:31.550Z] (+343ms) started autorun macro: !MMM-AUTORUN (owner: Michael \"GM\" B., privileged: true)"
"MMM [1.36.1/2024-02-13T19:32:31.550Z] (+3304ms) completed autorun macro: !MMM-AUTORUN (owner: Michael \"GM\" B., privileged: true)"

Notice how it says it has completed running the macro. Then, a few seconds later:
Possible infinite loop detected, shutting down.

I've intermittently observed that timings can be vastly different between the main game and the test game, but that apparently doesn't seem to affect the outcome. Your observation that libBasics 1.3.1 reliably works and 1.3.2 reliably produces the timeout (after a delay) appears to be spot on.

I've trial'd-and-error'd my way through various potential culprits, and I think I found what's triggering the issue.
It's the }} in this code:
!rem // chatButtonTable(label, rows)
!rem //
!rem //   Returns chat code for a button with the label and payload provided. If the optional style is given, it is applied.
!rem //
!mmm function chatButtonTable(label, rows)
!mmm   set chatButtonTableCode = "&" & "{template:default}" & "{{name=" & label & "}} "
!mmm   for row in rows
!mmm     if row.type eq "subheading"
!mmm       set chatButtonTableCode = chatButtonTableCode & "{{ " & row.label & " }} "
!mmm     else if row.type eq "multi-buttons"
!mmm       set chatButtonTableCode = chatButtonTableCode & "{{ "
!mmm       for singleButtonProps in row.buttonProps
!mmm         set chatButtonTableCode = chatButtonTableCode & "   " & chatButton(singleButtonProps.label, chatButtonPayload(singleButtonProps.scriptCommand, singleButtonProps.settings))
!mmm       end for
!mmm       set chatButtonTableCode = chatButtonTableCode & "}} "
!mmm     end if
!mmm   end for
!mmm   return chatButtonTableCode
!mmm end function

I'm guessing that seeing a }} in there makes Roll20's stone-age chat parser believe it has found the end of the {{ that started the whole autorun macro a few macro-insertion levels up, and thus everything in initGameGlobals that follows the insertion of libBasics (which is a lot) gets posted to the Roll20 chat engine in the egregiously imperformant line-by-line manner we've already painfully been acquainted to in the past, which leads to the timeouts we managed to avoid by using {{...}} around the whole script in the first place.
Replacing each }} with }\} in the MMM source makes the issue reliably go away. (It's set up in the main game like this right now.)

Thank you!
Strangely enough, I open the TEST game and nothing works, even with the fixed code in libBasics::chatButtonTable(). I go looking for further instances of }}, and lo and behold, there were a few in libMidgardRuneBlades, in a line that has been part of the code since Feb 2022 and has been AUTORUN ever since (8fc0165e43ba7ff25d1fe92bd1f485b5594a42cf). Perhaps because it was called rather toward the end of AUTORUN, it may not have been an issue before...? Fixing this line appears to have fixed TEST, and I'll copy the code over into the main game. Still, can't shake the feeling that this may not have been the end of this problem...

I'm sure you're right: The further down in the final macro expansion the stray }} appear, the moee of it gets sent to MMM as one large chunk of text and the less of it is parsed into individual chat lines by Roll20 and spoon-fed over by one to all API scripts.
The actual limit that triggers the "possible infinite loop" timeout is fuzzy, as is the exact mechanism that causes so much time to be consumed in the first place.
One thing we know for sure: It's literally the number of chat lines/chunks, and there's provably no relation to the total amount of data sehr that way – because evidently the same huge MMM script has no trouble going through when it's sent as one single chat chunk, but breaks the sandbox when it's first split into individual chat lines by Roll20.
I've suspected the JavaScript GC in the past (because the timeout is happening after API scripts are done executing), but maybe that's too esoteric.
Maybe Roll20 is sending every chat chunk into a database queue, which is fast enough if a single chat chunk gets inserted in a single transaction regardless of size but scales terribly wheb thousands of chat lines get inserted in thousands of individual transactions. (Why API chat lines would be sent to a database is anyone's guess, but I wouldn't put it past Roll20's devs.)
Or maybe the key difference isn't even in Roll20 itself but in one of the many other API scripts we have installed, especially in our main game: Each chat chunk is passed by every registered API script – perhaps one of the other API scripts unwittingly does something that produces terrible amounts of GC garbage (...so we'd be back at this core hypothesis) each time it sees a chat chunk pass by, regardless of size.


Mask features

@lukasalexanderweber thanks for this library, very useful to play with sticthing
I need to be able to mask some features in the stitching process so some region in the image are ignored in the sticthing process.
I found other people on the web having the same needs :

https://answers.opencv.org/question/192008/how-to-stitch-images-by-opencv-with-masks-only-stitch-parts-of-images/
https://stackoverflow.com/questions/73721536/how-to-stitch-images-by-opencvsharp-using-mask

I have implemented this feature in command line.
Currently i have been able to make it work like :
python stitching/cli/stitch.py img1.png img2.png --mask_names mask1.png mask2.png --no-crop --blender_type no --finder no --compensator no --affine

This is a first implem and more tests need to be done, but i would like to have an early feedback as if you are open to integrate this new mask_feature capability
What do you think ?

@lukasalexanderweber It's not needed since *args pass the mask directly to computeImageFeatures2 in
    def detect_features(self, img, *args, **kwargs):
        return cv.detail.computeImageFeatures2(self.detector, img, *args, **kwargs)

I have added the mask to the warper in the last commit

ahh nice I didn't knew that this functionality is already covered by OpenCV using the optional parameter mask
I thought you want to implement it in python ontop.
Thanks for the feedback and for working on it :)

@lukasalexanderweber in the last version i'm only wrapping the non-masked parts of the image.
This is a design choice and this can be discussed, but eventually, the question is to use feature_mask in the wrapping process as it is done in
https://github.com/lukasalexanderweber/stitching/blob/709906c4f7a66a08064eaf559813b9478dfe363c/stitching/warper.py#L66
On my use case, it makes total sense to do a wrap-masking
I'm not sure if other use cases would need feature-only masking (without wrap-masking)

On my use case, it makes total sense to do a wrap-masking

I don't quite get what you can achieve with the warped feature masks?

The use case is typically background stitching with foreground mask.
Input
First image

First Mask

Second image

Second mask

Command
python stitching/cli/stitch.py  frame00000.jpg frame00012.jpg --mask_names  mask00000.jpg mask00012.jpg --no-crop  --blender_type no  --finder no  --compensator no  --affine  --output tennis.png

Result with wrapping

Result without wrapping

https://github.com/BGU-CS-VIL/DeepMCBM#results -> wow this looks like magic
So to summarize:

the user can pass in masks to specify in which regions features should be searched
additionally, the user can have another option to let the masks be foreground mask for background stitching

the user can pass in masks to specify in which regions features should be searched

Yes

additionally, the user can have another option to let the masks be foreground mask for background stitching

Do you have an example in mind where we don't want to use the feature mask in the warping ?
In my use case (foreground masking), i'm masking parts of the image which won't match between images, if those part won't match, keeping them or not in the final result will be ambiguous and will depend on the order of the images.

Do you have an example in mind where we don't want to use the feature mask in the warping ?

yeah like when you have a thing in an image which has a lot of features but is technically not part of the image and leads to wrong matches.

I worked on it a while yesterday evening and thought that the proposed changes have the potential to bloat the standard stitching workflow. I thought that it might be possible creating some kind of BackgroundStitcher which extends Stitcher. What do you think of that? In the CLI, if feature masks are specified, we will then create an instance of BackgroundStitcher

@lukasalexanderweber ok !
Can you please comment the PR or make a PR over my PR with your changes so I can try to help you back in this ?
I'm afraid to work in parallel on different approaches.
Thanks

Hello @piercus!
I want to check back on you if you've solved the issue for you and if you still have interest in contributing?
I was busy improving the testability of the package so I should be able to help you faster now.
In https://github.com/OpenStitching/stitching/issues/94 only the first part of masking features is needed, independently of background stitching (I guess)

@lukasalexanderweber I feel we should keep it as simple as possible and maybe only focus on the first part of the masking features

I just came across related issues regarding passing a mask to the feature detector. I am very interested in having this feature in the stitching package but I noticed that this PR is open for some time.
Since no further progress has been made here, I opened another PR#104 to finish it.

continued in https://github.com/OpenStitching/stitching/pull/104


Add Cartrade tech logo to adopters

[ ] I added CHANGELOG entry for this change.
[x] Change is not relevant to the end user.

Changes
Added CarTrade tech to the adopter list
Verification
Followed the process defined here

Thanks a lot! Could you please sign the commit with git commit -s? 🙂

Thanks for the reply. Was not sure if sign-off was required for the adopter's commits too. Have amended the commit with sign off

Looks like DCO verification is still failing in the CI.

@fpetkovski DCO is succesful now

@naveadkazi sorry I had to rebase against main through the UI and Github decided to force-push a new commit :/

@fpetkovski should I force push again with new signed comments?

I think that would be the simplest way to resolve this. Sorry for the hassle.


HPPC-175: Method estimating memory usage

Create Interface Accountable and helper class RamUsageEstimator, most of them forked from Lucene.
Then estimate the memory usage using those utils

Looks good to me, Patrick, thank you! If you could add an entry to changes.txt (with your name as the contributor) it'd be ready for merging.

Thanks! I'll make that entry and merge then.

Thanks Patrick!

Hi David, maybe we could bump to new version so that this change is available? Since actually some feature I'm working on will benefit from this change :)

Hi Patrick. Yes, makes sense. I'll do it on Monday as I'm out of office this weekend.

Done, thanks Patrick!
https://github.com/carrotsearch/hppc/releases/tag/0.8.2

Thanks David!


Fix screen-width on HUAWEI VRP

Description
Fixes the sceen-width on HUAWEI VRP by manually setting the transport-options -> ptyprocess -> cols argument to 256.
See issue #18  for details.
Also seems to have updated all the documentation to a newer version. Hope that's OK.
Type of change
Please delete options that are not relevant.

[X] Bug fix (non-breaking change which fixes an issue)
[X] This change requires a documentation update

How Has This Been Tested?
Tested on devices running HUAWEI VRP where the CLI output was previously cut off.
Checklist:

[X] My code follows the style guidelines of this project (no GitHub actions compalints! run make lint before
committing!)
[X] I have commented my code, pydocstyle and darglint are happy, docstrings are in google docstring format, and all
docstrings include a summary, args, returns and raises fields (even if N/A)
[ ] I have added tests that prove my fix is effective or that my feature works, if adding "functional" tests please
note if there are any considerations for the vrnetlab setup
[X] New and existing unit tests pass locally with my changes

Boom! Thank you for another great contribution sir!


Add heartbeat to python client, and use a task manager to control background tasks

Added heartbeat
Use taskmanager to manage task
added CI

@zhenthebuilder Can this branch be deleted?


[Bug] $env serviceprincipalkey is not used with "az devops service-endpoint azurerm create"

Describe the bug
I execute these commands:
$SERVICE_PRINCIPAL_SECRET = az ad sp credential reset --name "$ServicePrincipalName" --query "password" --append
# SERVICE_PRINCIPAL_SECRET contains for example: "Kx8RRSAFDSEU1Wn-Vgx7ASDADwdasw2.H"
$env:AZURE_DEVOPS_EXT_AZURE_RM_SERVICE_PRINCIPAL_KEY=$SERVICE_PRINCIPAL_SECRET

az devops service-endpoint azurerm create `
            --name $serviceConnectionName `
            --azure-rm-service-principal-id $ServicePrincipalId `
            --azure-rm-tenant-id $tenantId `
            --azure-rm-subscription-id $subscriptionId `
            --azure-rm-subscription-name $subscriptionName

The issue is that it does not seem to pass/use the secret from the $env in the request.
To Reproduce
Azure Cli Version:  2.24.2
Azure-Devops extension version: 0.18.0
Steps to reproduce the behavior:

login using az login
setup defaults..
$SERVICE_PRINCIPAL_SECRET = az ad sp credential reset --name "$ServicePrincipalName" --query "password" --append
$env:AZURE_DEVOPS_EXT_AZURE_RM_SERVICE_PRINCIPAL_KEY=$SERVICE_PRINCIPAL_SECRET
az devops service-endpoint azurerm create          --name $serviceConnectionName
--azure-rm-service-principal-id $ServicePrincipalId          --azure-rm-tenant-id $tenantId
--azure-rm-subscription-id $subscriptionId `
--azure-rm-subscription-name $subscriptionName
cli says it will create service connection with:

{
  "administratorsGroup": null,
  "authorization": {
    "parameters": {
      "authenticationType": "spnKey",
      "serviceprincipalid": "<guid_here>",
      "serviceprincipalkey": null, <---- is null
      "tenantid": "<guid_here>"
    },
    ........

The serviceprincipalkey is null, and I checked in azure devops that it is not set, when I edit the service connection and press verify.
The issue is that it does not seem to pass/use the secret from the $env in the request.
NOTE: when I copy the Service_Principal_Secret value into service connection in azure devops itself, it does work!
Expected behavior
Expected behavior to send the serviceprincipalkey with the request or hide it in the request, but it is actually stored in the service connection, when I click verify.

Apparently -o tsv resulted in whitespaces behind the password. Now I use -o json and it works.


If elastalert can integrated with Elastic Stack 7.0.x

Hi all,
if anyone can use the elastalert with ELK 7.0.x, I have tried to configure elastalert with the instruction, but always failed, the instruction seems to be indistinct. I have setup the elastiflow to monitor network traffic, and I want the Elastic to generate an email notification when clients traffic exceed the threshold (eg:5GB) without ELK platinum license. so if does anyone to achieve the same alert function as me, I'm looking forward your help, thanks in advance.

when will be elastalert support ES 7.x version?

Note: If you're using Elasticsearch 7, you'll need to install a beta release of Elastalert: pip install "elastalert>=0.2.0b"

thank you @dan98765 , I had installed beta release and trying now.


test-branch/execute-job/quoted-authorization/796a4806-4789-43f4-a574-118322dbc6f5

PR for testing 5145a078b888074de5a20d155c7178450857175b

/canonical/self-hosted-runners/run-workflows 5145a078b888074de5a20d155c7178450857175b


[FEATURE] Spam kill protection

So there is attacker and victim
Attacker should gain 1 score per kill but if he spam killing the same person he shouldn't get any more point.
Victim also should be protected from spam kill and lose only 1 point if attacker is the same person.
You could add option for setting how much time must pass to gain score or lose score when killing or dying due the same player.
eg.
spam-kill-protection: 3600s

This will need further consideration as I know players want other things from score than that (points lost on death to always work not after a cooldown for example)
This feature will require an entire score system rework with a new config file for the new score system.
This is a large project, may not be added in the next update, but will be added in the future.

This will need further consideration as I know players want other things from score than that (points lost on death to always work not after a cooldown for example)
This feature will require an entire score system rework with a new config file for the new score system.
This is a large project, may not be added in the next update, but will be added in the future.

This feature has been implemented and will be included in the next update.

This feature has been implemented and will be included in the next update.

I'm very glad to hear that

I'm very glad to hear that


FlatList Compatibility + fix for setState on unmount issue

Context

I was looking to use this lib within a component being rendered as a FlatList item. When using a large list I started to see lots of 'react-native-text-ticker: could not calculate metrics', 'nodehandle_not_found' errors and the text-ticker was displaying weird behavior with text animations (The smallest list I seen this occur was ~30 items).

Change Description

Added disabled field so that the animation can be stopped. I have used this with the FlatList's onViewableItemsChanged callback so that any item which is not currently on screen will not animate.
Added makeCancellable() to allow promises to be cancelled in componentWillUnmount(). This will cleanup any unresolved promises to resolve the unmounted setState warning. resolve and reject have been nulled to help with GC.
NOTE: This is not a perfect fix, but I believe that it is the best approach for now. More Details:
https://github.com/reactjs/reactjs.org/pull/2562

Good catch on the warning! Thanks for contributing and updating the docs too with the additional prop. I've tried your branch locally and everything seems to work like you said. Will merge and push out an update to the package now. 👍🏻

@curtismenmuir
Warning: componentWillMount has been renamed, and is not recommended for use. See https://fb.me/react-unsafe-component-lifecycles for details.

Move code with side effects to componentDidMount, and set initial state in the constructor.
Rename componentWillMount to UNSAFE_componentWillMount to suppress this warning in non-strict mode. In React 17.x, only the UNSAFE_ name will work. To rename all deprecated lifecycles to their new names, you can run npx react-codemod rename-unsafe-lifecycles in your project source folder.

Please update the following components: TextMarquee

@dimbslmh Good catch, updated initialization to be set in constructor. See new PR: https://github.com/deanhet/react-native-text-ticker/pull/66


Style - remove line below input field

Is there a way to remove the thin line? In general, it would be nice, if one could use InputDecorators to change the style of the Widget.

Hello @DKAC ,
Thanks for your question and suggestion.
There is an InputDecorator parameter but this is probably not what you are looking for. Try the underline parameter instead.

Hi @lcuis,
thanks for the hint... putting an empty Widget into the underline parameter, is working fine (with null the default underline is shown).

Thank you for the confirmation @DKAC !


please provide nodejs sample for token exchange(SSO) between root bot and skill bot

please provide NodeJS/TypeScript sample for token exchange(SSO) between root bot and skill bot.
Also is this method supported in other authentication methods like Generic Oauth 2?
As per page below:
Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: ab060f59-0fc6-d90d-6b71-a0756f1a8aa0
Version Independent ID: f7574aeb-0002-5e07-47ed-3d135a42f4ce
Content: Add Single sign on to a bot - Bot Service
Content Source: articles/v4sdk/bot-builder-authentication-sso.md
Service: bot-service
GitHub Login: @Kaiqb
Microsoft Alias: kamrani

Hi @jonathanhotono ,
This is currently being tracked via the following issue, and is scheduled to be completed for the upcoming release.
https://github.com/microsoft/BotBuilder-Samples/issues/2811
Currently, a C# sample for this exists, so you can possibly look to that sample for the high-level concepts on what it takes to accomplish what you're trying to do.


Implement netcat functionality instead of suckblow

Hopefully we can use this to test connectivity through the tunnel

Completed with ee48c1a


Use Conan lockfiles to pin recipe revisions

Fixes the build and ensures that the build is not affected by recipe changes on ConanCenter in the future.
https://docs.conan.io/2/tutorial/versioning/lockfiles.html
The lockfile can be updated with either
conan lock create . -r conancenter

or
conan install . -r conancenter --lockfile-out=conan.lock --lockfile-clean

Also makes use of the auto_shared_fpic feature to simplify the recipe a bit (https://docs.conan.io/2/reference/conanfile/methods/configure.html#auto-shared-fpic)
and drops a compiler.cppstd that is no longer necessary in Conan v2.

why do we have ranges for the versions of the dependencies in the conanfile.py, but lock ourselves to a specific version in the lockfile? Can this not be accomplished by using discrete versions in the conanfile requirements?

why do we have ranges for the versions of the dependencies in the conanfile.py, but lock ourselves to a specific version in the lockfile? Can this not be accomplished by using discrete versions in the conanfile requirements?

It's to ensure that the builds are reproducible and that there are no unexpected build failures for consumers.As I mentioned in the other comment, Conan is very prone to version conflicts, unfortunately.
The lockfile is a bit redundant with the force=True, though. I'll keep it simple and drop it.


package "tests" in Python path?

🐛 Bug
PyTorch Lightning install its tests into "test" in the Python path. I would venture that this is not ideal.
To Reproduce
Install PyTorch Lightning
import pathlib
import pytorch_lightning
print(list((pathlib.Path(pytorch_lightning.__file__).parent.parent / 'tests').glob('*')))

This is with git a checkout from today.

I thought we fixed it, but probably we need to revisit it :]

I tried to fix this in #7614 but got blocked by our examples having a dependency on the tests. This comment summarizes the problem https://github.com/PyTorchLightning/pytorch-lightning/pull/7614#discussion_r671183652
@Borda still waiting on your thoughts :P

To my mind, moving them to pytorch_lightning.testing or, if you want to emphasize they are not for general consumption, pytorch_lightning._testing (with an underscore) would be relatively standard. Doing the latter would seem to leave room for further change if any more enlightened plan comes up...
Of course, pl_examples might be good to move out of the top level, too, but the tests directory seems like a very generic thing to install in the general namespace.

The other (not terribly related) thing is the default data dir for pl_examples, but I guess it's better in a separate issue, as it is not obvious where to put it (personally, I'd look at the hub dir - os.path.join(torch.hub.get_dir(), 'pl_examples')).

I think that the pl_examples shall be completely independent on tests and we shall drop tests from the package as it can easily happen that other package accidentally writes their own tests in the site-packages and so make our examples unexecutable... there is no reason to distribute PL tests within the package

as it can easily happen that other package accidentally writes their own tests in the site-packages

Soo many packages using setuptools do that accidentally. Switching to less legacy build systems helps …


One-to-many transforms for image datasets

🚀 Feature
It would be convenient if the standard torchvision datasets could handle one-to-many transforms more seamlessly.
Motivation

Currently there are two built-in examples of this, FiveCrop and TenCrop, which take in a PIL image and return a tuple of images. The documentation recommends handling these by stacking them with a lambda function, i.e.
transform = Compose([
    FiveCrop(size), # this is a list of PIL Images
    Lambda(lambda crops: torch.stack([ToTensor()(crop) for crop in crops])) # returns a 4D tensor
])

However this makes it more difficult to add further transformations to the Compose, and more significantly requires extra work in the train/test loop to reshape the batch.
Pitch

I suggest a kind of "flat transform" which would work like a flatMap function in some functional languages, i.e. transform each input into a sequence and combine them into a single output sequence. There would be an extra parameter "flat_transform" on image datasets which would take such a function instead of (or maybe in addition to) a normal transform. There would also be a "FlatCompose" transform which would act like Compose but for flat transforms, i.e. it would apply the first transform to a single input, and each additional transform to each output, flattening each time. Iterating over the whole dataset would chain the flattened outputs from all inputs.
As a special case, transforms that do not return a list or tuple would be treated as returning a list of size one, so you can mix one-to-one and one-to-many transforms in FlatCompose.
An example could look like,
flat_transform = FlatCompose([
    Resize(256),
    FiveCrop(224),
    AddFlipped(), # Transform that returns both original and flipped image
    AddRotated(90), # Transform that returns original image and 90-degree rotated image
    ToTensor(),
    Normalize(mean, std),
])

One problem is that most datasets are random-access, which would be impossible if each image could be transformed into an arbitrary number of outputs. To solve this we could require flat transforms to have a constant "expansion factor" which would be passed to the dataset as well (maybe in a tuple to the flat_transform parameter or as a separate parameter). The example flat transform would have expansion factor 5*2*2=20. Then random access would be implemented as:
def __getitem__(self, index):
    return flat_transform(data[index//expansion_factor])[index % expansion_factor]

cc @vfdev-5

Thumbs up, this would be useful. As of now, FiveCrop requires making changes inside the model to handle one more dimension in the input.


chore: android ci

Docker Image Builder: https://github.com/decentraland/dcl-godot-cli
Docker Image in Docker HUB: https://hub.docker.com/repository/docker/kuruk/dcl-godot-android-builder
TODO:

[ ] Copy Android FFMPEG libs
[ ] Build Linux x86-64 lib (it is required to export the project correctly)

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (9446361) 40.61% compared to head (36ab625) 40.64%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #120      +/-   ##
==========================================
+ Coverage   40.61%   40.64%   +0.02%
==========================================
  Files         117      117
  Lines       14504    14504
==========================================
+ Hits         5891     5895       +4
+ Misses       8613     8609       -4

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


connection_pool with thread_safe doesn't protect if CompletionToken has a bound executor

Is it safe to run mysql operation when using thread pool within another io context?
Currently I run all operation within my main iocontext which is separate from mysql ctx and I get unhandled exceptions quite often.
Judging by the exception (access violation) address it is somewhere here. Which looks like thread safety issue happening between .empty() and .front() operations.
https://github.com/boostorg/mysql/blob/1b6a908ba9dc260f24d100e8417096c1ee393c14/include/boost/mysql/impl/internal/connection_pool/connection_pool_impl.hpp#L220
My mysql setup ctx.
boost::asio::thread_pool mysql_pool( 4 );
auto shared_state = std::make_shared<boost::mysql::connection_pool>(
	boost::mysql::pool_executor_params::thread_safe( mysql_pool.get_executor() ),
	std::move( mysql_pool_config )
);

shared_state->async_run( boost::asio::detached );

// main ioctx used by the server itself running on some threads
boost::asio::io_context io_context;

The intent is that what you are trying should be safe, but I think there is a problem in the code when the completion token you pass to async_get_connection has a bound executor. Could you please post how are you calling async_get_connection?
On the other hand, what's the use case behind using a separate thread pool for the MySQL connection pool? There's a high chance that using separate thread pools will make your code less efficient. connection_pool doesn't perform any blocking operations, so it doesn't require dedicated threads (it's all async calls in the inside).

I understand that it is has no sense in separating them. It has been done as a temp solution to test out if things are working correctly after switching to internal pools.
`boost::asio::awaitable session_db_remote::contains( const std::string& string )
{
try
{
auto con = co_await _pool->async_get_connection( boost::asio::use_awaitable );
const auto find_sql = boost::mysql::format_sql( con->format_opts().value(),
"SELECT 1 FROM backups WHERE access_token = {}",
string );
	boost::mysql::results result;
	co_await con->async_execute( find_sql, result, boost::asio::use_awaitable );

	co_return !result.rows().empty();
}
MYSQL_CATCH_BLOCK_ASYNC( "db::contains" )

}`

Moved to a single context. Will try running under the debugger if any unhandled exception occurs.

That confirms my theory. For reference, the problem is:

When you create a connection_pool with the thread_safe function, the pool gets created with an asio::strand wrapping the executor you pass, in your case the thread_pool executor.
When you co_await async_get_connection(..., asio::use_awaitable), the completion token has an implicitly bound executor: the one that you used for co_spawn - this would be your server's io_context executor.
This line in async_get_connection attempts to get to run your operation within the strand (obj_->ex_ is the strand here).
asio::post(obj_->ex_, std::move(self)) posts to obj_->ex_, then dispatches to  get_associated_executor(self, obj_->ex_) (post to the strand, then dispatch to the server's I/O context). As the strand's underlying context is different to the server's I/O context, this effectively bypasses the strand.

You should be fine if:

You run everything using the same thread_pool.
You use co_await _pool->async_get_connection(asio::bind_executor(_pool->get_executor(), asio::use_awaitable));

I don't know if there is a portable way to fix this to make it work in the general case (asio::post behavior is non-obvious even for me). We should, ideally, fix it. Otherwise, document this, as it's non-obvious behavior.
Please let me know if you find more trouble or have any other questions.

No more issues in a single context as expected.
Not sure whether you should tag this behavior as a bug, as it is actually very logical behavior.
Maybe just add a "Thread Safety" article in the README specifying that you must either use a single context or use a pool executor for MySQL operations.


Zoom Controls

For users that are less mouse centric, are there map zoom controls (+/-) that can be included in geomap visualizations?

Duplicate of #76

Thanks for getting back to me, but I don't understand the resolution of #76? Are there zoom controls available or are they coming? I don't think they are needed, but I've had two people use the geomap visualization not knowing how to zoom in and out with current mouse implementation.

They are not available, but are coming. Issue #76 is still open, and I'll update it with more specifics.


fix: nickname can be null

This fix the "invalid type: null, expected a string" error I had when requesting stripe api :)

Published in 0.4.6