Adding JSON type

Adding a JSON type, so that JSON libraries don't need to be large monolithic codebases with a lot of duplication (basic types, conversion between Swift and JSON types, manipulation convenience functions, etc). Instead, one library will be able to focus just on e.g. parsing, another on pretty printing, another on providing a BSON representation and so on. The goal is to make all of them compatible and composable.

why not this?
public enum JSON {
     case object([String: JSON])
     case array([JSON])
     case integer(Int)
     case double(Double)
     case string(String)
     case boolean(Bool)
     case null
 }

I don't think this object belongs in C7, but if it has to be, I would prefer Paulo's implementation. I don't believe json distinguishes between Double and Int.
I think this violates the principle:

Only the code that is absolutely necessary for cross-project compatibility will be included.

Because the JSON spec doesn't define two types for numbers, just one. I think we should respect that.
The reason why I want to keep Int and Double separate is so that if you receive a JSON object e.g. {"myAge": 25}, parse it into JSON and turn it back into data, you won't get {"myAge": 25.00000}.

@LoganWright We discussed this with @paulofaria and agreed it probably should go into C7. Happy to hear why you think it's a bad idea.

Yeah my vote is Paulo's implementation. Even though JSON doesn't formally distinguish between integers and floating point numbers, I think it's much more Swifty to break it up into Int and Double.

@bradhilton My proposed type does keep Int and Double separate, but also respects the JSON spec by having just one number type.

I don't think its necessary for C7 to function which I believe violates the original principles. Maybe you guys could outline the value it's adding?
The number thing of {"number" : 25 }, I think this should still parse as 25.0 and be used as a double. The addition of the duplicate number types adds unnecessary complexity to end user and I think isn't always accurate depending on what unknown 3rd party is doing the parsing. All we know according to the Json spec is that it's a number.

@LoganWright you're talking about this actually, right?
public enum JSON {
     case object([String: JSON])
     case array([JSON])
     case number(Double)
     case string(String)
     case boolean(Bool)
     case null
 }

@paulofaria both, if we're including Json, I think it should absolutely be your suggested structure.
My main concern is why we're adding it in the first place. Json libraries aren't going to import C7 just for a Json struct, and I don't understand what the value add is.

About the motivation:

Only the code that is absolutely necessary for cross-project compatibility will be included.
Currently we have no cross-project compatibility for JSON. And I see value in having a common type for JSON. Don't you? @LoganWright

@LoganWright This might be getting too philosophical, but mixing Int and Double just feels wrong, exactly because you might actually be creating incorrect results, due to the different representation of Int and Double in a computer. Int's are precise, while Double's only have a certain accuracy. I think that if you send me certain JSON object, all I do is parse it and echo it back to you, there should be no risk of the meaning to actually change - which it might if you convert an Int into Double, you might lose/cut off the actual value.

We can come back to the number thing although I feel pretty strongly that it shouldn't be Int/Double to answer this:

And I see value in having a common type for JSON. Don't you? @LoganWright

No, I don't
90% of us are using a Json lib of some sort, Honza has one, GFX has one, Zewo has one, and all have their own struct.
While I agree, it would be nice to have one, I don't think this is the way to solve it.

@LoganWright Read the description of the PR for the reasoning behind adding this. Yes, cross-JSON-lib compatibility is exactly the reason why we want to add it. It stems from a concrete example, where my JSON lib Jay focuses on great parsing, while I'm not interested in writing all the boilerplate convenience functions around managing JSON, which can be done by someone else with a very good approach instead. We aim to make JSON libs composable.

I would personally vote for not including the JSON type, and instead move StructuredData to OpenSwift. That's actually the whole goal of StructuredData - a standard that can bridge between different data types. I agree with @LoganWright, while we do all have our own versions of JSON, its more important (in my opinion) to build a common bridge between them rather than making them all the same.

This is supposed to be minimum viable, I think if we're going to include full json support w/ convenience functions etc., we're crossing the line into building another web framework.

@LoganWright Sorry, you misunderstood. This file I just added is all I want to add into C7. But it will allow multiple authors to create compatible JSON libraries: one parser, one pretty printer, one for easy conversions etc.

@LoganWright the proposal is only about the enum.. no more.. this would be the MVP imo.

I think we should have both JSON and StructuredData in OpenSwift.

Then I disagree even more, why are these here. Maybe in open swift, but not this repo. It's not necessary, the mandate is only what's necessary as minimum viable interface to facilitate communications between web frameworks different components.

@bradhilton I agree.. both with just the bare enums.

While we're at it, XML? ;)

and csv

and yaml

@LoganWright C7 is for Swift standards, if I'm not mistaken, not web frameworks (that's what S4 is for, isn't it?)

Well OK could be other repo. But C7 comes from Core.. and I think it would fit here. Even because if you're gonna build a JSON parser you're gonna need Data which is here. So a JSON type already has to depend on C7 if it wants to be Open Swift compatible. If we create another repo for this then the JSON implementation is gonna have to depend on C7 and this other repo.

You're not going to need Data unless you fully opt in, I think it's naive to expect Json libs to include the C7 library.

Okay, can we bring it back to the issue at hand? I believe there should be a standard JSON struct, which JSON libraries could conform to and users could build their own utilities around, without having to rewrite everything if they change their parser. This serves that function, allows for composition of different JSON libs.
Is anyone opposed to the idea of having a C7 standard for JSON? If not, we can get into the details. But let's go step by step.

I think we need a standard. I don't know where it should go.

Well, as I said.. if you're going to have a JSON library that is Open Swift compliant it would have to depend on Data anyway...

Would it? It's just a definition, there won't be any parsing features.

Duplicating my idea from slack there
open-swift is for standarts. JSON is already standart. JSON library is implementations of standart. But instead we can write needed protocols. serialize deserialize e.g.

@paulofaria Technically, it won't have to depend on Data, I think @LoganWright makes a point here. I would be fine with having a separate repo in open-swift for this JSON spec, even if it'll be super tiny, just this file. Would that make everyone happy?

What I mean is... if I want to create a full fledged JSON library that is compatible with Open Swift. I'm gonna need a parser.. and that parser will have to depend on C7 to be considered an Open Swift parser. So if my JSON lib want's to be Open Swift compliant it will already depend on C7 so it makes all the sense in the world for JSON to be here.

why does it have to import C7 to be compatible? GFX Json parser works fine w/ open swift and doesn't touch C7

J9 just for JSON :P

w/ separate repo, as number(Double) 👍

@LoganWright Haha, I disagree, but let's not dilute this discussion with that (we'll have that elsewhere)

Well.. sure it can work. But that doesn't mean it's compatible in the sense that it talks directly with libs that are Open Swift compatible.. using the Data representation of binary data. If I want a JSON lib that is Open Swift compliant.. it has to depend on Open Swift somehow.. and the parser would depend on C7 for Data.

@paulofaria Yes, down the stack, the app itself will most likely import both, but up the dependency tree, you might have JSON-handling-only libs that will only touch J9, never C7. See what I mean?

But ok.. you can have a JSON lib that doesn't have a parser.. then you wouldn't need to depend on C7.

Why is a Json lib open swift compliant at all? (pre J9 or whatever)

@LoganWright What do you mean? Which library?

@LoganWright so that consumer libraries can feed it C7's Data.

Yeah, but I think most of the times you'll want a parser.. haha. So for most cases we would have a JSON lib depending on C7 and this other module. I think only depending on the JSON enum would be the exception.

gfx, the original Json serializer, not sure why more came after it in the first place,
You can easily extend to incorporate c7, that's what we do, it's like 20 characters

Because we want cross module compatibility.. that's why Open Swift was created.

    public init(_ value: Data) throws {
        self = try Json.deserialize(value.bytes)
    }

    public var data: Data {
        let bytes = serialize().utf8
        return Data(bytes)
    }

Hmm... Okay. So everyone cool with open-swift having a new repo for the JSON spec? (I quite like the name J9 now, any other suggestions?)

If we're going to have another repo.. (which I really don't think it's necessary) I don't think it should be J9 only for JSON. It should be a repo for base types like JSON.

Sure, so also for XML etc?

A repo for higher level data representantion. JSON, XML, MediaPack, StructuredData, etc..

Formats... F3? 😆

I think we should leave the 9 number for when we have a module starting with K. K9 😛

Why would I import all of those for a Json library?

@LoganWright To conform to a standard?

T9 maybe? @LoganWright cause importing a whole module for just an enum doesn't make sense.

So we would have a repo for each new higher level type?

adding code for the sake of adding code seems worse than importing a module for an enum

I agree with @LoganWright... Why should JSON import something that contains specs for XML, CSV, YAML, etc.

@Danappelxx So we would have a repo for each new higher level type?

Can we add only StructuredData and add protocols like StructuredDataConvertible? I don't see any downsides.

Guys, I understand you want to keep things pure, but let's be practical here. I don't think creating a new repo for every new format makes sense. And this is a pretty good compromise IMO.

@Danappelxx StructuredData isn't the same thing as JSON though. I want a strict JSON type, compiler-checked. This is the way to do it.

@Danappelxx  Cause StrucuteredData representation is not compatible with JSON 100% of the times.

Yeah I agree with @czechboy0

I want a Json type too, but with number as Double only. It's parsing "key" : 1 isn't always an integer and "key" : 1.0 isn't always a double.
It should always be double to acknowledge the uncertainty.

@LoganWright @Danappelxx Seems to me the biggest problem you're having.. is depend on other stuff when you only need one stuff. Right? But if you don't care about the other types.. just do nothing. Even if we add types for all common higher level types si gonna be like 100 LOC at most.

@LoganWright HEY! Stop jumping around, let's settle one thing first before we start another. The int/double thing will have its own PR, I promise you'll have a chance to present your views.

So, if we do this, where do we stop? Just at the enum? what about convenience accessors that almost all Json libs have, subscripting, etc.

@LoganWright Yes, just the enum. The convenience stuff is exactly an opportunity for a JSON-lib to solve.

Yeah the idea is to have other frameworks add the computed properties, parsers, serializers, etc.

No, my issue isn't about having too much in C7. My issue is that I don't think JSON belongs in OpenSwift at all. My "dream" would be for StructuredData to be the only representation of data that your code works with, and that sounds like a standard to me. JSON is already a defined standard and I don't see the benefit of putting in C7.

I'm with Dan. I just want StructuredData in C7.

Can we move StructuredData to open swift?

I also support @Danappelxx on this

@Danappelxx StructurredData is good for when you have to represent more than one higher level data abstraction in on type. But when you only have to deal with JSON. You need a JSON type.

@Danappelxx I understand and I think StructuredData is cool, but let's be realistic here: everyone uses JSON, and there are many JSON libs which are incompatible at the moment. And again, StructuredData and JSON are not the same thing, so I don't think you can make up for the lack of a JSON Swift standardization by having StructuredData.

@paulofaria all these «JSON-only» doesn’t make any sense to me, for example, because I don’t want to deal with JSON at all, but with StructuredData instead.

@dreymonde Nobody's forcing you. We're trying to allow for people who want to use JSON, to be able to use multiple JSON libraries that each solve a different part of the problem.

We should have JSON and StructuredData at C7 or if you want to separate.. put them in another repo for the higher level types. But StructuredData and JSON do not solve the same problem. They are not the same.

If we create a standard for JSON, there will be a lot of JSON libraries which will be completely useless for developers who use StructuredData, which is kind of a better practice than to stick with JSON only.

@dreymonde Once again, JSON and StructuredData aren't the same, don't solve the same problem. I'm not even sure why we're talking about StructuredData here, as the only thing this PR proposes is to standardize JSON libraries.

@dreymonde if you don't want to deal with JSON just don't use it. You're not thinking about the bigger scope. You're thinking about web apps and content negotiation. That's not what Open Swift is about. Open Swift is about standards for Swift in general. Some applications need to have a strict JSON representation. And there's no reason we shouldn't have standard just because some don't want to use it. If you don't want to use it. Just don't use it.

@dreymonde you know me.. you know I love StructuredData for content negotiation in web apps. and I will always suggest its use for this kind of application. But we can't disallow other usages in other scenarios because of that.

@paulofaria

You're thinking about web apps and content negotiation

That’s true, yes. Now I get it. Okay, I have no objections then.

Cool, so now that we've cover many aspects, what's the consensus? I still believe the best place to add this would be here, in C7. We've seen in the conversation above that even though many of use are working on web framework stuff, C7 is about Swift standards. And I believe that the community could benefit from one small standard for JSON representation, which libraries could conform to and thus work together nicely, just like S4 does for web frameworks.
This change is strictly about JSON - is anyone still against adding this file to C7, thus allowing (but not forcing) people to take advantage of a standard JSON type?

I vote for this representation in C7:
public enum JSON {
     case object([String: JSON])
     case array([JSON])
     case integer(Int)
     case double(Double)
     case string(String)
     case boolean(Bool)
     case null
 }

I've come around, I'm open to including it here.
Would like to revisit the number part.

I think I'd prefer @czechboy0 over that one @paulofaria

Okay, let's discuss the int/double number issue. I've outlined above why I believe separating int/double is necessary to preserve correctness of transferred data. And to also stay in line with the JSON spec, I created a nested enum for int/double.

I vote for adding JSON here. It belongs here nicely. Although if we’ll decide to include other formats and StructuredData (maybe) in OpenSwift, we should reconsider this.

I like @czechboy0 approach with unified number case, it’s good for standard. Although it’s pretty weird to use it as it is, I’m sure other libraries will add more convenient ways.

Consider this: there will be convenience functions to extract the types nicely, so the fact that there's a nested enum will be quickly made pretty much invisible of users of JSON libs, because those libraries will dictate how they treat numbers in detail (conversions between int and double etc). I believe we should keep the separation here, because libraries can still treat them as the same thing. If we merge them here, libraries won't be able to tell, what the data was originally parsed as.

I vote this one:
public enum JSON {
     case object([String: JSON])
     case array([JSON])
     case number(Double)
     case string(String)
     case boolean(Bool)
     case null
 }

A JSON value MUST be an
- object,
- array,
- number, or
- string, or one of the following three literal names:
- false
- null
- true

https://tools.ietf.org/html/rfc7159

I agree with @dreymonde, but as we have versioned API I think it should stay here for now. Open Swift is far from 1.0 (I suppose we all agree it will only happen when Swift 3 gets released) so we can postpone the decision of a new repo.

@tannernelson Yes, but when you look into the RFC how it defines a number, there's no reason to believe it's a floating point number, which this would force.

Consider this: there will be convenience functions to extract the types nicely, so the fact that there's a nested enum will be quickly made pretty much invisible of users of JSON libs, because those libraries will dictate how they treat numbers in detail (conversions between int and double etc). I believe we should keep the separation here, because libraries can still treat them as the same thing. If we merge them here, libraries won't be able to tell, what the data was originally parsed as.

I agree with this, but having the nested enum just to keep "adherence" to the standard is kinda convoluted.

   and precision of numbers accepted.  Since software that implements
   IEEE 754-2008 binary64 (double precision) numbers [IEEE754] is
   generally available and widely used, good interoperability can be
   achieved by implementations that expect no more precision or range
   than these provide```

> Since software that implements IEEE 754-2008 binary64 (**double** precision) numbers [IEEE754] is generally available and widely used

I think the storage is one thing, but we need to research more about the real-life problems which sending an Int and receiving a Double might cause.

I just don't like losing information. And I see no reason to discard information if we can nicely keep the information with the type system. Again, you'll rarely work with this directly, it'll all be solved in the JSON libs. So look at it from a purely functional view, not from "I'd prefer to do less work" (because you won't work with it).

I just don't like losing information

What kind of information are you losing? The JSON spec clearly indicates that serializers should not expect better than double precision implementations. Open Swift should adhere to the spec, not what we think might be better or more advanced. Too much accuracy is a bad thing in some cases, such as measuring latitude and longitudes.

@tannernelson the problem is if you have an API that parses to JSON and then serializers without touching it you would send {"foo":1} and receive  {"foo":1.0}. that could cause problems. I'm just saying we should investigate this.

@tannernelson You lose information, because the sender might send two payloads, {"foo":1} and {"foo":1.0}, a JSON parser will parse it, and if it's a good one, it'll give you the first one as [String: Int] and the second as [String: Double].
The presence or absence of a decimal point dictates whether it's an int or a floating point number, so if you get "1" and "1.0" and your parser represents them as the same thing, you're losing information (because the sender saw it as two completely different types).

Moreover, we're creating a top level, simple standard, which I believe should allow better resolution, if practical. Keeping ints and doubles separate doesn't hurt us and will benefit people who want to keep them separate. For people who want to treat them as the same thing, they'll use a library which provides such convenience, e.g. .getNumber() -> Double, which will convert the Int to Double if necessary. But I don't see a reason why we should prevent the first group of people from getting what they want, when we have a solution which can please both groups.

I think what you're inventing is something we should call SJSON (Swift-JSON). But JavaScript Object Notation was defined in RFC 7159 and created for a language that can't even differentiate floats from ints. We're adding complexity to it that simply isn't required or expected by the specification.

You are correct, it's not required by the JSON spec, but it's allowed (according to the spec, libraries are in charge of how numbers are represented), so here we chose to represent them in a way that preserves the information of how the sender intended the number to be represented (int/double). This is a feature, not a bug :) What do you see as a problem? What scenario are you thinking about where this will be worse?

If it's not explicitly mentioned in the spec that integers and floating point numbers should be maintained between parsing and serializing, then we're portraying false levels of accuracy to the users of the enum.
Take the example where someone makes a User with an identifier of 1 and it's an integer. They use any arbitrary RFC 7159 JSON spec serializer to send it from their API.
Now they are using an Open Swift project and they want to receive this integer. Well, naturally they would check for case .integer. But if the serializer they used doesn't care about the different between an integer and a double, because it's not required to, then their check for an integer will never pass and thus they have a bug.
I've seen this a million times in PHP where serializers will turn ints to doubles or even to strings sometimes. JSON is really not as precise as you're making it out to be and we shouldn't be promising something to the user of this enum that we can't back up.

Just wanted to add that {"foo":1.0} is not natural JSON - type in JSON.stringify({"foo":1.0}) into a javascript console and you will get the output of "{\"foo\":1}".

You're just arguing against using a strongly-typed representation of JSON in general, all this can be applied to the distinction between bools/strings/numbers, so I strongly disagree. I think we should keep as much information as we can from the received data, and let a library take care of how it treats it.
In your example, the user would use a library that has a number() -> Double convenience call, which does the conversion to Double.
I believe we should use the type system to our advantage. If someone wants to convert all types to String, they can. But I don't want to prevent people (like me), who want to preserve the information.
Think about this: you map a parsed JSON onto an object, one field is "gamesPlayed". The server sends me the number "1.2", I ask for Int and it fails - but that's a good thing! I'll go, debug, see the server is sending nonsense, and fix it.
On the other hand, if I had a always Double-casting type, I'd just take the input and cut it into an Int, getting the number 1. So the server sent 1.2, but I'm using 1, waiting for a nasty bug to creep in.
See what I mean? We can both create examples of where the other's method fails, so I believe we should preserve as much information for the user of the library to make that call, instead of us trying to do so here.
If there isn't a great reason to take away a capability, I don't think we should take it away.

Since there is no Number type in Swift it makes sense to map it to the equivalent which is Int and Double. Representing it as case number(Double) is less accurate.

@tannernelson I understand you wouldn't love this solution, but would you be okay with it making it in anyway? You see, what you want to do is possible with my solution (enables us both to do what we want), but your solution prevents me from doing what I want (can't discern number types anymore).
Does anyone else have any final feedback? If you're just joining, please read the full thread so that we don't cycle in already discussed points :)

Just going to officially say that my vote is for @paulofaria 's representation:
public enum JSON {
     case object([String: JSON])
     case array([JSON])
     case integer(Int)
     case double(Double)
     case string(String)
     case boolean(Bool)
     case null
 }```

@bradhilton: So you dislike this? https://github.com/open-swift/C7/pull/32/files

I don't think we need an enum representation of Number

@bradhilton Right, but it's a good compromise between respecting the JSON spec and also keeping the number types separate. I've used a setup like that for months, and it works really well, so unless there's a big issue with it, I think I'll keep the proposal to this. Flattening the structure and putting double and int as base JSON types feels like too far of a departure from the true JSON types, if it makes sense.

I think it adds too much complexity and isn't necessary. I disagree with having the Number type.

Read my earlier comments about nobody working with this type directly, meaning it's only for the benefit of the wrapper code (even today, most JSON parsers represent the parsed structure in a type-safe manner, but only present you with convenience functions to take a String out, or an Array etc). So this complexity is only for communicating the distinction of the parsed number, but won't be felt by users at all. Which I think is a good balance.

public enum JSON {
    case object([String: JSON])
    case array([JSON])
    case number(Double)
    case string(String)
    case boolean(Bool)
    case null
}

extension JSON {
    var integer: Int? {
        if case .number(let value) = self {
            return Int(value)
        }
        return nil
    }
}

Not the same thing, as if I send 1.0, it's a double but this function would actually claim it's an int. Just because something is convertible to int doesn't mean it was intended to be used as int. But just read the whole thread, I've explained my reasoning.

if you send 1.0 then use .number, if you using .integer - know what you doing.
How would you know someone sent a Double from client-side javascript while it was serialized as 1.
i like having
integer(Int)
case double(Double)

but it's not "standard".

I vote for my JSON representation! hahaha, does it count? 😆

@paulofaria
how do you propose to handle this [{ "type": 1 },  { "type": 3.14 }] when i'm expecting Double.
for item in json {
  switch item {
    case .integer(let value): // do things
    case .double(let value): // do things
    default: throw Error()
  }
}

like this?

@czechboy0 or like this? 🙂
for item in json {
  switch item {
  case .number(.double(let value)): //do things
  case .number(.integer(let value)): //do things
  default: break
  }
}

@antonmes it depends on the higher level lib implementation.. the higher level lib can have json.double()only if one prefers and never expose the integer.
extension JSON {
  func double() -> Double? {
    switch self {
    case integer(let i): return Double(i)
    case double(let d): return d
    default: return nil
  }
}

so what the point having this in standard if we'll use different api anyway.

Cause the important part will be cross-project friendly. Why have Open Swift if our web frameworks will have different API? Haha, is the same question.

So why you're not happy with Double if you're not going to work with it?
If it's just for cross-project communication? Why doing standard not standard then?
cons:
having non-standard json
having redundant parser checks
having redundant switch statements

i would prefer
for item in json {
  guard case .number(let value) = item else {
    default: throw Error()
  }
  // do things
}

The problem is that we can have bugs because of that. And don't say it's non-standard JSON, cause it is. The way the type is represented internally and how it is stored is only relevant in terms of its capability to effectively represent the raw data values. Meaning.. if you have a JSON like so:
{
  "foo": -1.5
}

And you represent -1.5 internally with two Int64 one for the significand and one for the base and a Bool for the negative sign. It's still going to be standard JSON. Cause it is capable of representing 1.5. Now it wouldn't be standard JSON if it wasn't capable of representing that. The parser checks won't be redundant cause JSON allows integer, floating points and exponential notation. All of which need different treatment in the parser.
{
  "foo": 1
}

{
  "foo": -1.5
}

{
  "foo": 1E10
}

On the consumer side you should never have redundant switch statements. This is a type that is supposed to be extended in a higher level library. That library will provide you with convenience computed properties. Even properties that could force convert from one type to another.
The main point about keeping two types for the number is about preventing bugs, that I can easily spot if we cast everything to double. We want things to adhere to the RFC (which it does) and also be bug repellent (which it does).

It supports those different types, yes, but they're not to be considered distinct according to Json spec. If Json spec said include . to indicate floating point, otherwise it's an Int, that's a different story.
Here we'd be actively imposing our desires onto a spec that doesn't adhere to them. Json includes number which is basically a floating type because that's how Javascript handles it.
If we're writing all the json parsers on both side and want a custom type that represents Integer that way, it's fine for a personal lib. But for global standards, we can't control every json serialization on the web and should stick to standard spec.
For all the specificity, integer, etc. I'd rather stick w/ zewo/StructuredData or qutheory/Node and use a specific serialization there that allows better accuracy.

integer + double = workaround in the core + nonspec parser + workaround for user-friendly api
number =  + workaround for user-friendly api

Ok! Taking in consideration what you guys are saying about keeping the number case and specifically what @czechboy0 and @bradhilton said:

Since there is no Number type in Swift ...
@bradhilton

Because the JSON spec doesn't define two types for numbers, just one. I think we should respect that.
@czechboy0

I really think we should create a Number type for C7. I think this makes everyone happy. Another plus is that the Number type can be useful for a plethora of applications. So my proposal is:
Number.swift
public enum Number {
    case byte(Byte)
    case integer(Int)
    case double(Double)
    ...
}

JSON.swift
public enum JSON {
    case object([String: JSON])
    case array([JSON])
    case number(Number)
    case string(String)
    case boolean(Bool)
    case null
}

And Number here should be self-sufficient (no need for external libs extensions), so we would add computed properties converting from one type to the other and what not. With this we can keep the number specified by the RFC and still be bug free.

I really don't think we should be redefining number, particularly to include byte, that's not really Json spec. Json spec says double.

@LoganWright this is definitely not for JSON. This is a Number type.. a type that represents a number. As I said this can be used for a whole lot of other things. I went to look for implementations in other languages and this is what I found for Java.
https://github.com/google/gson/blob/master/gson/src/main/java/com/google/gson/JsonPrimitive.java#L55
They use Number
https://docs.oracle.com/javase/8/docs/api/java/lang/Number.html
This is definitely the best type to represent a number.. can we agree with that? And if JSON states a number primitive that is.. a number.. what better type than Number? JSON RFC never states it has to be a double...

I disagree that it's the best type to represent a number, I don't think it's a necessary addition and only adds confusion to the json type. We have to remember that this is Javascript, not Swift.
RFC doesn't need to because javascript doesn't distinguish, it only has number type which is a double.

Yeah javascript number type might be a double (I don't know javascript, I'll have to believe you there). But no, this is not javascript. This is Swift. Just as Java uses their Number type I think it makes all the sense in the world for a number to be Number in Swift. All languages have a number abstraction. If Swift doesn't have one and we need it (I'm telling you, we need it. We can't get away with just casting to Double in every scenario and not even thinking about the consequences), we should create it. And again Number is not for JSON. It would be used by JSON but it wasn't made for it. It's just a number abstraction. I wish we could get away with just Double but unfortunately we can't.
Swift is great because of it's type-safety and we should harness that, not seclude it. Keeping type information is a great thing and I can't understand why you're being so against it.

Number abstraction is for another thread, and it is javascript. We're using it in Swift, but it is javascript. Javascript object notation. Unless we can control every parser on every endpoint, then we should follow suit to the type we're importing. I don't think it's necessary to bring a new number type into it, but maybe that's the answer.
Let me ponder on the Number type and see if others way in there.

@paulofaria because in this case we are not keeping it, we making it up, because we can't distinguish 1 and 1.5 which came from client-side

@antonmes I'm not sure I understand. Can you give me example?

so developer should decide, which type he expecting, and throw an error otherwise.
example: https://github.com/open-swift/C7/pull/32#issuecomment-219586570

Sorry I still don't understand.. so you parsed [{ "type": 1 }, { "type": 3.14 }] the representation will be:
.array([.number(1), .number(3.14)]) what about it?

in another words: we can't keep the type it because json doesn't.

In JSON, the string {"number":1.0} does not exist. If you serialize ["number":1.0] it will pop out as {"number":1}. Therefore, if you then parse that same JSON and assume that number is an Int, you are making up type information. Hence, it would be more accurate to parse it as a Double at all times, rather than assuming that it is an Int if it doesn't have a decimal point.

JSON doesn't keep the type? What do you mean.. the type is Number. That is the type.

In JSON, the string {"number":1.0} does not exist.

I can't understand this sentence at all haha

If you serialize ["number":1.0] it will pop out as {"number":1}

Where did you get this from? Isn't this implementation dependent?

https://github.com/open-swift/C7/pull/32#issuecomment-219566099

@paulofaria open devtools in your browser and paste it into the console JSON.stringify({num: 1.0})

Well! I'm sold by the fact that Java uses Number to represent well... numbers.. in their JSON library (specifically Google). And as I said before.. just because javascript represents number as double it doesn't mean we have to as well in Swift. I really can't understand why you'd prefer having less type information using Swift. I'm telling you this could be a source of bugs.

I wouldn't prefer less type information, I'd prefer honest type information.

I'm telling you this could be a source of bugs.

Adding the number type is a guaranteed source of bugs. As @Danappelxx and @antonmes showed in dev tools, 1.0 => 1 so would parse as an Integer instead of its true type a Double. This isn't more type information, it's inaccurate type information.
By using Double, as implied by the spec, we're at least honest about that uncertainty.

I would not refer to java api  😂 but.. they use it because java has it. Swift doesn't.
We don't have implicit casts as well, and this is more about making up the type, not saving it.
A agree that types matter, and if you expecting integer - you should check the value and cast it or throw an error, but it should be done by developer, not by a guessing parser.

The thing is Double is not implied by the spec. It is the dev tools implementation, simply because javascript can't do different. It doesn't mean we have to do it like javascript. Again, this is Swift. We wouldn't be misleading in any way. If I parse {"foo":1} and serialize it I expect  {"foo":1} not {"foo":1.0}. Just because javascript isn't capable of that doesn't mean that we should do it as well. I really really really don't know why you want to do the javascript way given what we have is Swift.

The spec is based on Javascript, is it not? Number in the context of the spec can be interpreted as a Javascript number, which is just a floating point (Double).

Hahaha nope.. stop it.. you're pushing it. And I really can't understand why. The spec only talks about the precision.

as i said before, we should check the type anyway, the difference is that done by parser and Number type, ooor it can be done through convenience propeties (optional) which way is more swifty?

Not convinced? Apple does it the same way. (excuse my Swift 2.2)
import Foundation
let dict = ["hello":1.0]
let data = try NSJSONSerialization.dataWithJSONObject(dict, options: .PrettyPrinted)
let json = String(data: data, encoding: NSUTF8StringEncoding)
print(json) // -> Optional("{\n  \"hello\" : 1\n}")

And again, we're not losing concrete number the type because json doesn't provide  it.
Conversely, we're splitting it in a different types and have to clue it later.

@Danappelxx I'm not sure that's the best example, since Apple also wraps BOOL in it's NSNumber type. I'm sure if they reimplemented it now it would be different.

Also with the Number type we would definitely have this  😂

As much as we'd like it not to, JSON uses Doubles. Anything else is dishonest and error prone. To extremely ubiquitous parsers, Chrome, and Apple both send 1.0 as 1. Although it comes as 1 w/o a ., it is a Double and should be treated as such.

@antonmes the issue is the parsing/serializing part..

If I parse {"foo":1} and serialize it I expect {"foo":1}

But ok this thread has become too long. I think it's by far the biggest thread hahah. My point is Number is a more powerful type abstraction which is something that Swift can provide which allows us to keep the parsing/serializing stuff above. If we don't use Number and use Double we'll be just like javascript. And I can't understand why you prefer that since the Number approach is still RFC compliant and more powerful. Both approaches would need the same computed properties so in the end for force-casting to a given type you want to receive would need the same API. The difference is that with Double you lose the format information.. so it would be less powerful. I prefer Number, but I'm OK with Double if that's the majority. Really hope we don't have bugs. But still.. why you prefer the javascript way will still forever be a mistery... hahahahahaha 😈

Regardless of Number being useful, JSON is not a Number, it's a Double. Perhaps Number has value, but should be discussed in another thread.

Really hope we don't have bugs

Honesty about uncertainty is less error prone and will have fewer bugs than false certainty.

why you prefer the javascript way will still forever be a mistery... hahahahahaha 😈

I don't prefer it, I accept it.

@paulofaria why javascript way? i don't want parser to decide for me is int integer or double. i thinks it's more error prone than having the original value and optional properties. without redundant matches.

JSON is not a Number, it's a Double.

In Javascript yeah. But Java, for example, no. See that's what I don't understand haha. Why you're using javascript as the definition of JSON. Just because javascript represents JSON numbers as Double it doesn't mean all languages should. JSON may have started with javascript but it's not javascript. The RFC never says it should be represented as Double it only talks about the precision. But this is getting far too old.

@paulofaria because Java doesn't have extensions and optional properties and we do. 😄

@antonmes it's all about this:

If I parse {"foo":1} and serialize it I expect {"foo":1}

to me keeping 1 as 1 would be the original value in:

i thinks it's more error prone than having the original value

let's move to slack

I want to make sure I refute some frequently repeated false assumptions

"JSON spec defines double" - it doesn't, it says the library is in charge of representation, which I'm proposing (by giving the library enough precision to make the right call)
"Apple mixes int and double" - they have to, because they use NSNumber. They also mix Booleans in, so are you suggesting we don't parse bool separately from numbers?
"It's too much work to have int and double separate" - it's not, as you'll never work with it directly, just as you don't work with it directly now, you use the library's convenience initializers and getters, so no, there's no downside
"JSON is Javascript, which only has double" - JSON is not Javascript, it's a notation with a similar syntax. JSON is to Javascript as much as Javascript is to Java.

I still don't understand why some people are against parsing as much type information as possible, when we have a chance to nicely represent it in Swift. There is not Number type, so we have to represent is in a way that makes sense in Swift. I'd be fine with @paulofaria's suggestion for adding a Number type (or the way I proposed it, either is fine with me).
We need to make a decision of how we represent a number. I am for int and double, depending on what we parse, because Int's and Double's are very different representations, so mixing them would always make the other less precise (for no benefit).
In Swift, we like type information, it makes the code safer.
Once again - with my proposal, both our and your way of usage is possible. Both parties can be happy. With yours, only you are happy, our side is suddenly robbed of type information, which would be very useful for higher strictness.
To pull it all together - please understand that I wrote a JSON library, read the JSON spec and spent quite some time thinking about it and testing it, and this feels like the best way to represent a number in a world where we can easily tell them apart. I prefer to give my users the maximum information that I parsed, so that they can use it, because Double/Int/Bool/String are all very different types to me (and most people). Just because on the web they sometimes get mushed together doesn't mean all applications do that.
If you're really so against this proposal, which I don't understand, as there are many more more disputable things in C7 that never had a peer review of this magnitude, I guess I'll have to step back and close this PR. I can't in good faith mix Doubles and Ints and call myself a computer scientist. If someone else wants to do it, go ahead, but I won't be behind a standard like that.
Once again, with my proposition, we can make everyone happy. With yours, only you will be happy. That alone should be something to think about.
I'd propose this still gets merged, and if we encounter real problems (which I don't believe we will), we can change it, I'd be fine with that. Unfortunately all the previous conversation was about very hypothetical cases, that I've never seen any actual evidence happen. If we do merge it, I already have two JSON libs that both can benefit greatly from being compatible, and down the road users will benefit.
Don't be selfish, if we can make both groups happy, I don't see why we can't.

as you said https://github.com/open-swift/C7/pull/32#issuecomment-219579822

Not the same thing, as if I send 1.0, it's a double but this function would actually claim it's an int. Just because something is convertible to int doesn't mean it was intended to be used as int. But just read the whole thread, I've explained my reasoning.

and with current implementation we would have integer while client was sending float. https://github.com/open-swift/C7/pull/32#issuecomment-219605570

open devtools in your browser and paste it into the console: JSON.stringify({num: 1.0})

so we have to handle both cases everywhere in our code.
you're saying you worry about losing type information, but we just literally can't know was it double or integer on the client side, so the parser makes assumption that it was an integer. Because 12.00 would be serialized as 12. And you can do the same check later on e.g. if value % 1.0 == 0.0
how is it better making assumptions on parser level vs checking values with optional properties?
this is only issue i worried about. not those assumptions you mention: the type is already lost.
(@paulo, by saying type i mean int vs double. not number 😜)

extension JSON {
    var integer: Int? {
        if case .number(let value) = self where value % 1.0 == 0.0 {
            return Int(value)
        }
        return nil
    }
}

public enum JSON {
case object([String: JSON])
case array([JSON])
case number(Double)
case string(String)
case boolean(Bool)
case null
}
extension JSON {
var integer: Int? {
if case .number(let value) = self where value % 1.0 == 0.0 {
return Int(value)
}
return nil
}
}
var answer = JSON.number(42)
var pi = JSON.number(3.14)
if let _ = answer.integer {
print("the answer")
}
guard let _ = pi.integer else {
print("pi zdec")
exit(0)
}

@antonmes I don't think I understand.
You're talking about what some Javascript parsers do - they can only do what their language allows them to. We have a better language that allows us to discern between the two, so I think we should.
If you send me "1.0", I'll parse it as Double, if you send me "1", I'll parse it as Int. Why is this bad?

@antonmes You also gave me the perfect example of how bad this looks
You're suggesting:
public enum JSON {
    case object([String: JSON])
    case array([JSON])
    case number(Double)
    case string(String)
    case boolean(Bool)
    case null
}

Let's read it from the top:

object -> dictionary (cool)
array -> array (cool)
number -> double? (what? why?)
string -> string (cool)
boolean -> bool (cool)

Once again, since Swift doesn't have a Number type, we need to create one for this purpose. And two fundamental ways to represent numbers are Int and Double.
I think I'm done explaining this. If anyone is unsure, read the whole conversation.
I believe my proposal is in line with the goals of the Swift language much more than mixing doubles and ints, so I'm going to keep this.
Even if you're not 100% happy with how it looks, the way you want to use it is still possible (mixing doubles and ints, with the help of a library), thus my proposal doesn't prevent you from doing anything (your modifications would prevent me, though). And once again, nobody will use this type directly except for convenience libraries and parsers, so the point is moot.
In light of this, I'll merge this the way it is. If you can gather actual information (data, not opinions) about how this is actively bad, I'll listen. But this cannot be a veto-based process, I just asked you all for feedback. Thanks for it, I hope I answered all your concerns. Even if you don't fully agree, I explained why I think this is the best solution and in order to get anywhere, we need to build things, not spend days on talking about something that can only be properly understood by being used in the wild.
I hope I can convince you about this being the right move in the following days, when I'm planning to make two different libraries compatible with this new standard Swift JSON, and you'll see how beautifully they can work together. This is why I started this and I believe the benefits will hugely outweigh any downsides.
Thanks everyone, it was exhausting, but I'm glad everyone had a chance to give their opinion. If you come up with something new (not already mentioned above, please read the full thread first), just comment below and I'll try to react.
To the future! 💯

I agree with you on everything except the forced merge. I reverted the commit. We should keep discussing this until we reach a consensus. I'll create another PR with your branch because this one is lost.

Feel free to keep discussing, but I'm out. Unfortunately the discussion
isn't about the technical points anymore and basic computer science
principles are ignored, so I'll go work on something else instead. Let me
know if this actually ever gets merged, and I'll be happy to reconsider.
On Tue, May 17, 2016 at 1:18 PM Paulo Faria<EMAIL_ADDRESS>wrote:

I agree with you on everything except the forced merge. I reverted the
commit. We should keep discussing this until we reach a consensus. I'll
create another PR with your branch because this one is lost.
—
You are receiving this because you modified the open/close state.
Reply to this email directly or view it on GitHub
https://github.com/open-swift/C7/pull/32#issuecomment-219700622


Please add an option to disable the auto redirecting to the "Roblox crashes or Doesn't launch" link.

Acknowledgement of preliminary instructions

[X] I have read the preliminary instructions, and I am certain that my problem has not already been addressed.
[X] I have thoroughly looked through the available Wiki articles and could not find a solution to my problem.
[X] I am using the latest version of Bloxstrap.
[X] I did not answer truthfully to all the above checkboxes.

Bloxstrap Version
v2.8.0
What problem did you encounter?
Roblox crashes or does not launch
Because I am using a modified Windows, it always gives me the error and sometimes doesn't open Roblox or just opens it while the error is still running. Add an option to disable the auto redirecting to the github, it affects my low-end PC. Before the Bloxstrap's update was forced my Roblox was working PERFECTLY fine and no issues or errors whatsoever.
Bloxstrap Log
N/A

Same issue, I hope they respond 😔

Fixed in 2.8.1 (assuming so, reopen if not)


Enable metadata-only Content Generator imports when not using v1

LGTM, merged. Haven't really tried running any prod builds with the recent PRs, hope that's OK.


VLAN is not a filter for ipam.prefix dynamic groups

Environment

Nautobot version (Docker tag too if applicable): v1.5.16
Python version: 3.10.11
Database platform, version: Postgres 15.1
Middleware(s): N/A

Steps to Reproduce

Create Dynamic Group with content-type of ipam.prefix
Edit Dynamic Group from Step 1.
Observe that VLAN is not a filter

Expected Behavior
Ability to filter ipam.prefix Dynamic Groups by VLANs

Observed Behavior
Unable to filter ipam.prefix Dynamic Groups by VLANs
Demo Nautobot Example (diff version but same result)
Slack thread for reference.
Glenn M. skillfully pointed to PrefixFilterForm missing the vlan_id & vlan_vid that do exists on the PrefixFilterSet

Related: #1512, #2782.


Add default to SchemaBuilder

I noticed default is missing from SchemaBuilder

Wait did I dismissed it

It was a misclick I suppose, you were right with default I will provide a new PR

Ah further confused. You already merged the proper one, so no problems.


Add Profiling

With this PR profiling is available to the ChainerIO. Format used in chrome://tracing is also supported, which enables to view the profile in chrome tracing.
The following changes have been made in this PR.

Add profile module to ChainerIO to replace the dummy one
A decorator has been added to all the I/O functions in ChainerIO for profiling
A Chainer like context manager has been added to mange the global and thread local configuration and context
A chrome://tracing-compilant format writer to output profile
Unit tests for the profile module

Closing in favor of new V2 API.


New Field: Residential Situation

For the FI it makes a difference in the affordability calculation if the applicant is renter or home_owner. depending on that the bank needs to consider other costs.
Therefore, we would like to have a new field in the object "Financial Situation":
name: residentialSituation
Type: string, enum: [home_owner, renter]

As mentioned, so far focus was on selfowned property. But rented object are on our agenda for the future.

Agreement MP / TKB on 29.09.22: Ok


feat(ui): add history for traces

History and Favourites
This PR adds the ability to

Store and view your recently viewed traces
Favourite a trace and refer back to it at a later time

Config
You can enable and disable history and or favourites in the config panel.
Favourites and history are enabled by default.

History
Each time a trace has loaded an entry will be stored in local storage and accessible via the history panel.

Favourites
You can click the star on the trace list to favourite a trace, and then it becomes available in the Favourites panel.

After second thought, I do wonder about the real value of having history and favorites togglable, and maybe they should just be enabled by default.


file beat Alpha (5.0)

I need to push my logs to kafka and for that I might need filebeat 5.0, though it is in alpha. But I am not sure about the repo and other settings and I am relying heavy on readymade cookbooks! Is there any cook available? Help is greatly appreciated.

@osmancis you can use the default['filebeat']['yum'] or default['filebeat']['apt'] settings documented in the readme to set repositories. 5.0-alpha repository and keys are documented here. I use "https://packages.elastic.co/kibana/5.0.0-alpha/debian stable main" for installing 5.0.
As for use, v5 largely extends syntax. This cookbook doesn't yet support the extended features of 5.0 in the LWRPs (e.g. no JSON input for prospectors), but if you're not in a place to add a pull request enabling them, you can always simply use the default['filebeat']['conf_dir'] attribute and add your own output YAML as a template.
The filebeat output configuration is not restrictive (nor is any other part of the node['filebeat']['config'] object); it's just a straight to-YAML rendering of the node['filebeat']['config']` object, and you're free to add any arbitrary elements via attributes.
I doubt you'll have any issues using the cookbook with 5 here. I'll try to send in a PR to support some of the newer parameters if I have time this week.

@osmancis  Currently beats 5.0.0-alpha* is not available to install via repository, at least not in yum.
# yum search filebeat --show-duplicate
Loaded plugins: priorities, update-motd, upgrade-helper
193 packages excluded due to repository priority protections
=========================================================================================================== N/S matched: filebeat ============================================================================================================
filebeat-1.0.0~beta4-1.x86_64 : Sends log files to Logstash or directly to Elasticsearch.
filebeat-1.0.0~rc1-1.x86_64 : Sends log files to Logstash or directly to Elasticsearch.
filebeat-1.0.0~rc2-1.x86_64 : Sends log files to Logstash or directly to Elasticsearch.
filebeat-1.0.0-1.x86_64 : Sends log files to Logstash or directly to Elasticsearch.
filebeat-1.0.1-1.x86_64 : Sends log files to Logstash or directly to Elasticsearch.
filebeat-1.1.0-1.x86_64 : Sends log files to Logstash or directly to Elasticsearch.
filebeat-1.1.1-1.x86_64 : Sends log files to Logstash or directly to Elasticsearch.
filebeat-1.1.2-1.x86_64 : Sends log files to Logstash or directly to Elasticsearch.
filebeat-1.2.0-1.x86_64 : Sends log files to Logstash or directly to Elasticsearch.
filebeat-1.2.1-1.x86_64 : Sends log files to Logstash or directly to Elasticsearch.
filebeat-1.2.2-1.x86_64 : Sends log files to Logstash or directly to Elasticsearch.
filebeat-1.2.3-1.x86_64 : Sends log files to Logstash or directly to Elasticsearch.
filebeat-1.2.3-1.x86_64 : Sends log files to Logstash or directly to Elasticsearch.

Perhaps adding an attribute to skip package install via this cookbook could be useful when using wrapper cookbook to install beats 5.x.
I will also give it a try to add support for 5.x configuration parameters.

@vkhatri Actually it is available via yum; you just need to use the docs that I linked to and set the repository parameter. They're using an alternate repo for alpha releases.
[kibana-5.0.0-alpha]
name=Kibana repository for 5.0.0-alpha packages
baseurl=https://packages.elastic.co/kibana/5.0.0-alpha/centos
gpgcheck=1
gpgkey=https://packages.elastic.co/GPG-KEY-elasticsearch
enabled=1

@bradq  For some reason, it did not work for me. Does it still work for you?

This is not working for me either, unfortunately. I can't seem to find the right repo.
Added this as /etc/yum.repos.d/beats.repo:
[beats]
name=Elastic Beats Repository
baseurl=https://packages.elastic.co/kibana/5.0.0-alpha/centos
enabled=1
gpgcheck=1
gpgkey=https://packages.elasticsearch.org/GPG-KEY-elasticsearch
metadata_expire=3h
sslverify=true

Then I did a yum clean all and a yum makecache.
But it seems that only Kibana is avaible in the repo:
$ yum --disablerepo="*" --enablerepo="beats" list available
Loaded plugins: priorities, update-motd, upgrade-helper, versionlock
Available Packages
kibana.i386                                                          5.0.0_alpha3-1                                                         beats
kibana.i686                                                          5.0.0_alpha5-1                                                         beats
kibana.x86_64                                                        5.0.0_alpha5-1                                                         beats

Since I'm using JSON logging, I need a 5.x version of Filebeat, but I can't think of any way to install it with this recipe.

@chopmo @vkhatri Sorry for the delayed response...I'm in a new job that doesn't use EL or Filebeat. I'll validate these in a few hours and get back to you, as it seems that their documentation might have changed?
I definitely used this without issue, though...there may have been a repo change as they get ready to shift, or I just recorded it here in a dumb way.

@bradq Looks like it was indeed removed for alpha versions but planning to add it back for 5.x beta.
@chopmo As beats are supposed to be available to install via official repository, i prefer to keep it that way. Adding capability to download and install from a package url is certainly possible, but it will increase the complexity.
However, i am open to suggestions.

@vkhatri I think it definitely makes sense to stick to the repos.
I think my solution for now will have to be writing a custom recipe to upload a binary package and install it. Then I can go back to using this recipe once 5.x is available from the repos.
Thanks for the responses, both!

Beats docs updated. Will update cookbook shortly.
https://www.elastic.co/guide/en/elasticsearch/reference/5.0/rpm.html
https://www.elastic.co/guide/en/elasticsearch/reference/5.0/deb.html

working on adding 5.x support

Added v5.x support iv v0.4.3.


Proxy re encryption with 3rd party encrypting message

Hello,
I'm trying to achieve the following scheme
C encrypt a message with A public key and send it to the Proxy
A creates a re-encryption key that the proxy can use to share data with B
I managed to achieve it only if the re-encryption is signed by C
pre_schema1_ReKeyGen(keysA.sk, keysB.pk, signingC);
This is not very practical in the following situation
A has a data repository, managed by Proxy
anyone can contribute to this data repository by positing messages encrypted with A.pk
A select who can read these data by creating reEncKey for each recipient.
In this situation it would mean that A would have to create n reEncKeys for each Contributor/Recipient pair.
Do I miss a point that would allow to have only one reEncKey per recipient ?
Here is a sample working code
let ecc;

async function main() {

  // client A setup public/private keys and signing keys
  const keysA = await ecc.pre_schema1_KeyGen();
  const signingA = await ecc.pre_schema1_SigningKeyGen();

  // client B setup public/private keys (signing keys are not used here)
  const keysB = await ecc.pre_schema1_KeyGen();

  // proxy server setup signing keys
  const signingProxy = await ecc.pre_schema1_SigningKeyGen();

  // client C is the origin encrypting party
  const signingC = await ecc.pre_schema1_SigningKeyGen();

  // client C select a plaintext message
  const message = await ecc.pre_schema1_MessageGen();
  const messageS = uintArrayToString(message);

  // client C encrypts the message with A key
  const ciphertextLevel1 = await ecc.pre_schema1_Encrypt(
    message,
    keysA.pk,
    signingC);

  // client A is able to decrypt ciphertextLevel1
  const messageDecrypted1 = await ecc.pre_schema1_DecryptLevel1(
    ciphertextLevel1,
    keysA.sk,
    signingC.spk
  );

  console.log(' Message decrypted == message ', (messageS == uintArrayToString(messageDecrypted1)));

  // client A allows client B to see the encrypted

  // client A creates a re-encryption key that the proxy can use
  // to re-encrypt the ciphertext (ciphertextLevel1) in order for
  // client B be able to recover the original message
  const reEncKey = await ecc.pre_schema1_ReKeyGen(
    keysA.sk,
    keysB.pk,
    signingC // ⚠ HERE I WOULD LIKE TO AVOID USING `C` signing Key
  );

  // the proxy re-encrypt the ciphertext ciphertextLevel1 with such
  // a key that allows client B to recover the original message
  const ciphertextLevel2 = await ecc.pre_schema1_ReEncrypt(
    ciphertextLevel1,
    reEncKey,
    signingC.spk, // ⚠ HERE I WOULD LIKE TO AVOID USING `C` signing Key
    keysB.pk,
    signingProxy
  );

  // client B is able to decrypt ciphertextLevel2 and the result
  // is the original plaintext message
  const messageDecrypted2 = await ecc.pre_schema1_DecryptLevel2(
    ciphertextLevel2,
    keysB.sk, signingProxy.spk
  );

  // now both client A and client B share the same plaintext message
  // messageDecrypted is equal to message

  console.log(' Message recrypted/decrypted == message ', (messageS == uintArrayToString(messageDecrypted2)));

}

function uintArrayToString(u8) {
  return Buffer.from(u8).toString('base64');
}

function stringToUintArray(b64) {
  return new Uint8Array(Buffer.from(b64, 'base64'))
}

(async () => {
  ecc = await import('@aldenml/ecc');
  await main();
})();

Hi @perki, I'm glad you are giving this library a try. You are right, the primitives in this "schema1" are not adequate for your use case, it does not scale.
I think your problem can be solved with an additional clever trick on the keys of contributors and recipients, and I believe this is solved in the paper "Cryptographically Enforced Orthogonal Access Control at Scale" by Bob Wall and Patric Walsh.

...allows the decision about the groups to which to encrypt a piece of data to be made independently and asynchronously from the decision about who belongs to a group and can therefore decrypt the data.

That's the foundation of their product of group management and PRE.
Still, there is a cost when an entity is added or removed from the group since keys need to be regenerated iirc.

Hello @aldenml,
Thank you for your answer.
My first successful implementation is with Ironcore's library from Bob Wall and Patric Walsh. Now I start testing others PRE flavors such as your implementation.
For my use case, I don't need to "add" / "remove" entities from a group.
But I need anyone to able to contribute.
A not fully end-end to encryption, but could be acceptable for my use case, is then to let the proxy encrypt plain text messages with A pk. Then A could generate a reEncryption key (implying that A knows the proxy's secret signing).
Thanks again and kind regards,
Pierre-Mikael

Feel free to re-open if you want to continue the thread


add native file sharing feature

Add native file sharing functionality and fix issue from https://github.com/flutter/plugins/pull/970

I signed it!

I signed it!

I signed it!

@ianloic @sethladd can you plz help us - we signed both the CLA but the commit email is diff. We cant change that. Can you set cla:yes plz

Would love to see this get merged, I've been using it for months now as a git dependency.

I signed it.

@ianloic @sethladd Hi guys, how many time we need to wait for this PR ? (opening in mars 2019.)

Btw I use this change in my main project with no problem during all this months.

Thanks for your fixes :). Didn't see this pull request. Could probably have saved myself time rewriting everything and start off from your branch!
I have updated my pull request #970 and actually fixed a small bug that seems to exist in this code (on iOS related to text + file if I remember correctly).
Please try my branch to see if that one works as well for you. Or if you have any comments there.


Automated cherry pick of #3510: Fix the issue of local probe bypassing flows on Windows

When proxyAll is enabled, kube-proxy can be replaced by AntreaProxy, then
Service traffic and non-Service traffic can be distinguished by ServiceCTMark
and NotServiceCTMark. Service traffic with ServiceCTMark should not bypass
Network Policies, and non-Service traffic generated by kubelet with
NotServiceCTMark should bypass Network Policies.
Signed-off-by: Hongliang Liu<EMAIL_ADDRESS>

/test-all-features-conformance
/test-conformance
/test-e2e
/test-flexible-ipam-e2e
/test-ipv6-conformance
/test-ipv6-e2e
/test-ipv6-networkpolicy
/test-ipv6-only-conformance
/test-ipv6-only-e2e
/test-ipv6-only-networkpolicy
/test-multicluster-e2e
/test-networkpolicy
/test-windows-conformance
/test-windows-e2e
/test-windows-networkpolicy
/test-windows-proxyall-e2e
/test-integration

Codecov Report

Merging #3715 (e38ce74) into release-1.6 (f3208e7) will decrease coverage by 9.33%.
The diff coverage is 45.00%.

@@               Coverage Diff               @@
##           release-1.6    #3715      +/-   ##
===============================================
- Coverage        64.19%   54.86%   -9.34%
===============================================
  Files              278      392     +114
  Lines            27837    43126   +15289
===============================================
+ Hits             17870    23659    +5789
- Misses            8045    17055    +9010
- Partials          1922     2412     +490

Flag
Coverage Δ

integration-tests
38.19% <ø> (?)

kind-e2e-tests
51.04% <45.00%> (-0.24%)
:arrow_down:

unit-tests
43.34% <20.00%> (+0.01%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

pkg/agent/openflow/pipeline.go
72.84% <26.66%> (-1.33%)
:arrow_down:

pkg/agent/openflow/client.go
68.65% <100.00%> (-0.77%)
:arrow_down:

pkg/agent/openflow/pod_connectivity.go
68.96% <100.00%> (+0.54%)
:arrow_up:

pkg/agent/controller/networkpolicy/reject.go
76.11% <0.00%> (-9.71%)
:arrow_down:

pkg/agent/nodeportlocal/k8s/annotations.go
83.87% <0.00%> (-9.68%)
:arrow_down:

pkg/agent/util/sysctl/sysctl_linux.go
38.88% <0.00%> (-5.56%)
:arrow_down:

pkg/controller/networkpolicy/status_controller.go
65.89% <0.00%> (-4.05%)
:arrow_down:

pkg/controller/egress/store/egressgroup.go
56.94% <0.00%> (-2.78%)
:arrow_down:

pkg/agent/controller/egress/egress_controller.go
75.36% <0.00%> (-2.20%)
:arrow_down:

pkg/agent/nodeportlocal/k8s/npl_controller.go
58.70% <0.00%> (-2.05%)
:arrow_down:

... and 123 more

I'm manually linking it so we will know the original PR has been backported to 1.6.
Cherry pick of https://github.com/antrea-io/antrea/pull/3510 on release-1.6.


UBIAI NLP annotation tools simulatio

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
How I can make buttons on pdf files
Describe the solution you'd like
A clear and concise description of what you want to happen.
FUNDS format data export?
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

Check this comment in the relative issue:
https://github.com/heartexlabs/label-studio/issues/2634#issuecomment-1251648670


Replace uses of llvm::integerPart to match llvm r299341.

LLVM r299341 removed the llvm::integerPart typedef and replaced it
with llvm::APInt::WordType. The integerPartWidth constant was replaced
by llvm::APInt::APINT_BITS_PER_WORD.

FYI @compnerd


Added Sync support

Support synchronous calls

Look good!
I will check tomorrow, if it works as expected, i will merge it and publish a new version of SDK.
THANKS!

Merged :D
I just upload a new version to bintray, it will be available in an hour or 2

AVAILABLE
compile 'com.github.alorma:github-sdk:1.0.1'


fix: Shellescape pane titles

Problem
Pane titles are currently not shell escaped, so multi word titles or titles containing shell characters break, i.e.
$ mux pane-names
usage: select-pane [-DdeLlMmRUZ] [-T title] [-t target-pane]
sh: 36: Syntax error: "(" unexpected

Session and window names are already escaped in this way.
name: pane-names (a test! | let's find out)
root: ~/dev/sandbox

windows:
  - editor (and others):
      layout: tiled
      panes:
        - vim and things:
          - vim
        - mostly just docker (maybe compose):
          - docker ps

Fixes #900

Is anything holding this up?


implement request timeout annotation

Motivation:
Provide a way to set a request timeout using annotation
Modifications:

Add @interface RequestTimeout
Add RequestTimeoutDecoratorFunction

Result:

Closes #4463
users can use RequestTimeout annotation to set request timeout to service methods

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.9 out of 10 committers have signed the CLA.:white_check_mark: j-min5u:white_check_mark: minwoox:white_check_mark: trustin:white_check_mark: jrhee17:white_check_mark: ikhoon:white_check_mark: davin111:white_check_mark: Dogacel:white_check_mark: ks-yim:white_check_mark: nao0811ta:x: armerianYou have signed the CLA already but the status is still pending? Let us recheck it.

I've made updates based on the comments and added a test. however, setRequestTimeout with TimeoutMode.SET_FROM_START doesn't seem to work as expected
`` `java
private void setTimeoutNanosFromStart(long timeoutNanos) {
checkArgument(timeoutNanos >= 0, "timeoutNanos: %s (expected: >= 0)", timeoutNanos);
if (timeoutNanos == 0) {
clearTimeout();
return;
}
if (isInitialized()) {
if (eventLoop.inEventLoop()) {
setTimeoutNanosFromStart0(timeoutNanos);
} else {
eventLoop.execute(() -> setTimeoutNanosFromStart0(timeoutNanos));
}
} else {
addPendingTimeoutNanos(timeoutNanos);
addPendingTask(() -> setTimeoutNanosFromStart0(timeoutNanos));
}
}
CancellationScheduler$setTimeoutNanosFromStart seems to do a saturated add with the default request timeout in `addPendingTimeoutNanos(timeoutNanos);` which results in the timeout being + 10000 (millisec) ed when annotating with @.RequestTimeout

I forget noting grpc-timeout header which clears or appends the original timeout value.
https://github.com/line/armeria/blob/49054ad7f22a7f2d64faaeb694674b0df09ad165/grpc/src/main/java/com/linecorp/armeria/server/grpc/FramedGrpcService.java#L228-L246
It would not make sense if the timeout is updated by the client header after @RequestTimeout is applied. Should we avoid taking the grpc-timeout header when @RequestTimeout is run? It is a bit tricky to know whether @RequestTimeout is applied though.
We may need a specialized logic only for gRPC that uses reflection to know if @RequestTimeout is annotated to a gRPC stub method.

CancellationScheduler$setTimeoutNanosFromStart seems to do a saturated add with the default request timeout in addPendingTimeoutNanos(timeoutNanos);

It sounds like a bug. We should use setPendingTimeoutNanos(timeoutNanos) instead.

which results in the timeout being + 10000 (millisec) ed when annotating with @.RequestTimeout 🤔

In fact, pendingTimeoutNanos is a temporary value to return a scheduled timeout before a request or response is being subscribed. IIRC, the value stored in pendingTimeoutNanos field is discarded and addPendingTask(() -> setTimeoutNanosFromStart0(timeoutNanos)) actually affects the final timeout. So could you possibly double check if it really causes an unexpected result?

I've looked in to more of armeria's requestTimeout implementation, and confirmed that service.serve() before requestCancellationScheduler.init() is called. scheduler initialization occurs on AggregatedHttpResponseHandler construction, which happens at the end of HttpServerHandler$handleRequest.
This is why when ServiceRequestContext$requestTimeoutMillis() is called inside the service, there is a possibility that the pendingTimeout of ServiceRequestContext is returned (because armeria uses hot datasources).
I've added two commits:

fix the bug where wrong pending timeout is set (656a866) -> this fixes the previously failing testRequestTimeoutSet
variable name change suggestion: because they shadow the class variable, and seemed to cause confusuion (9979739)
additional test for the CancellationScheduler initialization which fails for the current implementation

It would not make sense if the timeout is updated by the client header after @RequestTimeout is applied. Should we avoid taking the grpc-timeout header when @RequestTimeout is run? It is a bit tricky to know whether @RequestTimeout is applied though.
We may need a specialized logic only for gRPC that uses reflection to know if @RequestTimeout is annotated to a gRPC stub method.

as for this issue, it seems somehow wrong that the client could override the request timeout, causing the call to last longer than the time set on the server-side, not just for @.RequestTimeout, but for any timeout set by the server.
How about making request timeout change by grpc-timeout header only possible if the given timout value of the header is shorter than the requestTimeout set to the current context?

additional test for the CancellationScheduler initialization issue which fails with current implementation (53ed701)

@minwoox You're right, it's a testcase to show that there's an issue with the current request timeout implementation. How would you suggest to fix it?

in HttpServerHandler$handleRequest service.serve() is called before requestCancellationScheduler.init()
this seems to be the cause of the issue

it's a testcase to show that there's an issue with the current request timeout implementation.

Do you mean testCancellationSchedulerInit() test?
Could you explain more about the test?
I think I don't get the purpose of the test. 😓
The response.contentUtf8() is 1230 and you want to convert it to a Boolean value?

How about making request timeout change by grpc-timeout header only possible if the given timout value of the header is shorter than the requestTimeout set to the current context?

That is fair enough. 👍

@minwoox
Oops, I fixed the testcase as intended: prove that RequestCancellationScheduler is not initialized when service code is run

prove that RequestCancellationScheduler is not initialized when service code is run

Thanks! Now I got it. 😄
I don't exactly remember the reason, but calling init() method later was what we intended so that the HttpResponse from the service layer is correctly cancelled with the timeout.
Do you have any reason you want to change the current behavior?

How about making request timeout change by grpc-timeout header only possible

It makes sense. The contract of useClientTimeoutHeader() already mentioned that request timeout is only respected when useClientTimeoutHeader() is set to false. I was wondering why this option is an opt-out though. It would be worth noting in the Javadoc of @RequestTimeout that if the annotation is used for gRPC services, the timeout value is only valid when useClientTimeoutHeader is disabled.
https://github.com/line/armeria/blob/c888c86b288e85bb489b2938f2498095b2303e55/grpc/src/main/java/com/linecorp/armeria/server/grpc/GrpcServiceBuilder.java#L789-L796

if the given timout value of the header is shorter than the requestTimeout set to the current context?

For simplicity, I prefer to apply either grpc-timeout or @RequestTimeout. So I don't think it is a good idea to select a shorter value between grpc-timeout and @RequestTimeout.

@minwoox

Thanks! Now I got it. 😄
I don't exactly remember the reason, but calling init() method later was what we intended so that the HttpResponse from the service layer is correctly cancelled with the timeout.
Do you have any reason you want to change the current behavior?

Ah, I see. I though't that if the servicecode is run before timeout is initialized, the actual timeout applied could be longer than desired. But if that was intended, we could ignore it. I'll remove the failing testcase
@ikhoon

For simplicity, I prefer to apply either grpc-timeout or @RequestTimeout. So I don't think it is a good idea to select a shorter value between grpc-timeout and @RequestTimeout.

I'll revert changes in 71aeac2 and add the additional documentation to @RequestTimeout thanks!

Thaaaaanks again. 😄


Can't build it either on Docker or on local machine

Hi @josephschorr,
I tried to compile your program either by docker build or go build and I have the same error:
# github.com/coreos/registry-monitor
./monitor.go:443:18: undefined: dockerClient

Could you please fix it ?
Thanks in advance

@josephschorr Problem is gone since #11
Thanks a lot


Publish templates to azure samples browser

Let's get the templates published here:
https://docs.microsoft.com/en-us/samples/browse/
We need to add necessary yaml header to READMEs and the azure-samples publisher account to azure-dev

Since its a P2 and also, we have the templates gallery site https://aka.ms/awesome-azd/
Moving it to backlog for now.

This is a dupe of https://github.com/Azure/azure-dev/issues/425. @jongio which do you want to leave open?

two diff imo.
This one is "get them out there".   #425 is work with docs team to make azd samples standout in the browser with a blue check.

How can we onboard our samples to Learn?
Understanding you have a repo in Azure-Samples org, we need to ensure that the ingestion process is aware of your repo. To do this, we need to complete this onboarding form: https://forms.office.com/Pages/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbR17IcoC-CnxGgErwh2l4_cBUNENaVE5FUk1CQ0kwTlo0QkVZUVhCUThXRC4u
What is the process for external contribution on Learn Code Samples today?
As it stands, our external contribution is though GitHub. We do not offer another modality for community publishing on samples at this time. https://review.learn.microsoft.com/en-us/help/contribute/samples/concepts/basics?branch=main#how-can-customers-outside-microsoft-contribute

@rajeshkamal5050 This should probably go into 0.7 or 0.8 milestone.

Onboarding docs are available here: https://review.learn.microsoft.com/en-us/help/contribute/samples/process/onboarding?branch=main#supported-metadata-fields-for-readmemd

Do we have to make the title "ToDo" at all? I've been thinking that we might want to reframe the templates to be more about the scenario that they enable. For instance, with the ToDo templates, we are really looking at a web app written in C# using Azure SQL Database. Most users won't necessarily care about the ToDo part and will rip that source code out in favour of their own code. That said, I worry that by promoting the ToDo part too heavily, some users might not think that the template is relevant to them.
That said, could we do something like "Web application with a C# API and Azure SQL Database"? Or is that too generic?
cc: @austinauth for any additional thoughts here (because we talked about this extensively for awesome-azd)

A few thoughts:
Re: Calling out service names in the title/description
The official Microsoft sample site is fairly restrictive around how a user can discover new templates. You cannot filter by things like service or framework, and only two metadata tags are surfaced on the card itself. So while, "Web application with a C# API and Azure SQL Database" is slightly generic, I think its important to highlight the services in the title or the short description like you have.
Re: Including ToDo in the name
I'm still split on this. ToDo isn't really a use case, but I do think it helps affirm that this is a simple application you can conceptualize. As opposed to something more esoteric like some of the examples I've seen: Django SQL Driver or Collision. That being said, we should be good removing it from the title as long as its in the description.
Re: My initial thoughts on naming & use cases
These are technically tutorials, but I like how Docker consistently starts their tutorial name with a verb. IMO it makes everything seem more approachable.

Another interesting example is AWS's Solution Library. While its not easy to find a sample by a particular service, they do a pretty good job of enabling discovery by a particular use case or vertical.

@savannahostrowski @Austinauth
Thanks for the good callout. One thing to consider here is that by default, the sample metadata will use the first Markdown heading (H1) as the title, which we have "ToDo application" in the title. Our options are here to either override it, or maybe we should make the "scenario" apparent in the description?
To simplify things, here's all are templates below. We need to come up with specific wording for the Title (displayed in the top of the card), and 150-word Description (display in the body of the card):

Template
Title
Description

todo-csharp-cosmos-sql
ToDo Application with a C# API and Azure Cosmos DB SQL API

todo-csharp-sql
ToDo Application with a C# API and Azure SQL Database

todo-csharp-sql-swa-func
ToDo Application with a C# API and Azure SQL DB on Static Web Apps and Functions

todo-java-mongo
ToDo Application with a Java API and Azure Cosmos DB API for MongoDB on Azure App Service

todo-java-mongo-aca
ToDo Application with a Java API and Azure Cosmos DB API for MongoDB on Azure Container Apps

todo-nodejs-mongo
ToDo Application with a Node.js API and Azure Cosmos DB API for MongoDB on Azure App Service

todo-nodejs-mongo
ToDo Application with a Node.js API and Azure Cosmos DB API for MongoDB on Azure App Service (Terraform)

todo-nodejs-mongo-aca
ToDo Application with a Node.js API and Azure Cosmos DB API for MongoDB on Azure Container Apps

todo-nodejs-mongo-swa-func
ToDo Application with a Node.js API and Azure Cosmos DB API for MongoDB on Static Web Apps and Functions

todo-python-mongo
ToDo Application with a Python API and Azure Cosmos DB API for MongoDB on Azure App Service

todo-python-mongo
ToDo Application with a Python API and Azure Cosmos DB API for MongoDB on Azure App Service (Terraform)

todo-python-mongo-aca
ToDo Application with a Python API and Azure Cosmos DB API for MongoDB on Azure Container Apps

todo-python-mongo-swa-func
ToDo Application with a Python API and Azure Cosmos DB API for MongoDB on Static Web Apps and Functions

Card for reference:

@gkulin has offered to help get these titles and descriptions tee'd up!

We need to provide a less than 150-word description that shows up on each sample's card in the gallery.

@weikanglim Do you mean 150 characters? Based on the descriptions you suggested, I think that's what you meant. But as I am looking through the gallery, I am coming across quite a few descriptions that are more than 150 characters (but not by much, I think the longest I saw was 167). Where did you get that 150 limit from?

re: Wei's chart with the info we need to fill.
I went with Savannah's suggestion of not having "ToDo" in the title since that isn't really (most of the time) relevant to the user
One thing to note is that the descriptions are essentially the title but in more words.
Wei had the idea of a standard description that ties into azd which I think might work well. So, the standard description I would suggest:

A complete ToDo list web app using Azure Developer CLI (azd) to build, deploy, and monitor

Template
Title
Description

todo-csharp-cosmos-sql
Web Application with a C# API and Azure Cosmos DB SQL API
A complete ToDo app with C# API and Azure Cosmos DB SQL API for storage. Uses Azure Developer CLI (azd) to build, deploy, and monitor

todo-csharp-sql
Web Application with a C# API and Azure SQL Database
A complete ToDo app with C# API and Azure SQL database for storage. Uses Azure Developer CLI (azd) to build, deploy, and monitor

todo-csharp-sql-swa-func
Web Application with a C# API and Azure SQL DB on Static Web Apps and Functions
A complete ToDo app with C# isolated Azure Functions for the API and Azure SQL database for storage. Uses Azure Developer CLI (azd) to build, deploy, and monitor

todo-java-mongo
Web Application with a Java API and Azure Cosmos DB API for MongoDB on Azure App Service
A complete ToDo app with Java API and Azure Cosmos for MongoDB for storage. Uses Azure Developer CLI (azd) to build, deploy, and monitor

todo-java-mongo-aca
Web Application with a Java API and Azure Cosmos DB API for MongoDB on Azure Container Apps
A complete ToDo app on Azure Container Apps with Java API and Azure Cosmos API for MongoDB for storage. Uses Azure Developer CLI (azd) to build, deploy, and monitor

todo-nodejs-mongo
Web Application with a Node.js API and Azure Cosmos DB API for MongoDB on Azure App Service
A complete ToDo app on Azure App Service with Java API and Azure Cosmos API for MongoDB for storage. Uses Azure Developer CLI (azd) to build, deploy, and monitor

todo-nodejs-mongo-terraform
Web Application with a Node.js API and Azure Cosmos DB API for MongoDB on Azure App Service (Terraform)
A complete ToDo app on Azure App Service with Java API and Azure Cosmos API for MongoDB for storage. Uses Azure Developer CLI (azd) to build, deploy, and monitor

todo-nodejs-mongo-aca
Web Application with a Node.js API and Azure Cosmos DB API for MongoDB on Azure Container Apps
Complete ToDo app on Azure Container Apps with Node.js API and Azure Cosmos API for MongoDB for storage. Uses Azure Developer CLI (azd) to build, deploy, and monitor

todo-nodejs-mongo-swa-func
Web Application with a Node.js API and Azure Cosmos DB API for MongoDB on Static Web Apps and Functions
A complete ToDo app with Node.js API and Azure Cosmos API for MongoDB for storage. Uses Azure Developer CLI (azd) to build, deploy, and monitor

todo-python-mongo
Web Application with a Python API and Azure Cosmos DB API for MongoDB on Azure App Service
A complete ToDo app with Python FastAPI and Azure Cosmos API for MongoDB for storage. Uses Azure Developer CLI (azd) to build, deploy, and monitor

todo-python-mongo-terraform
Web Application with a Python API and Azure Cosmos DB API for MongoDB on Azure App Service (Terraform)
A complete ToDo app with Python FastAPI and Azure Cosmos API for MongoDB for storage. Uses Azure Developer CLI (azd) to build, deploy, and monitor

todo-python-mongo-aca
Web Application with a Python API and Azure Cosmos DB API for MongoDB on Azure Container Apps
A complete ToDo app with Python FastAPI and Azure Cosmos API for MongoDB for storage. Uses Azure Developer CLI (azd) to build, deploy, and monitor

todo-python-mongo-swa-func
Web Application with a Python API and Azure Cosmos DB API for MongoDB on Static Web Apps and Functions
A complete ToDo app with Python FastAPI and Azure Cosmos API for MongoDB for storage. Uses Azure Developer CLI (azd) to build, deploy, and monitor

@gkulin Thanks for looking through it.
I did mean 150 characters, not words 😆. The specification here mentioned no more than 150. On the cards, I do notice some truncation at ~160

Looks like we still need:

Merge templates from staging -> master
Submit the onboarding form for each template
Wait for them to publish and verify what it looks like on the review site

Submitted onboarding requests for each template
Should see the changes in staging tomorrow.

Completed as part of #425


Streamline some log outputs

this PR is NFC.
It modifies the log output to the console in some few places, to make it more consistent and symmetric. This makes the output not only better to read but also makes doing greeps, searches and such things easier and more effective. ;)
It also displays the msg id in some places for better identification of whats going on.
It replaces "known endpoints" with "known components", since this is what is displayed.
:):)

Please squash first 3 commits

Otherwise changes LGTM

I didn't know how to do the squash properly, so just created new PR #210
sorry for this
closing this one here


Add mimetype module

Added a module to centralize MimeType parsing

Also added a debug configuration for using the debugger to easily walkthrough examples.

Fixed failing doctests.

NOTE: Some of the method signatures were slightly out of date. Although no_run flag is added in the comment, while running cargo test on nightly, the docs would not compile. However the docs remain unchanged for the most part, as all the code added is just to satisfy cargo from compiling the code.

Not really, but it was quite useful to debug the examples and see what was going on.
Can .gitignore if it feels unnecessary.
( FWIW the configurations were auto added by vscode )


Error with getProductVariationById

Hi
I am working with Product Variations to get a list of all variations of a product and I get an error at
WooProductVariation variations = await wooCommerce.getProductVariationById(productId: 31);
The reason is, since the variation parameter is not passed, it was converted to null and the query becomes Query : products/31/variations/null
I managed to pass empty string like below and the query becomes valid Query : products/31/variations/
WooProductVariation variations = await wooCommerce.getProductVariationById(productId: 31, variationId: '');
I get the following error
Unhandled Exception: type 'List<dynamic>' is not a subtype of type 'Map<String, dynamic>'
E/flutter ( 5726): #0      WooCommerce.getProductVariationById (package:woocommerce/woocommerce.dart:537:53)
E/flutter ( 5726): <asynchronous suspension>
E/flutter ( 5726): #1      _MyHomePageState.getProducts (package:ecommerce/main.dart:66:56)
E/flutter ( 5726): <asynchronous suspension>
E/flutter ( 5726): #2      _MyHomePageState.initState (package:ecommerce/main.dart:76:5)
E/flutter ( 5726): #3      StatefulElement._firstBuild (package:flutter/src/widgets/framework.dart:4702:58)
E/flutter ( 5726): #4      ComponentElement.mount (package:flutter/src/widgets/framework.dart:4538:5)
E/flutter ( 5726): #5      Element.inflateWidget (package:flutter/src/widgets/framework.dart:3508:14)
E/flutter ( 5726): #6      Element.updateChild (package:flutter/src/widgets/framework.dart:3266:18)
E/flutter ( 5726): #7      SingleChildRenderObjectElement.mount (package:flutter/src/widgets/framework.dart:5892:14)
E/flutter ( 5726): #8      Element.inflateWidget (package:flutter/src/widgets/framework.dart:3508:14)
E/flutter ( 5726): #9      Element.updateChild (package:flutter/src/widgets/framework.dart:3266:18)
E/flutter ( 5726): #10     ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:4589:16)
E/flutter ( 5726): #11     Element.rebuild (package:flutter/src/widgets/framework.dart:4280:5)
E/flutter ( 5726): #12     ComponentElement._firstBuild (package:flutter/src/widgets/framework.dart:4543:5)
E/flutter ( 5726): #13     ComponentElement.mount (package:flutter/src/widgets/framework.dart:4538:5)
E/flutter ( 5726): #14     Element.inflateWidget (package:flutter/src/widgets/framework.dart:3508:14)
E/flutter ( 5726): #15     Element.updateChild (package:flutter/src/widgets/framework.dart:3266:18)
E/flutter ( 5726): #16     SingleChildRenderObjectElement.mount (package:flutter/src/widgets/framework.dart:5892:14)
E/flutter ( 5726): #17     Element.inflateWidget (package:flutter/src/widgets/framework.dart:3508:14)
E/flutter ( 5726): #18     Element.updateChild (package:flutter/src/widgets/framework.dart:3266:18)
E/flutter ( 5726): #19     SingleChildRenderObjectElement.mount (package:flutter/src/widgets/framework.dart:5892:14)
E/flutter ( 5726): #20     Element.inflateWidget (package:flutter/src/widgets/framework.dart:3508:14)
E/flutter ( 5726): #21     Element.updateChild (package:flutter/src/widgets/framework.dart:3266:18)
E/flutter ( 5726): #22     ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:4589:16)
E/flutter ( 5726): #23     StatefulElement.performRebuild (package:flutter/src/widgets/framework.dart:4737:11)
E/flutter ( 5726): #24     Element.rebuild (package:flutter/src/widgets/framework.dart:4280:5)
E/flutter ( 5726): #25     ComponentElement._firstBuild (package:flutter/src/widgets/framework.dart:4543:5)
E/flutter ( 5726): #26     StatefulElement._firstBuild (package:flutter/src/widgets/framework.dart:4728:11)
E/flutter ( 5726): #27     ComponentElement.mount (package:flutter/src/widgets/framework.dart:4538:5)
E/flutter ( 5726): #28     Element.inflateWidget (package:flutter/src/widgets/framework.dart:3508:14)
E/flutter ( 5726): #29     Element.updateChild (package:flutter/src/widgets/framework.dart:3266:18)
E/flutter ( 5726): #30     SingleChildRenderObjectElement.mount (package:flutter/src/widgets/framework.dart:5892:14)
E/flutter ( 5726): #31     Element.inflateWidget (package:flutter/src/widgets/framework.dart:3508:14)
E/flutter ( 5726): #32     Element.updateChild (package:flutter/src/widgets/framework.dart:3266:18)
E/flutter ( 5726): #33     SingleChildRenderObjectElement.mount (package:flutter/src/widgets/framework.dart:5892:14)
E/flutter ( 5726): #34     Element.inflateWidget (package:flutter/src/widgets/framework.dart:3508:14)
E/flutter ( 5726): #35     Element.updateChild (package:flutter/src/widgets/framework.dart:3266:18)
E/flutter ( 5726): #36     ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:4589:16)
E/flutter ( 5726): #37     StatefulElement.performRebuild (package:flutter/src/widgets/framework.dart:4737:11)
E/flutter ( 5726): #38     Element.rebuild (package:flutter/src/widgets/framework.dart:4280:5)
E/flutter ( 5726): #39     ComponentElement._firstBuild (package:flutter/src/widgets/framework.dart:4543:5)
E/flutter ( 5726): #40     StatefulElement._firstBuild (package:flutter/src/widgets/framework.dart:4728:11)
E/flutter ( 5726): #41     ComponentElement.mount (package:flutter/src/widgets/framework.dart:4538:5)
E/flutter ( 5726): #42     Element.inflateWidget (package:flutter/src/widgets/framework.dart:3508:14)
E/flutter ( 5726): #43     Element.updateChild (package:flutter/src/widgets/framework.dart:3266:18)
E/flutter ( 5726): #44     ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:4589:16)
E/flutter ( 5726): #45     Element.rebuil

Please handle the null exception and the error above.
Please help.

I have managed to solve this myself. Please see my pull request and add a new function.

Thanks. Just Seeing this as well. Your PR has been merged.


Integrate with tink

We should figure out how to use tink for easier key management and encryption functions.

We need more details about this would entail.  Closing for now.


Update law

Check this repository to see if it is still up to date

There is no official court form to cross check with, but there were no obvious issues when I went through the interview.


BUG: shapely.ops.substring issue with LineString Z in shapely 2.0.0

I used the shapely.ops.substring method in my project. There was no issue in shapely 1.8.4. Today I upgraded to shapely 2.0.0 and I'm getting an error from calling this:
shapely.ops.substring(trajectory, dist1, dist2)

where the trajectory is a LineString type (LINESTRING Z), it contains Z coordinates.
setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (14,) + inhomogeneous part

It looks like only X, and Y of the start_point and end_point is used in the vertex_list for constructing the LineString
I'm using:
Ubuntu 20.04
shapely 2.0.0

Would you be able to make a small reproducible example from the failure in your project? (or try to give some more details about the characteristics of this specific linestring)
Taking a small example based on the docs, this works for me using shapely 2.0.0:
from shapely.geometry import LineString
from shapely.ops import substring
ls = LineString((i, 0) for i in range(6))
# <LINESTRING (0 0, 1 0, 2 0, 3 0, 4 0, 5 0)>

substring(ls, start_dist=0.5, end_dist=3.5)
# <LINESTRING (0.5 0, 1 0, 2 0, 3 0, 3.5 0)>

Sure. I think just adding the z coordinate is enough.
from shapely.geometry import LineString
from shapely.ops import substring
ls = LineString((i, 0, 0) for i in range(6))

# this fails
substring(ls, start_dist=0.5, end_dist=3.5)

Ah, yes, you already mentioned you have Z coordinates. Thanks!
So the substring function doesn't correctly create start / stop coordinates including z coordinates, but just x and y. That means that for the above example, we try to create a new LineString from a list of mixed dimension coords:
ipdb> coords
[(0.5, 0.0), (1.0, 0.0, 0.0), (2.0, 0.0, 0.0), (3.0, 0.0, 0.0), (3.5, 0.0)]

and this now started to fail with shapely 2.0 (with the exact error depending on the numpy version). While with shapely 1.8 it took the dimensionality of the first coordinate pair, and just ignored the z values of the remainding coordintes:
>>> print(LineString([(0.5, 0.0), (1.0, 0.0, 0.0), (2.0, 0.0, 0.0), (3.0, 0.0, 0.0), (3.5, 0.0)]))
LINESTRING (0.5 0, 1 0, 2 0, 3 0, 3.5 0)

We should fix this in substring by properly including a z coordinate if the linestring is 3D.
And we should also improve the error message in the LineString(..) constructor to give an informative error about it (instead of the current confusing error from numpy). But the fact that it errors I would say is actually an improvement, since before it silently ignored values (but if we keep it, we should mention this in the release notes of 2.0 as a breaking change).

Also running into this.
Changing these lines (end_point.x, end_point.y) to this sort of expression tuple(*end_point.coords) fixes it for me.
Will submit a pull request in case this approach is agreeable.


update file windows-powershell-commands.data

Places to look for updating the list:

https://www.pdq.com/powershell/
https://github.com/MicrosoftDocs/PowerShell-Docs

I've only added the windows powershell commands here.
Maybe we need a new file like the unix-shell.data but for windows with the names from the lolbas-project?

This was done in https://github.com/coreruleset/coreruleset/pull/2669.