Fix files content endpoint

This PR addresses issues #421 and #450.
It fixes the client.files.content API endpoint. The current implementation requires parsing the response body to JSON. However, some responses (like files generated by the Assistant API) need to be handled as pure response body content, such as a PNG file.
All Submissions:

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same update/change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?

+1 since the Batches examples in the docs fail when retrieving the final content results for this same reason.

Thanks @willywg @mepatterson -- in my understanding Faraday should only try to handle as json if the response header is Content-Type: application/json or similar -- wonder if we could do something similar so that users don't need to do anything different? In your examples OpenAI sends application/octet-stream so I think this could work

Makes sense to me

i'm now facing the same issue #421 ,  this could do the trick following @alexrudall comment
def get(path:, parameters: nil)
  response = conn.get(uri(path: path), parameters) do |req|
    req.headers = headers
  end

  content_type = response.headers['content-type']
  content_type == 'application/octet-stream' ? response.body : parse_jsonl(response.body)
end

should i open a new PR?


Add :order_by opt to has_many

Encountered a problem today where I wanted to order a has_many association by a field other than its id. Let's say I have a User model that has_many :feedbacks, Feedback. I preload :feedbacks in user so I can display them on the user's profile.
Since the most relevant feedback is the latest one, I want to arrange all feedback to display the newest one first and the oldest one last. There doesn't seem like a way to do this on a preloaded association unless I manually order them myself after the query. Was thinking of adding an :order_by option to has_many/3 which accepts the same args as Query.order_by/3.
Is that okay?

If we go down this road we should make it more general, maybe each association should be a queryable or you should be able to specify how to query the association when preloading it.

That approach is definitely more flexible and gives the user more power over how to query the associations. :+1:

So this is tougher than I have originally thought. I have thought about first tackling this as:
has_many :public_comments, from(c in Comment, where: c.public == true, order_by: ...)

However, this has an issue in that from(c in Comment) will try to load the comment, so that is not a viable option. One alternative I have in mind is to allow the following:
has_many :public_comments, c in Comment, where: c.public == true, order_by: ...

has_many/3 is already a macro, so we would need to add some sugar in order to support where and order_by, but we are mostly fine. Furthermore, the option above would allow us to at least sanity check, because we truly can't support much more than where and order_by without going into all sorts of edge cases.
Thoughts?

@gjaldon I would like to tackle this one because it is very complex and there are still some moving parts I have not solved yet.

@josevalim sure, please go ahead. :)
Passing the queryable to has_many would have been the ideal since that way we could possibly pass functions that return a query like has_many :public_comments, ^Comment.scope. No idea how if it's possible though given the issue of the Comment model being loaded at compile-time.
If not possible, then the next option sounds great too!
Btw, would also be great to have preload/3 have a 'scope query' so we can customize how associations are queried on a per-query basis rather than per-model like above.

We can't have preloads because if you do assoc(p, :public_comments) in a query there is no way we can get preloads from that.

Sorry if I'm noob and this is a bad idea, but what about just making it easier for users to create their own has_many derivative in the schema section? For example, my current solution for this is the following:
In my model web/models/dock.ex is to straight-up use the Schema.association macro
schema "docks" do
...
association :many,
  :on_site_trucks,
  App.HasManyOnSite,
  foreign_key: :dock_id,
  queryable: App.Truck
...
end

and in web/scopes/has_many_on_site.ex I just sort of "inherit" Ecto's has_many association by defining my own :
defmodule App.HasManyOnSite do
...
  def assoc_query(refl, values) do
    from x in refl.queryable,
      where: x.is_on_site == true and is_nil(x.deleted_at),
      where: field(x, ^refl.assoc_key) in ^values
  end
...
end

And copy+pasting a ton of code from https://github.com/elixir-lang/ecto/blob/master/lib/ecto/association.ex#L240
I mean, I realize the copy+paste portion isn't ideal (o rly?), but what if we made it easier to copy+paste through a __using__ in Ecto.Association.Has? Lowly end users like me would be empowered by having the much coveted ability to customize our association queries, but at the same time, we would avoid having to fiddle too much within Ecto's internals.

@foxnewsnetwork this is a very good suggestion. The trouble in support those things though is not the assoc_query but the joins_query which needs to be expanded and analyzer by Ecto's query planner as in this example:
join: p in assoc(p, :comments)

That's the root of the issue. For example, if :comments above has a limit, how would that limit be applied when used as a join? we literally can't solve this, so we need to maintain associations small so they can be used properly throughout Ecto. If you need more than this then you should just build and extend a regular query.

I have been thinking more about this issue. One of the things I really liked on Rails associations is that conditions given to the association was also used when building it. For example, if I declared:
has_many :public_comments, Comment, conditions: [public: true]

When I did:
build(post, :public_comments)

The returned post would have public: true:
%Comment{public: true}

I would love to have this behaviour in Ecto. However, if we are willing to support this feature via the syntax I mentioned above:
has_many :public_comments, c in Comment, where: c.public == true, order_by: ...

We would need to traverse the where and extract those expressions. Which I am not a fan of. Right now I am thinking about supporting this syntax:
has_many :public_comments, Comment, where: [public: true], order_by: [:published_at]

However, we don't support it in queries (which we may or may not need to do too). Thoughts?
/cc @drewolson @jeregrine (pinging more folks because this one has been tough)

If we're talking about rails - the ability to pass procs to association instead of a clunky hash with strange attributes was a great improvement. The old behaviour was very limited in scope and usage, the new one is much better (I know this as I was migrating two apps from rails 3 to rails 4 when this change was made - it simplified things a lot).
My main problem with a keyword is that you can't use it for relations other then equality - <, >, etc, are all unusable. I'm still thinking that ability to pass a function name or a fun would be the best one. You could do something like:
has_many :current_entries, Entry, fn ->
  from e in Entry, where: e.entered_at >= ^beginning_of_day and is_nil(e.left_at)
end
has_many :current_visitors, Person, through: [:current_entries, :visitor]

Good points. Maybe we should allow an on_build option that sets a parameter on build?
has_many :public_comments, c in Comment, where: c.public == true, order_by: ..., on_build: [public: true]

Thoughts?

I like the function approach for setting a 'default scope' for the association since we could do more powerful queries. It would be a challenge though to build an association with default values from conditions like Rails does.
How 'bout we have a :defaults option for setting default values for the built association and use the fun for preloading the association? Something like:
has_many :public_comments, Comment, fn (query) ->
  from c in query, where: c.public == true
end, defaults: %{public: true}

Was about to post, before your :on_build opt suggestion popped up. :) But yeah, that sounds great! :on_build or :defaults is more explicit than what Rails provides.

I can't support anonymous functions because of the reasons I mentioned above. If you do a limit, how does that even behave when we do a join in that association? So I would rather whitelist the supported operations.
@gjaldon I like :defaults more than :on_build, let's go with it.

Ah, I see. Ok sounds good to me! :)

:+1: on defaults.
Could a solution exist where association queries exist but certain actions
don't apply like limit? And we just document or even log a warning when
used as a preload on a join?
On Tue, Jul 14, 2015 at 6:57 AM, Gabriel Jaldon<EMAIL_ADDRESS>wrote:

Ah, I see. Ok sounds good to me! :)
—
Reply to this email directly or view it on GitHub
https://github.com/elixir-lang/ecto/issues/659#issuecomment-121215194.

I personally feel like I'd steer clear of using a feature like this and rather just use functions on my models. The asymmetry of the API (a difference syntax for association conditions and queries) is worrying to me.
I do think :defaults is the best compromise, but again, I'd probably just use functions to accomplish the same (either fetching the association with a scope on a separate query or doing the join + scope).

@drewolson the last proposal would still have a symmetric API:
has_many :public_comments, c in Comment, where: c.public == true, order_by: ..., defaults: [public: true]

Would you still steer away from this?

I am with Drew, I don't super use build at all. Just set the post_id
directly :P
I like the scoped queries but currently in elixir I just use functions and
its working out okay
Comment.for_post(post) => from(c in Comments, where: c.post_id == ^post.id)
On Tue, Jul 14, 2015 at 7:57 AM, José Valim<EMAIL_ADDRESS>wrote:

@drewolson https://github.com/drewolson the last proposal would still
have a symmetric API:
has_many :public_comments, c in Comment, where: c.public == true, order_by: ..., defaults: [public: true]
Would you still steer away from this?
—
Reply to this email directly or view it on GitHub
https://github.com/elixir-lang/ecto/issues/659#issuecomment-121231476.

or I just use preload all over. :P
On Tue, Jul 14, 2015 at 8:02 AM, Jason S<EMAIL_ADDRESS>wrote:

I am with Drew, I don't super use build at all. Just set the post_id
directly :P
I like the scoped queries but currently in elixir I just use functions and
its working out okay
Comment.for_post(post) => from(c in Comments, where: c.post_id == ^post.id
)
On Tue, Jul 14, 2015 at 7:57 AM, José Valim<EMAIL_ADDRESS>wrote:

@drewolson https://github.com/drewolson the last proposal would still
have a symmetric API:
has_many :public_comments, c in Comment, where: c.public == true, order_by: ..., defaults: [public: true]
Would you still steer away from this?
—
Reply to this email directly or view it on GitHub
https://github.com/elixir-lang/ecto/issues/659#issuecomment-121231476.

@josevalim yeah, I understand that it's symmetric now in the proposal, but it feels like we kind of tacked it on to the "standard" query syntax for the purposes of symmetry. I'd probably still just use functions. That said, if I'm alone on this I'd happily see it as a feature in Ecto. I don't think it takes anything away from the current API, I'm just generally wary of increasing the surface area of the (currently lovely) API.

Ok, let's close this. We can add a guideline that associations are meant to work on data, with the help of the upcoming support for associations in changeset and it doesn't make sense to pollute your structs with complex associations if you are mostly worried about querying. Given the complexity of all the discussions in here, it does sound like the most sensible decision to make.

...add a guideline ...

Yes, so much this! From a lowly plebeian end-user point of view, it isn't that Ecto is missing features or that the has_many doesn't support scopes and defaults. It's more that there is no Ecto homepage with a reasonably comprehensive guide to the canonical use patterns of Ecto's existing features. The Simple example and the existing guide attached to the Phoenix framework are a great start, but they don't (yet) cover all the more "intermediate-to-advanced" patterns required in building a more ambitious app such as many-to-many polymorphism, association scopes, where to even put queries (should I be polluting my controller? My model? In a separate file?), etc.
But, as a new-to-Ecto rails refugee, a lot of the problems that I used to solve simply by following existing convention now must be solved by explicitly making design decisions. And this isn't good because, as end-users, my feeble brain-power is hilariously limited and I would prefer to focus on solving my business-logic problems rather than having to figure out framework implementation. In other words, it's not so much that I must have rails convention in Ecto, it's more that I would really like some sort of convention that I could follow like a mindless lemming.
So, TL;DR: if you good folks on core are already building a guide website for Ecto, great, I'd love to contribute. If not, I can scrap together something like Phoenix has.

That's very insightful @foxnewsnetwork. I am not sure how guidelines would work for Ecto but let's indeed try to get something running, maybe start a github repo with some patterns so we start sketching what makes sense.

Sounds good! Would be glad to contribute there too.
Btw, @josevalim how to preload comments for user and order them at the same time(just one visit to the DB)? Doesn't seem possible at the moment. It would be helpful for us to be able to customize how associations are queried. Will this be made possible with associations in changeset?

@gjaldon I think this will support it: https://github.com/elixir-lang/ecto/issues/784

@josevalim but doesn't Repo.preload/2 expect a fetched model so would take 2 trips to the db?
Repo.get(User, 1) |> Repo.preload(:comments)

@gjaldon it is actually better to make two trips in the majority of case. The issue with assoc(...) in queries is that if you have posts and comments, the amount of data returned from the database will be posts_size x comments_size because it uses a join. So we end-up sending more data and spending more time processing this data in Ecto.
With two queries, the result set is posts_size + comments_size.

Sorry if I'm being persistent here, just want to know what's the best way to go about this. In the context of a Phoenix show action, I load a User who has many comments. I want to be able to get just the active comments. I was initially thinking to do something like:
User |> Ecto.Query.preload(:comments) |> Repo.get(1)

Problem is I can't set how the comments are preloaded. In this case, I could use the queryable support in Repo.preload/2:
Repo.get(User, 1) |> Repo.preload(:comments, Comment.active_comments)

But this would result in 2 trips to the DB which may be negligible, but isn't the ideal.
Shouldn't Ecto make it easy for users to do the more ideal/performant query?

The first case is also doing two trips to the database too. Again, it is actually better to make two trips in the majority of cases, see my answer above. If you really care about doing one trip:
query = from u in User,
  join: c in assoc(c, :comments),
  where: c.active,
  preload: [comments: c]
Repo.get(query, 1)

I see. Thanks for the explanations and the patience. :) Just wondered how best to do preloading for the case I mentioned. It looks like Repo.preload/2 is the way to go.

What is status of this issue in Ecto 2? Has a syntax support for order_by in has_many been introduced?

I don't think it has but now we support custom queries when preloading which can be used to solve this issue.

I am back on a computer. Today you will be able to do:
Repo.preload(posts, comments: from(Comment, order_by: [desc: :inserted_at]))

And that's going to work. You can also specify it as part of a query:
def with_recent_comments(query) do
  from p in query,
    preload: [comments: from(Comment, order_by: [desc: :inserted_at]))]
end

@josevalim Thank you for the answer.
But how can I set a default order? For example, I have position column and want associated entities to be always sorted by position. Setting this sort order manually in each preload or introducing a new function (and therefore a new way of preload for each such a case) looks tedious and error-prone for such the common task.

Btw, I didn't get how make it work with N:M associations:
Repo.preload(category, category_properties:
  from(CategoryProperty, order_by: [asc: :position], preload: [:property])
)

fails to preload :property association.

Well, since preload(posts, :comments) isn't going to apply a default order, I guess the only reusable solution is to write a function to preload_comments and use that everywhere instead.

@nathany Correct! But it doesn't look like a reasonable solution, more like a workaround.

I like the idea to be able to use something like
has_many :public_comments, Comment, fn (query) ->
  from c in query, where: c.public == true
end, defaults: %{public: true}

It would be much more convenient and consistent.

The above would be quite nice, I think this would also allow for polymorphic associations.
For example :
has_many :comments, MyApp.Comment, foreign_key: :commentable_id,  fn (query) ->
  from c in query, where: c.commentable_type == "articles"
end

I think it's a shame that we can't get defaults: [public: true]. This is a handy feature.
In my case, which I don't think is unusual, I have a Membership schema that links Account and User. The Membership schema has a role column, staff or admin. An Account will always define its associations based on that column. So...
has_many staff_users, Membership, defaults: [role: :staff]
has_many admin_users, Membership, defaults: [role: :admin]

There's no need here for a generic has_many :users. So the lack of ability to define this condition seems to make the association rather redundant.

It would be great to have something like this. Is there any update on this with regards to Ecto 2? (Maybe it is already possible in some way).

I want to dynamically preload entries from different tables based on a column value.
Writing a quizzing application where Question can have multiple choices, stored in different tables. Preload should happen based on the question type column in Question table
Question -> single_choices
-> multiple_choices
-> FIB_choices

Since this issue gets bumped by search engines I need to leave this here:
I have more efficient way of @smeevil 's solution
has_many :comments, MyApp.Comment, foreign_key: :commentable_id, where: [commentable_type: "articles"]


1.11.0.beta error on {{input-group selection=model.foo content=model.foos}}

If no value is set,  there is no property so cleanProperty throws an error:
cleanProperty: Ember.computed('property', 'modelPath', function () {
    return this.get('property').replace(this.get('modelPath'), '');
    // Uncaught TypeError: Cannot read property 'replace' of undefined
}),

If both value and selection are passed, it doesn't error, but treats it as a 'value' select; properties such as optionValuePath and optionLabelPath are not passed on to the {{view 'select'}} in app/templates/form-inputs/select.hbs.

Quick-n-dirty workaround (using both value and selection):
{{input-group
  selection=model.foo
  value=model.foo
  content=model.foos
  optionValuePath="content.id"
  optionLabelPath="content.title"}}

// app/templates/form-inputs/select.hbs
{{#if selectionBinding}}
{{view 'select'
  id=inputId
  content=content
  name=name
  selection=selection
  prompt=prompt
  disabled=disabled
  required=required
  invalid=isInvalid
  optionValuePath=optionValuePath
  optionLabelPath=optionLabelPath
}}
{{else}}
{{view 'select'
  id=inputId
  content=content
  name=name
  value=value
  prompt=prompt
  disabled=disabled
  required=required
  invalid=isInvalid
  optionValuePath=optionValuePath
  optionLabelPath=optionLabelPath
}}
{{/if}}

Thanks for opening the issue. I'll try and get a release pushed to the beta branch soon.

Has this been addressed? I ask because even thought I use optionLabelPath and optionValuePath in my code, the property on the model is still saved as the entire object used as the content

{{input-group}} has been removed as of the 2.0.0 release because Ember EasyForm now has a 1.0.0.rc-1 release for form components.


Generics - Difference between instantiation (and change afterwards) and direct instantiation of Partial

TypeScript Version:  3.5.0-dev.20190417
Search Terms:
Generics, Partial
Code
interface StateResource {
    state: string;
}

function test1<T extends StateResource>() {
    const foo: Partial<T> = {};
    foo.state = "finished"; // Works fine

    // Type '{ state: "finished"; }' is not assignable to type 'Partial<T>'.
    const foo2: Partial<T> = { state: "finished" };
}

Expected behavior:
foo1 and foo2 should not compile.
Actual behavior:
{state: "finished"} is not assignable to Partial, but {} is assignable to Partial and can be changed to state: "finished" afterwards.
Playground Link:
http://www.typescriptlang.org/play/#src=
interface StateResource {
    state%3A string%3B
}

function test1<T extends StateResource>() {
    const foo%3A Partial<T> %3D {}%3B
    foo.state %3D "finished"%3B

    const foo2%3A Partial<T> %3D { state%3A "finished" }%3B
}
Related Comment:
https://github.com/Microsoft/TypeScript/issues/20204#issuecomment-346191513

Ideally this line would be an error
foo.state = "finished";

but we don't deeply propagate the fact that .state originated in a constraint - property access only sees the constraint type, and reports the type as string rather than "at least string". In general TS isn't sound against supertype alias writes, which this one is certainly an example of.


Dynamic kubelet config support

What feature would you like to be added?
What are use cases of the feature?

@kron4eg is this still in the pipeline?

@youssefazrak @kron4eg There are some discussions on deprecating this feature (https://github.com/kubernetes/enhancements/issues/281#issuecomment-692241654), so I don't think we should proceed until it's not clear is it going to be deprecated.

@xmudrii cheers, thanks for the link. In this case, I will proceed and close it.


Custom network with image augmentation layer

Hi all,
First off thanks for the hard work you have put in creating stable baselines 3, it's helped me a bunch.
I have a fairly simple suggestion that I think fits in the contrib repo. In my work I've been using a custom network with image augmentation (almost exclusively random translations) applied, this seems to help boost performance and stabilize training.
There's been a few fairly recent papers that have applied this effectively: CURL, RAD, DrQ.
I've been using Kornia as a simple drop in to apply augmentations before feeding into the feature extractor layers, so the network looks something like this.
'''
class ImageAugNatureCNN(BaseFeaturesExtractor):
def __init__(self, observation_space: gym.spaces.Box,
             features_dim: int = 512,
             apply_augmentation: bool = True,
             shift_range: List[float] = [0.0, 0.0],
             zoom_range: List[float]  = [1.0, 1.0] ):

    super(ImageAugNatureCNN, self).__init__(observation_space, features_dim)
    # We assume CxHxW images (channels first)
    # Re-ordering will be done by pre-preprocessing or wrapper
    assert is_image_space(observation_space), (
        "You should use NatureCNN "
        f"only with images not with {observation_space} "
        "(you are probably using `CnnPolicy` instead of `MlpPolicy`)"
    )

    self.apply_augmentation = apply_augmentation
    self.augmentation = nn.Sequential(K.RandomAffine(degrees=0,
                                                     translate=shift_range,
                                                     scale=zoom_range))

    n_input_channels = observation_space.shape[0]
    self.cnn = nn.Sequential(
        nn.Conv2d(n_input_channels, 32, kernel_size=8, stride=4, padding=0),
        nn.ReLU(),
        nn.Conv2d(32, 64, kernel_size=4, stride=2, padding=0),
        nn.ReLU(),
        nn.Conv2d(64, 64, kernel_size=3, stride=1, padding=0),
        nn.ReLU(),
        nn.Flatten(),
    )

    # Compute shape by doing one forward pass
    with th.no_grad():
        n_flatten = self.cnn(th.as_tensor(observation_space.sample()[None]).float()).shape[1]

    self.linear = nn.Sequential(nn.Linear(n_flatten, features_dim), nn.ReLU())

def forward(self, observations: th.Tensor) -> th.Tensor:

    if self.apply_augmentation:
        observations = self.augmentation(observations)

    # visualise_augmentation(observations)

    cnn_out = self.cnn(observations)
    return self.linear(cnn_out)

'''
Let me know if you think this would be a good fit or if there's any improvements you could think of.
Edit: My bad, hit submit a bit early...

Hello,
Can you elaborate a bit more, what do you want to implement exactly?
Where should it be included? Which algorithms do you plan to support?

@araffin
I believe this is not only specific algorithms, rather a set of wrappers (or tools) to do image augmentation.
I think this would be a nice fit, although I am not sure if the library you linked would work here (observations are numpy arrays up until the algorithm, and I'd rather have the augmentations in the wrappers).

I think this would be a nice fit, although I am not sure if the library you linked would work here (observations are numpy arrays up until the algorithm, and I'd rather have the augmentations in the wrappers). For completeness-sake implementations of CURL/RAD could be added so one can try the setup as described by the authors in the original paper, but I think general wrappers/tools would be more useful to people.

Yeah this is what I meant. The reason for applying the augmentation prior to the network instead of as a wrapper is to make best use of multiple pass-throughs of the data. e.g. instead of storing an augmented image and reusing that same augmentation, apply each random augmentation as the images are sampled from the buffer. I'm not sure on how much difference this would actually make in practice, but I think this way round is closer to whats done in the RAD and DRQ papers. I agree that there's probably a better way instead of relying on another library, this was just the simplest method I could think of. Perhaps there's a better place post sampling from buffer whilst still np arrays?

The reason for applying the augmentation prior to the network instead of as a wrapper is to make best use of multiple pass-throughs of the data. e.g. instead of storing an augmented image and reusing that same augmentation, apply each random augmentation as the images are sampled from the buffer.

What you are describing applies mostly to off-policy then and could be implemented at sampling time?
What you propose in the code example however applied to on-policy algorithms too (which is fine).
The reason I'm asking for more details is that I don't want to have something half-working half-useful, that will then not be used. If we don't have a concrete reference, it will be hard to validate the implementation.
And for image augmentation, I would rather go for imgaug if we use numpy array, otherwise kornia is better when handling pytorch tensors.

The reason I'm asking for more details is that I don't want to have something half-working half-useful, that will then not be used. If we don't have a concrete reference, it will be hard to validate the implementation.

The linked RAD and QrD are extensions of existing algorithms with augmentation so I think we have reference results (and likely also implementations available). I think these would be a nice addition to repo, especially if the augmentation tools (e.g. buffer that augments samples when reading from it) are something that can be used with other algorithms too.

What you are describing applies mostly to off-policy then and could be implemented at sampling time?

Yeah mostly applies to off policy, the RAD paper shows some generalization gains when using PPO though so I think it should be applicable to both.

The reason I'm asking for more details is that I don't want to have something half-working half-useful, that will then not be used. If we don't have a concrete reference, it will be hard to validate the implementation.

Makes sense, I think we could eliminate CURL as a reference as I believe RAD is a simplified follow up which has most of the performance gains. RAD and DRQ are concurrent work that practically do the same thing, I think DrQ has some extra features, there's some discussion between authors on the differences here

The linked RAD and QrD are extensions of existing algorithms with augmentation so I think we have reference results (and likely also implementations available).

Yep there are author implementations of both here and here.
I guess in more detail I'd propose an alternative replay buffer that can take augmentation params as arguments, that applies these when data is sampled from the buffer (using imgaug). Would start with off policy as maybe more care needs to be taken when doing this for on policy algos? And aim to recreate the results from the RAD paper (as the DrQ paper has additional algo changes, although both should probably be cited). I'm not sure I'll have the time (or the mujoco license) to run all of the experiments however.

And aim to recreate the results from the RAD paper (as the DrQ paper has additional algo changes, although both should probably be cited). I'm not sure I'll have the time (or the mujoco license) to run all of the experiments however.

This sounds good! Should be not too difficult to implement and not too messy. Implement them as wrappers and/or as a separate algorithm if you need to modify buffers.
I think you can focus on some of the clearest results in the paper to see if your implementation is correct (ProcGen/Coinrun is open and free). You can also use PyBullet via this trick, altho I can not say what the results should be like.


Add 24bestex.com to the list of merchants

What's the name of your business?
24bestex.com
What's the URL of the website?
https://24bestex.com/
Write a brief description of your business/service (max 50 characters)
24bestex - cryptocurrency exchanger
Enter the URL where a Monero mention can be found
https://24bestex.com/en/
How do you accept Monero? (choose only one option)

[ ] Monero integrations (github.com/monero-integrations)
[ ] Globee
[x] Native wallet
[ ] Other payment gateaway (Coinpayments, Morphtoken, etc. Please specify)

How would you categorize your business? (choose only one option)

[x] Exchange
[ ] Block Explorer
[ ] Payment Gateway
[ ] Web Hosting
[ ] Tool
[ ] Service
[ ] Goods
[ ] Entertainment

Comment

They list Monero, but we are not going to add this merchant with the description safe cryptocurrency exchanger, the word safe might give the feeling that is the getmonero.org website to consider it safe, but that's not the case. We can add 24bestex.com - cryptocurrency exchanger

They list Monero, but we are not going to add this merchant with the description safe cryptocurrency exchanger, the word safe might give the feeling that is the getmonero.org website to consider it safe, but that's not the case. We can add 24bestex.com - cryptocurrency exchanger

I agree with your statement

No reason to not add it as far as i can tell.


require test-upgrade to pass before releasing

https://cloudfoundry.slack.com/archives/C7A0K6NMU/p1531934002000205


Update multihash dependency and make Cid Copy

Summary of changes
Changes introduced in this pull request:

Updating due to security vulnerability. Was a decent amount of breaking changes but now that the alloc size is limited (and our max is 32 bytes afaik unless there is an identity hash that rust impl allows to be greater), Cid can now be clone

cc @vmx let me know if you think anything should be changed from this :)
Reference issue to close (if applicable)

Closes #837
Other information and links

You mean Cid can now be copy?


[Deliver] Release to beta testers

Hi there!
First, thanks for Fastlane, it's amazing and the huge amount of work you all put here it's outstanding. THANK YOU! ❤
Now, I was looking to a way of sending my IPAs™ to the App Store, submitting them to the review process AND sending them to the beta testers (internal and externals). And I've hit with:

It's hard to know which one to use (I mean: Pilot, Deliver or TestFlight?), because looks like Pilot it's a "mix" of Deliver and TestFlight, but missing some features. After some search, I suppose the ones we all must use are Deliver and TestFlight (right?). If that's the case, I can make a PR trying to specify that on the docs :)
Then, let's say I want to send the IPA to the App Store, submit it to the review process (with some custom submission_information parameters), and also send the same version to the beta testers. Is there some way of doing this, or do I have to upload the binary twice (first with TestFlight, then with Deliver)?

Thanks!
🍻

Pilot, Deliver or TestFlight
testflight is just an alias for pilot 😃
I think you can do this:
first, you can use testflight to upload and then distribute. Then you can use deliver(submit_for_review: true, ipa: nil) to submit to review. Specifically settingipa: nil because I see in the docs, that deliver can submit without uploading: (https://docs.fastlane.tools/actions/deliver/#ipa) if you don't pass in an ipa (I think the default is to find an ipa and upload that, so you have to set it to nil)

Ah! Good to know, I'll try that, thanks @taquitos!

Hi, I'm doing also this and it works pretty well.
But you may want to run deliver immediately after pilot to create the new version in iTunes Connect. Otherwise the build is there but there is no version created.
This is what I do on my release lane:
        pilot( # testflight
            groups: ["QA"]
        )
        deliver(
            skip_app_version_update: false, # Create the new version
            # Skip
            force: true,
            skip_screenshots: true,
            skip_metadata: true,
            skip_binary_upload: true
        )

@alexito4 Thanks! I suppose using skip_binary_upload instead of ipa: nil is the same?

@Lascorbe I haven't taken a look at the implementation but I can confirm that it doesn't upload the ipa 2 times.
And from the docs:

skip_binary_upload | Skip uploading an ipa or pkg to iTunes Connect

Should be fine 💯


Incorporate workloads from mlperf.org

https://github.com/mlperf/reference ... pick any reference and ensure Kubebench workflow can incorporate this

+1. We could use those individual workloads to verify the usability/flexibility of the kubebench workflow.

/priority p3

@ddutta @xyhuang
I am wondering if we could see the result of mlperf, they have a website to send submissions but I can not find any result now https://github.com/mlperf/submissions

@gaocegege I think their submissions are not open yet, should be open in the next few months.

Bumping this because we have v0.5 results from MLPerf out.
https://mlperf.org/results/

Resurrecting an old issue.
On Tue, Apr 9, 2019 at 5:10 PM Adhita Selvaraj<EMAIL_ADDRESS>wrote:

Bumping this because we have v0.5 results from MLPerf out.
https://mlperf.org/results/
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kubeflow/kubebench/issues/22#issuecomment-481485091,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA1h-oP0jLMRVLL5J3vJZySSUXV1_pUAks5vfSv_gaJpZM4Tx22Q
.

--
-Debo~


A native login option, not just oauth

https://auth0.com

doesn't that make it easy to make fake accounts? 🤔

Maybe, but it is still easier to use and to code.

I don't mind just having oauth but I do think we should give more options such as:

Login with Apple
Login with Google
Login with Facebook

no, this is extremely annoying, especially for bot devs. this is one of the most important issues right now

Discord OAuth is in progress (awaiting help from an elixir contributor). A seperate bot api will be added in the future, but e-mail authentication is a terrible idea, it makes it so easy for fake account spamming and overloading the server with fake requests etc.

Its all handled by Auth0 so u dont have to worry bout secuirty and overloading but faking accounts is a problem

Make a verified badge i guess

Just wanna pop in and say that there is also Userbase which is completely serverless. (Unless you want to self-host it that is)
It is also GDPR compliant.

Auth0 will cost money at a certain user limit, so maybe userbase is a good idea

@daantje1 Userbase's free plan is capped at 100 users, but the production plan is unlimited users, has one-click stripe integration if Ben wants to sell out and if you dont wanna pay or be more secure you can always self-host


[#2] Add subject and let! to the rbi methods

Fixes https://github.com/FooBarWidget/sorbet-rspec/issues/2

Thank you!


[BUG] Onboarding process cannot be completed

After clicking on accept notifications on last screen during the onboarding process home screen doesnt show up.

Duplicate of #122

close because of duplicate


[Java][recognizer-text-number] Apply linting format to .java files

Apply linting format to several .java files
linting-rules.xml:

Change property 'allowedAbbreviationLength' from 1 to 4
Set property 'VariableDeclarationUsageDistance' to 5
Delete deprecated value from linting-rules.xml (LeftCurlyBraces->'maxLineLength' property)

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: GasparAcevedoZainSouthworks sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


GDI+ exception when converting some tif files to jpeg

Description
When converting a specific tif file to jpg, using ImageProcessor, an GDI+ Exception is thrown if metadata is preserved.
Note that the conversion completes successfully if preserveExifData: false

Steps to Reproduce
Download file:
https://drive.google.com/file/d/1EHEAb_EOMyg7zzzaWnA0Q4FPhq7_YXZM/view?usp=sharing
Covert to jpeg
using (var imageFactory = new ImageFactory(preserveExifData: true))
{
     imageFactory.Load(inStream)
          .Format(new ImageProcessor.Imaging.Formats.JpegFormat())
          .Save(outStream);
}

System Configuration

ImageProcessor version: 2.9.0
.NET Framework version: 4.5.2

Also, It might be interesting that in this case, the order of the parameters in the querystring from imageprocessor.web will have an impact when using meta=false, to ignore metadata when requesting an image.
?meta=false&format=jpg -> This image converts
?format=jpg&meta=false -> This throws the gdi+ exception

Turns out there is a proprietary PropertyItem Tag 37724 in the image which is a Photoshop-specific TIFF Tag
https://www.adobe.com/devnet-apps/photoshop/fileformatashtml/#50577413_pgfId-1039502
I can filter out non standard EXIF items when saving jpegs. It doesn't seem to affect any other formats.

Thank you for your fast response to this issue.

Now worries, I had an hour to kill over lunch.
Fix is here.
https://www.nuget.org/packages/ImageProcessor/2.9.1


Please clarify buffer source configuration

The README mentions:
" Setup buffer configuration
autocmd FileType markdown lua require'cmp'.setup.buffer {
\   sources = {
\     { name = 'buffer' },
\   },
\ }

I guess this means something like "use the buffer source only for Markdown files", but it's not clear. Is that what the code is meant to do? And if we want to do this per-filetype configuration, should we leave out { name = 'buffer' } from the sources in the main call to setup?

Thank you.
Your opinion is correct. I will improve document.
I will update docs as using cmp-nvim-lua source o Lua filetype.

@amenonsen I've update the document to a more meaningful example.


TAURI with tauri-plugin-sql throws this error when invoking the plugin on the JAVASCRIPT side

ERROR in ./src/App.js
Module parse failed: The top-level-await experiment is not enabled (set experiments.topLevelAwait: true to enabled it)
File was processed with these loaders:

<EMAIL_ADDRESS>./node_modules/babel-loader/lib/index.js
./node_modules/source-map-loader/dist/cjs.js
You may need an additional loader to handle the result of these loaders.
Error: The top-level-await experiment is not enabled (set experiments.topLevelAwait: true to enabled it)
at /home/mokhtar/restopos/node_modules/webpack/lib/dependencies/HarmonyDetectionParserPlugin.js:54:11
at Hook.eval [as call] (eval at create (/home/mokhtar/restopos/node_modules/tapable/lib/HookCodeFactory.js:19:10), :7:16)
at Hook.CALL_DELEGATE [as _call] (/home/mokhtar/restopos/node_modules/tapable/lib/Hook.js:14:14)
at JavascriptParser.walkAwaitExpression (/home/mokhtar/restopos/node_modules/webpack/lib/javascript/JavascriptParser.js:2314:29)
at JavascriptParser.walkExpression (/home/mokhtar/restopos/node_modules/webpack/lib/javascript/JavascriptParser.js:2244:10)
at JavascriptParser.walkVariableDeclaration (/home/mokhtar/restopos/node_modules/webpack/lib/javascript/JavascriptParser.js:2098:33)
at JavascriptParser.walkStatement (/home/mokhtar/restopos/node_modules/webpack/lib/javascript/JavascriptParser.js:1592:10)
at JavascriptParser.walkStatements (/home/mokhtar/restopos/node_modules/webpack/lib/javascript/JavascriptParser.js:1453:9)
at JavascriptParser.parse (/home/mokhtar/restopos/node_modules/webpack/lib/javascript/JavascriptParser.js:3338:9)
at /home/mokhtar/restopos/node_modules/webpack/lib/NormalModule.js:1082:26
@ ./src/index.js 7:0-24 11:33-36

@emothek can you please provide a reproducible example?

Closing due to inactivity.
The solution is probably to either wrap everything in an IIFE to simulate a top-level async function for vannilajs or to switch to Promise.then instead or whatever your frontend stack offers, anyway not really specific to the plugin,


[FEATURE] Utility Delegates

What has been done?

Mutliple delegate features have been added to reduce the tediousness and bugs caused by rewriting code many times, and as a by-product make the code look a lot more tidy while keeping readability (depending of course on how you name stuff)
Added the functionality to use an AttributeKey<T> as a delegate for a Pawn extension property returning T. Example: var Player.protectItem by PROTECT_ITEM_ATTR
-- You can operator invoke the AttributeKey<T> with a T value to return an AttributeDelegate<T> with a default value that you passed. Example: var Player.protectItem by PROTECT_ITEM_ATTR(false)
Added the functionality to set a Varp as a T delegate for a Player extension property returning T.
-- IntVarp is included as a sample implementation of VarpDelegate. Example: var Player.questPoints by IntVarp(101)
Added the functionality to set a Varbit as a T delegate for a Player extension property returning T.
-- IntVarbit is included as a sample implementation of VarbitDelegate. Example: var Player.slayerPoints by IntVarbit(4068)
-- BoolVarbit is included as a sample implementation of VarbitDelegate. Example: var Player.gargoyleSlayer by BoolVarbit(4027)

Fixed VarbitDelegate.getVarbit using varbitId rather than varbitDef.varp resulting in either the wrong value being gotten or an exception being thrown.


Reduce tutorial mdoc noise

Current there is duplication / noise in the mdoc output from hardcoded result comments:

This PR removes these comments as mdoc computes and displays them for us.

I think without seeing the output, those examples don't really work. There are now just a lot of calls to .parse. What are they returning?
It isn't clear.
If we do this, we should add some asserts or something so people can understand if the parse succeeds or fails.

Hey Oscar, that's for taking a look at this :)
Just to be clear, the output will still appear on the site, mdoc executes the code and inserts the results with their types in the final output.
This PR is just removing the additional "manual" result line comments, which are not type checked by mdoc, and add extra noise to the site output.
Here's a Before and After screenshot:
Before

After

This is also nice because mdoc is a bit more clear in how it prints some output.
Compare mdoc's Right(value = ("", 't')) to the prior manual annotation Right((,t))

got it, thanks for clarifying.


How to generate corresponding instructions in a new Town? Such as Town13?

Hi!
I found that both './tools/data_parsing' and 'InstructionPlanner' have some predefined values, such as some coordinates corresponding to each Town. If these values are in a new Town, such as leaderboard2.0's Town13, how can these be generated?
Looking forward to your reply, thank you~
For example:

leaderboard/team_code/planner.py
'self.highway_mapping = {"Town04":[[-487.84,361.47,2.84,44.26],[-19.73,18.43,-279.10,278.82],[94.88,333.41,-360.95,-398.73],[-376.92,-93.35,400.16,440.26],[-517.71,-478.04,37.87,319.51]],
"Town05":[[-257.43,-217.99,-179.75,175.86],[184.14,218.66,-175.28,174.62],[-204.68,162.79,-217.05,-181.86],[-210.39,179.10,182.47,218.67]],
"Town06":[[-302.75,625.72,-8.10,-26.18],[-278.19,651.95,35.46,54.74],[-286.53,649.00,135.70,155.15],[-323.99,647.78,236.17,254.11],[656.33,673.07,12.02,228.79],[-372.63,-359.37,13.01,230.13]] }'

tools/data_parsing/turn_rules.py
class TurnFalse(Turn):
def init(self, direction):
super().init()
self.sample_range = 100
self.direction = direction
self.direction_index_mapping = {"left": 0, "right": 1}
self.direction_command_mapping = {"left": 2, "right": 1}
self.total_range = 128
self.Tjunction_loc_mapping = {"1": [[[74.31,2.50,2],[140.33,2.33,2],[320.09,2.32,2],[92.69,72.83,2],[173.38,55.50,2],[334.70,41.42,2],[92.29,147.24,2],[334.53,114.74,2],[92.81,215.65,2],[334.66,179.34,2],[107.98,326.93,2],[354.03,326.60,2]],[[107.74,-1.54,2],[173.88,-1.73,2],[352.97,-1.82,2],[88.03,41.60,2],[139.77,59.79,2],[339.00,73.74,2],[88.19,115.30,2],[339.00,148.54,2],[88.08,181.66,2],[338.96,214.46,2],[73.55,330.99,2],[319.83,331.13,2]]],
"2": [[[-3.23,205.09,2],[29.32,192.01,2],[117.29,192.16,2],[189.37,175.07,2],[45.80,253.94,2],[148.54,237.29,2],[189.51,225.80,2],[58.12,302.90,2]],[[-7.67,176.45,2],[58.12,187.84,2],[150.38,187.84,2],[193.96,204.03,2],[41.60,223.79,2],[119.50,241.48,2],[193.75,253.49,2],[28.17,307.19,2]]],
"3": [[[-18.78,-195.73,3],[149.05,-151.50,3],[84.34,-58.19,3],[147.97,-88.64,3],[103.13,-4.93,3],[172.18,-3.80,3],[233.99,-22.35,3],[232.47,37.33,3],[146.53,62.99,2],[22.64,195.33,3]],[[26.14,-205.40,3],[153.95,-116.42,2],[152.70,-58.14,2],[79.87,-88.97,2],[58.55,6.42,3],[129.03,7.83,3],[243.73,27.13,3],[190.25,58.98,2],[241.73,86.48,3],[-24.61,204.59,3]]],
"4": [[[179.01,-369.13,5],[242.51,-307.31,2],[81.06,-173.40,2],[148.32,-172.98,2],[388.35,-193.76,5],[272.62,-122.34,2],[386.55,-90,5]],[[272.72,-310.59,2],[44.36,-169.96,2],[116.76,-169.19,2],[239.88,-118.47,2]]],
"5": [[[-149.31,-136.65,3],[-101.78,145.94,3],[7.10,-189.40,5],[55.46,191.61,5],[33.34,162.25,3],[151.32,-18.97,2],[36.20,-129.62,3]],[[-103.21,-143.83,3],[-149.73,153.17,3],[30.16,-165.07,3],[26.36,125.75,3],[155.49,17.09,2]]],
"6": [[],[[665.92,66.59,5],[664.88,168.47,5]]],
"7": [[[-197.95,-147.58,2],[-85.62,-158.09,2],[-4.47,-171.80,2],[-5.13,-121.57,2],[-98.38,-50.71,2],[-5.83,-76.91,2],[-199.07,-22.44,2],[-99.01,15.57,2],[-199.03,64.57,2],[-138.96,48.95,2],[10.28,58.64,2],[-95.56,116.56,2]],[[-202.47,-174.34,2],[-202.13,-49.13,2],[-203.63,36.88,2],[-122.42,117.61,2],[-0.51,-145.93,2],[-1.19,-50.03,2]]],
"10": [[[64.47,64.40,3]],[[19.83,71.62,3]]]}
self.intersection_loc_mapping = {"1": [[90.30,0.51,25],[156.93,1.09,25],[336.86,1.39,25],[337.33,326.93,40],[90.95,327.01,40],[92.37,196.73,30],[91.87,131.36,30],[92.17,57.97,25],[156.05,55.61,25],[335.12,57.68,25],[335.78,130.58,30],[336.32,196.97,30]],
"2": [[43.31,304.10,30],[-5.34,190.45,30],[192.52,189.99,30],[190.68,239.30,30],[134.06,238.50,30],[43.77,238.49,30],[43.51,190.68,30],[133.09,189.44,30]],
"3": [[3.93,-199.79,35],[236.43,0.77,30],[237.27,61.02,30],[-1.39,196.76,35],[151.53,-132.98,30],[149.59,-72.75,30],[80.90,-74.44,30],[148.68,-5.98,30],[78.58,-5.19,30],[169.12,64.11,30],[-226.23,-2.30],[-223.15,103.26],[83.79,-257.12],[157.84,-256.18],[-146.60,-1.44],[-84.86,133.58],[-2.82,132.36],[-81.72,-137.82],[2.44,-135.59],[83.89,-135.75],[85.39,-199.39],[153.65,-198.61]],
"4": [[257.15,-308.29,25],[256.30,-122.12,25],[128.78,-172.50,30],[61.36,-174.60,25],[15.01,-172.33,25],[205.67,-364.69,30],[393.50,-171.28,25],[381.09,-67.54,30],[203.01,-309.33],[202.12,-247.58],[200.61,-171.29],[256.94,-248.01],[256.49,-170.93],[313.26,-248.37]],
"5": [[34.01,-182.82,20],[40.02,-147.67,20],[153.47,-0.52,25],[40.85,142.48,25],[30.24,198.96,30],[-126.12,-137.57,20],[-124.06,148.97,25],[-268.82,-1.19,30],[-189.88,-90.40,30],[-189.49,0.79,30],[-190.41,89.65,30],[-127.13,-89.45,30],[-126.58,1.19,30],[-125.56,89.59,30],[-49.85,-89.76,30],[-49.13,0.86,30],[-49.28,89.65,30],[31.55,-89.33,30],[29.53,0.28,30],[29.20,89.69,30],[101.55,-0.07,30]],
"6": [[662.70,41.96,40],[662.41,144.54,40],[-1.63,-17.53,40],[-1.84,49.77,40],[-0.50,141.78,40],[1.29,244.84,40]],
"7": [[-197.22,-161.53,40],[-1.85,-238.09,40],[67.08,-1.04,35],[67.25,60.09,35],[-109.01,113.97,35],[-198.61,49.24,25],[-198.65,-36.34,25],[-151.27,48.35,25],[-100.17,-34.76,15],[-100.46,-63.77,15],[-101.47,-96.25,10],[-85.31,-111.70,10],[-73.35,-159.14,30],[-3.43,-159.27,30],[-4.05,-107.83,15],[-4.45,-64.86,20],[-4.79,57.83,35],[-101.62,53.08],[-3.78,-1.48],[-150.54,-35.13]],
"10": [[-44.76,-55.94,30],[96.00,-21.14,30],[96.84,68.01,30],[-46.40,127.21,30],[-99.79,19.70,30],[-38.44,65.96,30],[41.59,66.94,20],[41.08,30.14,20],[-47.38,19.22]]}

Hi!
These specific coordinates are manually annotated by the authors :)
If you also need to obtain these values, I could give you some suggestions:

generate the corresponding .xodr map file by the Carla Python API
open the website: https://odrviewer.io/ , and upload the .xodr map file to this website. This website can help you visualize the Carla map
find the coordinates that you need. Some coordinate transforms may need to be applied. I remember that the y-axis value should multiply -1 when used in the Carla simulator.

Thank you for your detailed reply. I have studied and tried it based on it these days, and I think I understand it.
In addition, I found a problem. In the town map used in the currently provided project（Town01, Town10HD...）, in the ''geoReference'' field of the a file, the displacements of ''lat_0=0 +lon_0=0'' are all 0, but in the town13 used in Leaderboard2.0, ''lat_0=41.09374957104912 + lon_0=-81.59374804575259'', which will cause some problems:
In calculations involving "gps to location", such as those used in the following two files, I think the mean and scale used in different towns will change with lat_ref and lon_ref:
1. leaderboard/team_code/lmdriver_agent.py
2. leaderboard/team_code/planner.py
self.mean = np.array([0.0, 0.0])  # for carla 9.10
self.scale = np.array([111324.60662786,<PHONE_NUMBER>45])  # for carla 9.10
I referred to the code in leaderboard (leaderboard/leaderboard/utils/route_manipulation.py -> _location_to_gps), I learned from it and worked backwards, and got what I think is the correct conversion method:
`def _gps_to_location(lat_ref, lon_ref, lat, lon, z):
"""
参考自 leaderboard/leaderboard/utils/route_manipulation.py -> _location_to_gps
"""
EARTH_RADIUS_EQUA = 6378137.0   # pylint: disable=invalid-name
scale = math.cos(lat_ref * math.pi / 180.0)
mx = scale * lon_ref * math.pi * EARTH_RADIUS_EQUA / 180.0
my = scale * EARTH_RADIUS_EQUA * math.log(math.tan((90.0 + lat_ref) * math.pi / 360.0))

mx_ = lon / 180.0 * (math.pi * EARTH_RADIUS_EQUA * scale)
my_ = math.log(math.tan((lat + 90.0) * math.pi / 360.0)) * (EARTH_RADIUS_EQUA * scale)
location_z = z

location_x = mx_ - mx
location_y = -my_ + my

return {'x': location_x, 'y': location_y, 'z': location_z}`

Hi!
Sorry! I'm not familiar with the leaderboard 2.0. I remembered that the coordinate system is a little special(The GPS value was corrupted by noise. If the AV didn't move, the GPS of it was still changing?).
I think the best way is to check your config file by running it in the simulator.

Thanks！


There is an warning message

Below is my code as your example, it ran without error but there is a warning message like image in attachment. I installed stanford-corenlp-3.8.0 from nuget. Could you let me know what issue here ?
`var jarRoot = @"......\StanfordNLP\stanford-english-corenlp-models";
        // Annotation pipeline configuration
        var props = new Properties();
        //props.setProperty("annotators", "tokenize,ssplit,pos,lemma,ner,parse,dcoref");
        props.setProperty("ner.useSUTime", "0");

        // We should change current directory, so StanfordCoreNLP could find all the model files automatically
        var curDir = Environment.CurrentDirectory;
        Directory.SetCurrentDirectory(jarRoot);
        var pipeline = new StanfordCoreNLP(props);
        Directory.SetCurrentDirectory(curDir);

        // Annotation
        var annotation = new Annotation(sentence);
        pipeline.annotate(annotation); `

Thanks,

This is known warnings - sorry. I do not know yet a way to fix or work around them.
I guess that the reason is that StanfordNLP group does not ship slf4j in zip packages, so we do not recompile them and .NET runtime could load classes at runtime.


refactor(data_classes): add from future import annotations

Issue number: #4958
Summary
Changes
Add from __future__ import annotations to data_classes package
User experience
Discussed in #4607
Checklist
If your change doesn't seem to apply, please leave them unchecked.

[X] Meet tenets criteria
[X] I have performed a self-review of this change
[X] Changes have been tested
[ ] Changes are documented
[X] PR title follows conventional commit semantics

Is this a breaking change?
RFC issue number:
Checklist:

[ ] Migration process documented
[ ] Implement warnings (if it can live side by side)

Acknowledgment
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

This is the last one left @ericbn!! I hope to review and merge this tomorrow morning 🙏


develop <- master

Merge PR to develop along with master branch push for article publication. Created by this action: https://github.com/falgon/roki-web-post/actions/runs/5421616197

Build log
Preview build with this PR (b65963079d27eb86726210998889d173fd7146e0)

more info:cloud: Storage Free space   8.16 GiB    54.42%:open_file_folder: 111.50 MiB  CI_WORK/GitHubActions/roki-web


[fix] add icns icon

fix #395

When I try npm run package:mac with this PR (directly from your branch), I get this fatal error:
  • electron-builder version=20.44.4
  • loaded configuration file=package.json ("build" field)
  • writing effective config file=release/builder-effective-config.yaml
  • rebuilding native production dependencies platform=darwin arch=x64
  • packaging       platform=darwin arch=x64 electron=3.1.7 appOutDir=release/mac
panic: runtime error: index out of range

goroutine 1 [running]:
github.com/develar/app-builder/pkg/icons.doConvertIcon(0xc000164380, 0x6, 0x8, 0xc0000c37e0, 0x2, 0x2, 0x7ffeefbfc007, 0x4, 0x7ffeefbfc072, 0x36, ...)
	/Volumes/data/Documents/app-builder/pkg/icons/icon-converter.go:223 +0xf19
github.com/develar/app-builder/pkg/icons.ConvertIcon(0xc0000b29c0, 0x0, 0x0, 0x0)
	/Volumes/data/Documents/app-builder/pkg/icons/icon-converter.go:56 +0xd6
github.com/develar/app-builder/pkg/icons.ConfigureCommand.func1(0xc000203710, 0x100b9a3, 0x17408a0)
	/Volumes/data/Documents/app-builder/pkg/icons/icon-converter.go:33 +0x6e
github.com/alecthomas/kingpin.(*actionMixin).applyActions(0xc000208b58, 0xc000203710, 0x0, 0x0)
	/Volumes/data/go/pkg/mod/github.com/alecthomas/kingpin@v2.2.6+incompatible/actions.go:28 +0x6d
github.com/alecthomas/kingpin.(*Application).applyActions(0xc000246000, 0xc000203710, 0x0, 0x0)
	/Volumes/data/go/pkg/mod/github.com/alecthomas/kingpin@v2.2.6+incompatible/app.go:557 +0xdc
github.com/alecthomas/kingpin.(*Application).execute(0xc000246000, 0xc000203710, 0xc00014b480, 0x1, 0x1, 0x0, 0x0, 0x0, 0x1)
	/Volumes/data/go/pkg/mod/github.com/alecthomas/kingpin@v2.2.6+incompatible/app.go:390 +0x8f
github.com/alecthomas/kingpin.(*Application).Parse(0xc000246000, 0xc0000200d0, 0xb, 0xb, 0xc000246000, 0xc00023ff88, 0x10076ff, 0xc000038478)
	/Volumes/data/go/pkg/mod/github.com/alecthomas/kingpin@v2.2.6+incompatible/app.go:222 +0x203
main.main()
	/Volumes/data/Documents/app-builder/main.go:78 +0x350
Error: /Volumes/500GB/Code/readium-desktop/node_modules/app-builder-bin/mac/app-builder exited with code 2
    at ChildProcess.childProcess.once.code (/Volumes/500GB/Code/readium-desktop/node_modules/builder-util/src/util.ts:244:14)
    at Object.onceWrapper (events.js:286:20)
    at ChildProcess.emit (events.js:198:13)
    at maybeClose (internal/child_process.js:982:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:259:5)
From previous event:
    at MacPackager.resolveIcon (/Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/src/platformPackager.ts:594:101)
    at /Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/src/platformPackager.ts:573:31
    at Generator.next (<anonymous>)
    at MacPackager.getOrConvertIcon (/Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/src/platformPackager.ts:572:54)
    at Lazy.MacPackager._iconPath (/Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/src/macPackager.ts:37:43)
    at Lazy.get value [as value] (/Volumes/500GB/Code/readium-desktop/node_modules/lazy-val/src/main.ts:18:25)
    at /Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/src/macPackager.ts:52:27
    at Generator.next (<anonymous>)
    at MacPackager.getIconPath (/Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/src/macPackager.ts:51:20)
    at /Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/src/macPackager.ts:275:29
From previous event:
    at MacPackager.getOrConvertIcon (/Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/src/platformPackager.ts:572:54)
    at Lazy.MacPackager._iconPath (/Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/src/macPackager.ts:37:43)
    at Lazy.get value [as value] (/Volumes/500GB/Code/readium-desktop/node_modules/lazy-val/src/main.ts:18:25)
    at /Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/src/macPackager.ts:52:27
    at Generator.next (<anonymous>)
From previous event:
    at MacPackager.getIconPath (/Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/src/macPackager.ts:51:20)
    at /Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/src/macPackager.ts:275:29
    at Generator.next (<anonymous>)
From previous event:
    at MacPackager.applyCommonInfo (/Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/src/macPackager.ts:268:60)
    at /Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/src/electron/electronMac.ts:77:18
From previous event:
    at _createMacApp (/Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/out/electron/electronMac.js:262:24)
    at createMacApp (/Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/out/electron/electronMac.js:120:24)
    at /Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/src/electron/ElectronFramework.ts:67:11
    at Generator.next (<anonymous>)
From previous event:
    at _beforeCopyExtraFiles (/Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/out/electron/ElectronFramework.js:152:32)
    at beforeCopyExtraFiles (/Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/out/electron/ElectronFramework.js:111:32)
    at ElectronFramework.beforeCopyExtraFiles (/Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/src/electron/ElectronFramework.ts:119:12)
    at /Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/src/platformPackager.ts:222:23
    at Generator.next (<anonymous>)
    at runCallback (timers.js:705:18)
    at tryOnImmediate (timers.js:676:5)
    at processImmediate (timers.js:658:5)
From previous event:
    at MacPackager.doPack (/Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/src/platformPackager.ts:167:165)
    at /Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/src/macPackager.ts:90:63
    at Generator.next (<anonymous>)
From previous event:
    at MacPackager.pack (/Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/src/macPackager.ts:82:95)
    at /Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/src/packager.ts:430:24
    at Generator.next (<anonymous>)
    at xfs.stat (/Volumes/500GB/Code/readium-desktop/node_modules/fs-extra-p/node_modules/fs-extra/lib/mkdirs/mkdirs.js:56:16)
    at callback (/Volumes/500GB/Code/readium-desktop/node_modules/graceful-fs/polyfills.js:289:20)
    at FSReqWrap.oncomplete (fs.js:154:5)
From previous event:
    at Packager.doBuild (/Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/src/packager.ts:396:24)
    at /Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/src/packager.ts:366:57
    at Generator.next (<anonymous>)
    at /Volumes/500GB/Code/readium-desktop/node_modules/graceful-fs/graceful-fs.js:111:16
    at /Volumes/500GB/Code/readium-desktop/node_modules/graceful-fs/graceful-fs.js:45:10
    at FSReqWrap.args [as oncomplete] (fs.js:140:20)
From previous event:
    at Packager._build (/Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/src/packager.ts:335:133)
    at /Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/src/packager.ts:331:23
    at Generator.next (<anonymous>)
    at runCallback (timers.js:705:18)
    at tryOnImmediate (timers.js:676:5)
    at processImmediate (timers.js:658:5)
From previous event:
    at Packager.build (/Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/src/packager.ts:288:14)
    at build (/Volumes/500GB/Code/readium-desktop/node_modules/app-builder-lib/src/index.ts:59:28)
    at build (/Volumes/500GB/Code/readium-desktop/node_modules/electron-builder/src/builder.ts:228:10)
    at then (/Volumes/500GB/Code/readium-desktop/node_modules/electron-builder/src/cli/cli.ts:49:19)

The error disappears when I switch back to:
    "mac": {
      "target": "default",
      "icon": "dist/assets/icons/icon.icns"
    },

I looked at the ICNS icon you created for MacOS (using Preview.app), and I noticed that it is not multi-resolution.
When I created the icon for the r2-testapp-js, I made sure to include 512-256-128 all the way down to low resolution. See:
https://github.com/readium/r2-testapp-js/blob/develop/resources/icon.icns
If I remember correctly I used one of the tools recommended on this documentation page: https://www.electron.build/icons


Consider using absolute package versions in package.json instead of carets

Short description of the problem:
Today, angular 2.1 was released.
The current dependencies list in package.json that is generated from ionic start looks like this:
"dependencies": {
  "@angular/common": "^2.0.0",
  "@angular/compiler": "^2.0.0",
  "@angular/compiler-cli": "^0.6.2",
  "@angular/core": "^2.0.0",
  "@angular/forms": "^2.0.0",
  "@angular/http": "^2.0.0",
  "@angular/platform-browser": "^2.0.0",
  "@angular/platform-browser-dynamic": "^2.0.0",
  "@angular/platform-server": "^2.0.0",
  "ionic-angular": "^2.0.0-rc.0",
  "ionicons": "^3.0.0",
  "@ionic/storage": "^1.0.3",
  "ionic-native": "^2.0.3",
  "rxjs": "^5.0.0-beta.12",
  "zone.js": "^0.6.21"
}

This is currently causing the following issues when npm install completes:
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.

A quick check of npm ls --depth=0 yields the following:
├──<EMAIL_ADDRESS>├── UNMET PEER DEPENDENCY<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├── UNMET PEER DEPENDENCY<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├── UNMET PEER DEPENDENCY<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├── UNMET PEER DEPENDENCY<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>└──<EMAIL_ADDRESS>
npm ERR! peer dep missing<EMAIL_ADDRESS>required by<EMAIL_ADDRESS>npm ERR! peer dep missing<EMAIL_ADDRESS>required by<EMAIL_ADDRESS>npm ERR! peer dep missing<EMAIL_ADDRESS>required by<EMAIL_ADDRESS>npm ERR! peer dep missing<EMAIL_ADDRESS>required by<EMAIL_ADDRESS>npm ERR! peer dep missing<EMAIL_ADDRESS>required by<EMAIL_ADDRESS>npm ERR! peer dep missing<EMAIL_ADDRESS>required by<EMAIL_ADDRESS>npm ERR! peer dep missing<EMAIL_ADDRESS>required by<EMAIL_ADDRESS>npm ERR! peer dep missing<EMAIL_ADDRESS>required by<EMAIL_ADDRESS>npm ERR! peer dep missing<EMAIL_ADDRESS>required by<EMAIL_ADDRESS>npm ERR! peer dep missing<EMAIL_ADDRESS>required by<EMAIL_ADDRESS>npm ERR! peer dep missing<EMAIL_ADDRESS>required by<EMAIL_ADDRESS>
So, as you can see, the caret in our package.json has caused some updated packages to be installed, but these now create unsatisfied peer dependency errors. The reason for this, is that the angular packages which define their peer dependencies do so without a caret, i.e. they explicitly reference a particular version.
It could be argued that angular shouldn't be doing this, but to stop broken installs, I suggest that the ionic start command should create a package.json dependencies list without carets:
"dependencies": {
  "@angular/common": "2.0.0",
  "@angular/compiler": "2.0.0",
  "@angular/compiler-cli": "0.6.2",
  "@angular/core": "2.0.0",
  "@angular/forms": "2.0.0",
  "@angular/http": "2.0.0",
  "@angular/platform-browser": "2.0.0",
  "@angular/platform-browser-dynamic": "2.0.0",
  "@angular/platform-server": "2.0.0",
  "ionic-angular": "2.0.0-rc.0",
  "ionicons": "3.0.0",
  "@ionic/storage": "1.0.3",
  "ionic-native": "2.0.3",
  "rxjs": "5.0.0-beta.12",
  "zone.js": "0.6.21"
}

I'd argue that this should be the default behaviour for apps anyway - we should be locking down our dependencies unless we explicitly want to update them.

+1

This issue was moved to driftyco/ionic2-app-base#101


Is yarn supported?

Hi, not sure if this is a bug or a feature request. I'm also not a Frontend dev, so maybe I am missing something obvious, but is "yarn" supported as an ecosystem?
I get this error:

Starting: dependabot
    ==============================================================================
    Task         : Dependabot
    Description  : Automatically update dependencies and vulnerabilities in your code
    Version      : 1.26.627
    Author       : Tingle Software
    Help         : For help please visit https://github.com/tinglesoftware/dependabot-azure-devops
    ==============================================================================
    /usr/bin/docker run --rm -i -e DEPENDABOT_PACKAGE_MANAGER=yarn -e DEPENDABOT_OPEN_PULL_REQUESTS_LIMIT=1 -e DEPENDABOT_DIRECTORY=/ -e DEPENDABOT_TARGET_BRANCH=develop -e DEPENDABOT_IGNORE_CONDITIONS=[{"dependency-name":"*","update-types":["version-update:semver-major"]}] -e DEPENDABOT_COMMIT_MESSAGE_OPTIONS={"prefix":"🤖"} -e DEPENDABOT_FAIL_ON_EXCEPTION=true -e AZURE_ORGANIZATION=<redacted> -e AZURE_PROJECT=<redacted> -e AZURE_REPOSITORY=<redacted> -e AZURE_ACCESS_TOKEN=*** -e AZURE_MERGE_STRATEGY=squash ghcr.io/tinglesoftware/dependabot-updater-yarn:1.26 update_script
    Unable to find image 'ghcr.io/tinglesoftware/dependabot-updater-yarn:1.26' locally
    docker: Error response from daemon: Head "[https://ghcr.io/v2/tinglesoftware/dependabot-updater-yarn/manifests/1.26"](https://ghcr.io/v2/tinglesoftware/dependabot-updater-yarn/manifests/1.26%22): denied.
    See 'docker run --help'.
    ##[error]The process '/usr/bin/docker' failed with exit code 125
    Finishing: dependabot

Here is my pipeline definition:
trigger: none # Disable CI trigger
schedules:
  - cron: '0 4 * * 1-5' # daily at 4am UTC every weekday
    always: true
    branches:
      include:
        - develop
    batch: true
    displayName: Run dependabot every weekday
pool:
  vmImage: 'ubuntu-latest' # requires macos or ubuntu (windows is not supported)
steps:
  - task: dependabot@1
    inputs:
      setAutoComplete: false

And my dependabot config:
version: 2
updates:
  - package-ecosystem: "yarn"
    directory: "/"
    schedule:
      interval: "daily"
    target-branch: "develop"
    open-pull-requests-limit: 1
    commit-message:
      prefix: "🤖"
    ignore:
      - dependency-name: "*"
        update-types: [ "version-update:semver-major" ] # Ignore major version updates

Browsing the package list here, it seems that there is no package for "yarn" available: https://github.com/orgs/tinglesoftware/packages?repo_name=dependabot-azure-devops.
My suspicion is that I am doing something wrong. 45 minutes of googling and reading issues here in this repo didn't yield a result, so I thought I'd best open a new issue.

Thanks, that did it! 👍

Why yarn is not supported ?


Create case_statuses Table and add required_status to letter_types

Business Context:
As we roll out the app to more organizations, we understand that different orgs will have different case statuses.  Therefore we would like to store case statuses in a database table
As an organization adopting police-data-manager
I want to be able to customize the statuses through which a case goes
So that the status of a case in police-data-manager reflects how I would categorize it in my flow
In Scope

Migrate database to add case_statuses table and required_status field to letter_types table that is a foreign key to case_statuses
Add a seeder that populates the cases_statuses table with the current case statuses and links the letter types to the appropriate required status
Use the statuses from the case_statuses table to populate the CaseStatusStepper on the CaseDetails page
Update cases table so that the status field (or a new field that replaces it) is a foreign key to the case_statuses table (and make the necessary updates in the server-side code so that the API responses remain the same

Out of Scope

Determining when a letter can be generated based on the required_status field
Customizing statuses
Changing the wonky way we set next status on the front-end
Eliminating front-end constants that refer to statuses that will now be fungible

Open Questions

Acceptance Criteria

Scenario:
Given
When
Then

Scenario:
Given
When
Then

Mockups
Checklist (should be reviewed before grooming as the BA or Pair are writing the card—NOT reviewed for the first time at grooming)

Discuss the technical implementation of this card.
Discuss how this feature would be included in or affect the Audit Log.
In respect to security, what could go wrong? Brainstorm threats using STRIDE model, especially if this card introduces a new technical architecture or design pattern. (TIMEBOX: 7 mins) https://docs.google.com/document/d/18AFl1d3SRieI4d_8R3GyEazqQOycLNDHGN_vOYIHuiE/edit#heading=h.aio9zb5j63wl
Ensure any open questions have been resolved with stakeholders.
Determine if documentation should be referenced or created.
Ensure to include all necessary mockups to this card.
Discuss what success/failure user feedback messages would be present, if any occur (front end/back end).
Discuss why this feature would not be implemented behind a toggle / what would happen if this card was partially done while a release branch was being cut.
If this card is part of a sequence of cards, discuss if this card should be blocked until the testing of previous  cards in the sequence have been finished.
Have we added edge cases as ACs that we're considered from a QA perspective
Does this card overlap with any entities outlined in API Housekeeping?

Notes:

Future state of the database


[android] fix the LP32 armv7/i686 android build

This fixes the errors related to the st_mode structure being operated on with mode_t values, and the use of pw_gecos error.

cc @compnerd

@swift-ci please test

CC @jmschonfeld

@swift-ci please test

@swift-ci please test

@jmschonfeld I dropped the operator & overload, how does it look now?

@jmschonfeld could you take another look please 🙏 , we would like to get this in to make further build progress

Thanks!


Large performance increase for select, deselect, select_all checkbox …

…operations
Improvements with large number of sites for the select, deselect and select_all checkbox operations. For the application I am using jstree in, this reduced the load time of a ~17,000 node tree from 1 minute 40 seconds to ~11 seconds on my test machine when all data points are loaded from the start. Most of the gains are from using memoization.

Thank you for this pull request. Sorry it took me so many months to react - I just haven't gotten around to properly checking it out. I will merge this ASAP.
Once again - thank you.
Best regards,
Ivan

Sorry, I could not merge this solution - most of the improvements depended on two non-documented external variables. However I took the other parts, rewrote them to match jstree's style and compiled a changeset.
Once again - thank you.
Ivan


VDYP-120: Parsing the the input files used by FIPStart

Implementation of VDYP-163 and VDYP-165

Replace with #17


Add a detailed documentation on how to build the Arduino to LPT converter

The Arduino to LPT converter repository from FCBG is messy.
A complete tutorial on how to build one would be a nice addition to the documentation.

This has been moved to https://github.com/fcbg-hnp-meeg/byte-triggers


Update deployment script to use certificate auth for PnP

Suggestion
Update the deployment script to use certificate auth for connecting to PnP.
Create a separate script for creating the PnP AD app with the required permissions or documentation on how to do this.

Wouldn't it be better and even easier to default to use a Managed Identity?


Add html_filter_full_covered option to excoveralls

Why
By default the excoveralls set html_filter_full_covered as false so files with 100% coverage is displaying in the HTML report.
With the large codebase, this is hard to find the missed test file as there is many files

We could set the html_filter_full_covered to true so files with 100% coverage won't be displayed in the HTML report, the HTML report will include only the missing test files.
Who Benefits?
Developer

I did not know about the option html_filter_full_covered 👍

I did not know about the option html_filter_full_covered 👍

Me too, it's just available since May 12
https://github.com/parroty/excoveralls/commit/6482d8e0e4cc4a3d80a3cd7ab03cb28224fa8b8a

https://github.com/parroty/excoveralls/releases/tag/v0.14.5
Yeah nice addition. I normally have to look at this file because the file it's reported as lacking coverage doesn't fit in the terminal as it's too long so I still have no idea what the file is.


Whitelist bots

This PR completely[1] kicks off[2] the site any bot[3] that is not whitelisted.  The whitelisted bots are:

Googlebot
Bingbot

This is a step toward realizing the objective -- discussed in MO Technical meetings -- of improving website response by limiting site access to whitelisted robots and logged-in users.
The PR delivers Pivotal #181166642.
No manual test is suggested.[4]
The PR also has a very small amount of incidental RuboCopping.

[1] Absolutely no actions or pages allowed.
[2] Returns html status code 403 Forbidden.
[3] bot == user_agent that is recognized as a bot by the browser gem.
[4] There might be a way to manually test with curl, but I don't have the time to figure it out. If you're interested and can specify the user_agent in curl, then GET http://localhost:3000/observer/intro with user_agent Baidu.

Coverage increased (+0.0003%) to 92.95% when pulling c40ed88a962b16aa4775d9e5f42f30921599ae5a on jdc-whitelist-bots-181166642 into a05b741bc8b2dee788402a8480061078c3ee2ab7 on master.

re: class variables -- I admit I never remember the difference since I use
them so frequently.  I perfectly happy to use whatever rubocop, etc. prefer.
re: anonymous API use -- We should check if robots ever use the API.  I
don't think so.  If they do, let's revisit this.  If we were to change this
we might break an unknown number of API users out there.  That's a whole
"thing".  We'd have to figure out how to alert them of the up-coming
change, but without an API key we have no way to link the use to an account
or email address, so... I don't know, I guess we might have to institute
some sort of temporary thing where we check for anonymous API use and
somehow send them a message in the response... but of course they aren't
going to know to look for a message in the response... so... I have no idea
how to do it!  Site banner, I suppose?  As I said, it's a big thing.  I'd
rather continue to avoid it as long as possible! :)
On Mon, Feb 7, 2022 at 2:02 PM Joseph D. Cohen @.***>
wrote:

@.**** commented on this pull request.
In app/classes/robots.rb
https://github.com/MushroomObserver/mushroom-observer/pull/851#discussion_r800960422
:

 @@allowed_robot_actions["#{args[:controller]}/#{args[:action]}"]

 @allowed_robot_actions["#{args[:controller]}/#{args[:action]}"]

Both RuboCop and the official Ruby documenation
https://www.ruby-lang.org/en/documentation/faq/8/ suggest using class
instance variables instead of class variables in most cases. The point is
prevent surprise modification of the variable by subclasses.
Robots doesn't have any subclasses, so no big deal. But IMO it's best to
start with the default.
In app/controllers/application_controller.rb
https://github.com/MushroomObserver/mushroom-observer/pull/851#discussion_r800963468
:

@@ -262,7 +261,7 @@ def refresh_translations

Keep track of localization strings so users can edit them (sort of) in situ.
def track_translations

@language = Language.find_by_locale(I18n.locale)

@language = Language.find_by(locale: I18n.locale)

Incidental RuboCopping.
https://docs.rubocop.org/rubocop-rails/cops_rails.html#railsdynamicfindby
In test/controllers/observer_controller_test.rb
https://github.com/MushroomObserver/mushroom-observer/pull/851#discussion_r800965633
:

@@ -3901,14 +3901,24 @@ def test_normal_permissions
assert_equal(200, @response.status)
end

def test_robot_permissions
@request.user_agent = "Googlebot"
get(:intro)

def test_whitelisted_robot_permissions
@request.user_agent =

 "Mozilla/5.0 (compatible; Googlebot/2.1; " \

I grabbed the user agent strings from the interwebs for a little more
reality (and to help convince me that the browser bot was working.
In app/classes/robots.rb
https://github.com/MushroomObserver/mushroom-observer/pull/851#discussion_r800966893
:

class Robots
class << self

def allowed?(args)

 populate_allowed_robot_actions unless defined?(@@allowed_robot_actions)

 return true  if args[:controller].start_with?("api")

 return false if args[:ua].downcase.include?("yandex")

Is the robot authorized to be on the site?

def authorized?(user_agent)

 # Googlebot not followed by a hyphen excludes Googlebot-Image, etc.

 /Googlebot(?!-)|bingbot/.match?(user_agent)

end

def action_allowed?(args)

 populate_allowed_robot_actions unless<EMAIL_ADDRESS>

 return true if args[:controller].start_with?("api")

Allows anonymous API use.
Do we want to require a key?
—
Reply to this email directly, view it on GitHub
https://github.com/MushroomObserver/mushroom-observer/pull/851#pullrequestreview-875111715,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAYTNNIR6QDSDQZQ42LSCSDU2AJMHANCNFSM5NYIW4HQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you are subscribed to this thread.Message
ID: @.***
com>

Note that you will still need to let /graphql/user/login through without a
token/key, Nimmo.  For the old API this is not an issue because you create
the API key in person on-line first, not through the API.
On Mon, Feb 7, 2022 at 4:11 PM andrew nimmo @.***>
wrote:

@.**** commented on this pull request.
In app/classes/robots.rb
https://github.com/MushroomObserver/mushroom-observer/pull/851#discussion_r801061299
:

class Robots
class << self

def allowed?(args)

 populate_allowed_robot_actions unless defined?(@@allowed_robot_actions)

 return true  if args[:controller].start_with?("api")

 return false if args[:ua].downcase.include?("yandex")

Is the robot authorized to be on the site?

def authorized?(user_agent)

 # Googlebot not followed by a hyphen excludes Googlebot-Image, etc.

 /Googlebot(?!-)|bingbot/.match?(user_agent)

end

def action_allowed?(args)

 populate_allowed_robot_actions unless<EMAIL_ADDRESS>

 return true if args[:controller].start_with?("api")

This is a question that seems to relate to the next GraphQL PR, too.
Somehow my initial API test HttpRequestTest was getting identified as a
robot, and I could only get it to run by modifying line 14 above with
something like .start_with?("api", "graphql").
On the other hand, per consultation with Jason, we may want to set up the
graphql api to require an API key regardless - and the SvelteKit front end
would use its own API key.
I don't know if the last part is strictly necessary, for example if the
eventual front end is on another port of the same URL, as it is when I run
the front end locally.
—
Reply to this email directly, view it on GitHub
https://github.com/MushroomObserver/mushroom-observer/pull/851#discussion_r801061299,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAYTNNKJALEXZDCUICXGTYTU2AYO7ANCNFSM5NYIW4HQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you commented.Message ID:
@.***>

Message understood - although for the record there won't be any routes like that.
Everything just goes as JSON to /graphql/. The user and token are parsed from the http request header, then sent as part of a "context" object to the request parsing classes in /app/graphql/. Those classes determine whether the user in "context" has a right to do anything.
If we were to require an API key for graphQL requesta, it would need to be part of the http request header for every request. Since API keys are per-user, it obviously wouldn't map to the user in therequest - the "owner" of that api key would be SvelteKit.

On Feb 7, 2022, at 3:22 PM, Jason Hollinger @.***> wrote:
Note that you will still need to let /graphql/user/login through without a
token/key, Nimmo. For the old API this is not an issue because you create
the API key in person on-line first, not through the API.
On Mon, Feb 7, 2022 at 4:11 PM andrew nimmo @.***>
wrote:

@.**** commented on this pull request.
In app/classes/robots.rb
https://github.com/MushroomObserver/mushroom-observer/pull/851#discussion_r801061299
:

class Robots
class << self

def allowed?(args)
populate_allowed_robot_actions unless defined?(@@allowed_robot_actions)
return true if args[:controller].start_with?("api")
return false if args[:ua].downcase.include?("yandex")

Is the robot authorized to be on the site?

def authorized?(user_agent)

Googlebot not followed by a hyphen excludes Googlebot-Image, etc.

/Googlebot(?!-)|bingbot/.match?(user_agent)
end

def action_allowed?(args)
populate_allowed_robot_actions unless @.***_robot_actions)
return true if args[:controller].start_with?("api")

This is a question that seems to relate to the next GraphQL PR, too.
Somehow my initial API test HttpRequestTest was getting identified as a
robot, and I could only get it to run by modifying line 14 above with
something like .start_with?("api", "graphql").
On the other hand, per consultation with Jason, we may want to set up the
graphql api to require an API key regardless - and the SvelteKit front end
would use its own API key.
I don't know if the last part is strictly necessary, for example if the
eventual front end is on another port of the same URL, as it is when I run
the front end locally.
—
Reply to this email directly, view it on GitHub
https://github.com/MushroomObserver/mushroom-observer/pull/851#discussion_r801061299,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAYTNNKJALEXZDCUICXGTYTU2AYO7ANCNFSM5NYIW4HQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you commented.Message ID:
@.***>

—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you commented.

@pellaea I agree that we should leave the API stuff as is, at least for now. I don't know how much the API is used without a key. If that becomes a problem, we can deal with it in the future.

Thanks for the clarification, Nimmo.  I keep forgetting how different
GraphQL is.
re: SvelteKit having its own API key that it would always use -- Ah, right,
of course, that makes sense.  Then yes we can require an API key to use
GraphQL for all requests. ... Which is good, because as you point out,
there's no way to know what request is being made without parsing the
context!
On Mon, Feb 7, 2022 at 7:02 PM andrew nimmo @.***>
wrote:

Message understood - although for the record there won't be any routes
like that.
Everything just goes as JSON to /graphql/. The user and token are parsed
from the http request header, then sent as part of a "context" object to
the request parsing classes in /app/graphql/. Those classes determine
whether the user in "context" has a right to do anything.
If we were to require an API key for graphQL requesta, it would need to be
part of the http request header for every request. Since API keys are
per-user, it obviously wouldn't map to the user in therequest - the "owner"
of that api key would be SvelteKit.

On Feb 7, 2022, at 3:22 PM, Jason Hollinger @.***> wrote:
Note that you will still need to let /graphql/user/login through without
a
token/key, Nimmo. For the old API this is not an issue because you create
the API key in person on-line first, not through the API.
On Mon, Feb 7, 2022 at 4:11 PM andrew nimmo @.***>
wrote:

@.**** commented on this pull request.
In app/classes/robots.rb
<
https://github.com/MushroomObserver/mushroom-observer/pull/851#discussion_r801061299

:

class Robots
class << self

def allowed?(args)
populate_allowed_robot_actions unless
defined?(@@allowed_robot_actions)
return true if args[:controller].start_with?("api")
return false if args[:ua].downcase.include?("yandex")

Is the robot authorized to be on the site?

def authorized?(user_agent)

Googlebot not followed by a hyphen excludes Googlebot-Image, etc.

/Googlebot(?!-)|bingbot/.match?(user_agent)
end

def action_allowed?(args)
populate_allowed_robot_actions unless @.***_robot_actions)
return true if args[:controller].start_with?("api")

This is a question that seems to relate to the next GraphQL PR, too.
Somehow my initial API test HttpRequestTest was getting identified as a
robot, and I could only get it to run by modifying line 14 above with
something like .start_with?("api", "graphql").
On the other hand, per consultation with Jason, we may want to set up
the
graphql api to require an API key regardless - and the SvelteKit front
end
would use its own API key.
I don't know if the last part is strictly necessary, for example if the
eventual front end is on another port of the same URL, as it is when I
run
the front end locally.
—
Reply to this email directly, view it on GitHub
<
https://github.com/MushroomObserver/mushroom-observer/pull/851#discussion_r801061299
,
or unsubscribe
<
https://github.com/notifications/unsubscribe-auth/AAYTNNKJALEXZDCUICXGTYTU2AYO7ANCNFSM5NYIW4HQ

.
Triage notifications on the go with GitHub Mobile for iOS
<
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675

or Android
<
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub
.
You are receiving this because you commented.Message ID:
@.***>

—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you commented.

—
Reply to this email directly, view it on GitHub
https://github.com/MushroomObserver/mushroom-observer/pull/851#issuecomment-1032065715,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAYTNNI5VASAUZZSJAHKF6LU2BMSRANCNFSM5NYIW4HQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you commented.Message ID:
@.***>

@nimmolo:
We use the browser gem to detect robots.
The relevant code is in ApplicationController#kick_out_robots (line 180):
return true unless browser.bot?
The rules that browser uses to detect bots are here:
https://github.com/fnando/browser#bots.
If you have an empty user_agent, it thinks you are a bot.
If that's the problem, then -- as you suggest -- simply create a user agent
string that browser won't regard as a robot.
If that doesn't work, please let me know so we can figure it out.
On Mon, Feb 7, 2022 at 5:04 PM andrew nimmo @.***>
wrote:

@.**** commented on this pull request.
In app/classes/robots.rb
https://github.com/MushroomObserver/mushroom-observer/pull/851#discussion_r801185358
:

class Robots
class << self

def allowed?(args)

 populate_allowed_robot_actions unless defined?(@@allowed_robot_actions)

 return true  if args[:controller].start_with?("api")

 return false if args[:ua].downcase.include?("yandex")

Is the robot authorized to be on the site?

def authorized?(user_agent)

 # Googlebot not followed by a hyphen excludes Googlebot-Image, etc.

 /Googlebot(?!-)|bingbot/.match?(user_agent)

end

def action_allowed?(args)

 populate_allowed_robot_actions unless<EMAIL_ADDRESS>

 return true if args[:controller].start_with?("api")

Hi Joe,
I can only reproduce it on my PR branch, because it's a graphql controller
test that is blocked as a robot. I can show you how over there.
But - on the other hand maybe i can resolve it another way - like giving
the test request an arbitrary user agent string? Is that how we detecting
robots in the first place?
—
Reply to this email directly, view it on GitHub
https://github.com/MushroomObserver/mushroom-observer/pull/851#discussion_r801185358,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAALDFHWPUZHMGT67D56HCDU2BTYFANCNFSM5NYIW4HQ
.
You are receiving this because you authored the thread.Message ID:
@.***>

Thanks Joe! I'll do that, and if Rails will let me set and fake that header, I bet that clears it up.

@JoeCohen That worked.
I'd like to figure out a way to set that user agent by default for all the graphql tests, possibly using a Capybara "driver".

@nimmolo: Can you use setup to do that?
At the top of the testing class:
def setup
  @request.user_agent = "Whatever you're using as the UA string"
end

Merged and deployed to webserver

Thanks!
On Tue, Feb 8, 2022 at 10:39 PM Joseph D. Cohen @.***>
wrote:

Merged and deployed to webserver
—
Reply to this email directly, view it on GitHub
https://github.com/MushroomObserver/mushroom-observer/pull/851#issuecomment-1033311666,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAYTNNK6RQLKO4CBEM744TLU2HOWTANCNFSM5NYIW4HQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID:
@.***>


v-editable not working with non-blok fields

I don't know if it is even possible, but I'm trying to add the v-editable directive to non-blok fields. I can see the dashed outline on the component but whenever I click it it gives the following console error:

Is it even possible to use the v-editable directive without Bloks?

Hi, @christiancode14, thanks for reporting your issue!
Indeed, the v-editable is only meant to be used in the blok since it is used to connect the FE component instance with the corresponding block inside the SB visual editor. What are you trying to achieve by using it outside a component? Just in case I can guide you towards the solution.
Only one thing, next time, use the SDK bug report template to help us have a clear understanding of your issue, thanks in advance for that <3

Thanks, @christiancode14, for letting me know your use case. I will create a Feature Request internally since it is such a nice idea and could be helpful for other users! Are you a developer from a Partner? Just to let my team know from where is coming 💜

Thanks! That would be great!
Yes, I'm from CODE14


Observer data file timestamps should be in UTC, not the local timezone

Currently the Observer writes data files with names such as:

20210827T000724347-MicWaveIn.flac
20210827T001004378-Keypresses.protobuf

The timestamps in the filenames are in the local timezone, which adds to confusion. For simplicity and robustness, they should be changed to UTC.

@TeamGleasonDev @joncamp Please let me know what you think.


Addresses #2180 NTR mammary luminal epithelial cells

Addresses #2180 NTR mammary luminal epithelial cells
Context: these cell types were requested during the breast atlas jamboree. The names were decided then, and therefore there are no references for them yet (a manuscript is coming soon).

#gogoeditdiff

#gogoeditdiff

Thank you for the review. I have applied the changes requested. About the name the authors of the dbxrefs were present during the breast jam atlas and decided the names. There was a bioinformatic analysis that showed that cells found in the respective scRNA-seq and had different names were indeed the same cell type, and they proceeded to find the most adequate name. Since this is not published yet, my ORCID can be used as a dbxref.

#gogoeditdiff

"luminal hormone sensing cell" potentially sounds broader than the definition - I'm sure there are hormone sensing cells in the lumen's of many organs.  Consider making clear that this is breast/mammary gland specific from name.

"luminal hormone sensing cell" potentially sounds broader than the definition - I'm sure there are hormone sensing cells in the lumen's of many organs. Consider making clear that this is breast/mammary gland specific from name.

Ok, both labels can be broader than what is intended, therefore I will add "of mammary gland" in both.


gf: init at e0d6d2

Description
gf is a simple GDB frontend with minimal dependencies.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

It certainly does not fit contributing.md

@AndersonTorres and @viraptor thanks for the review and the suggested changes.

@AndersonTorres  thanks for your help!


Write a test that verifies the idempotency of processing the same transaction more than once

The situation when Electric receives the same transaction more than once via the replication stream can arise in the following scenario. When the database cluster shuts down, all of Electric's connections to the database will close and Electric will go into the reconnection-with-backoff loop. Once Postgres comes back up, Electric will be able to resume its logical replication stream from the same point at which it stopped streaming previously thanks to the persistent replication slot it had created.
If Electric has processed a transaction (by persisting it to the shape log) but hasn't acknowledged its WAL offset to Postgres because the connection was closed, the next time Postgres starts, Electric will read the already seen transaction from the logical stream once again.

I had a brief chat with @icehaunter about this. It looks like our storage will re-write over the key for the offse.

Let's make the test to validate that
Should we skip writing if the shape is ahead of the log from PG?


Suggestion: Text selection context menu optional

An option to disable "Unity Tools: Open Documentation for Selection" in the text selection context menu.

That's fair. I can actually add that right now

Great, the context menu gets crowded pretty fast by all the extensions and I find ctrl+' more convenient.

Fixed in 0.1.16! Let me know if that works for you.
(I also made it only appear in C# when even enabled)

All works fine, thanks!