Allow for overloading/aliasing of route path

Per this question: http://stackoverflow.com/questions/27004847/provide-alternate-international-spelling-for-defined-swagger-route/27013027#27013027
It would be great to allow aliasing of route paths for such uses as international spellings of words

I think that my change in #440 would also address this use case.

Nice! Looks like it.

Looking back at the question and the reply (SO), there's potentially a way to do that with 2.0 but it may not be a convenient way in some cases. Let's revisit this.
Parent: #560.

Propose rejecting this per #639

Not clear how setting allowed languages would solve the problem of internationalized spellings for routes? If you can elaborate, that'd be great.

It's not, but this feels a bit of a niche case that doesn't warrant special treatment. Since Path Item Object can be referenced, it should suffice for this case.
Voting to reject explicit support for this.

If we reject building in something for this, could we at least document the workaround here (with an example YAML)?

I've also wanted a solution for this, not so much for internationalized variants, but for normal API evolution. For example, a path is called x, but after feedback from your users, you (they) decide it should really be called y. To avoid a breaking change, it would be better to declare x is an alias for y (or y is an alias for x) and mark x as a deprecated operation. One could even hide or suppress the doc for x (in Swagger UI for example), but code generation or generated SDKs or other tools could route requests correctly (i.e. a generated handler/controller could return a 301 Moved Permanently, or simply call the new implementation).
Note that a simple $ref "solution" is less than ideal because one might want to define more than just reuse - marking an operation as deprecated or adding other x- extensions would be useful/necessary, but using $ref may prohibit that since a reference object may contain only the $ref property and no others.

Agreed David, I wonder if this belongs in a new thread as a proper feature request?

@grapefruit since this is already tagged at "OpenAPI.Next Proposal" and remains Open, this ticket should suffice. If think adding another won't help.

Sorry if I'm wrong, but I would like to use this feature also for format spec. In http://wiki.open311.org/GeoReport_v2/ you can choose between xml and json via suffix. I don't get how to put it in route (/services.{format} does not work), so I would like to make a route /services and a route /services.xml and a route /services.json, which is an alias for services?format=json

There must have been a different error in my .yaml, now it works like expected! Thanks very much for the very fast response andI wish all the programmers a happy new year!

I'm researching migrating our existing API to OpenAPI, and this would be a killer feature to help with migration.

Also looking for this feature!

@mattbishop The problem with this approach is that it is hard in automatic fashion to know what is content schema to expect in the response after the redirect. There should be some way to be able to say that Location will redirect always to some other route, so that client generators could know what response to expect and return. The problem is further complicated because generally those redirects are done automatically by the client (e.g., for fetch in a browser you cannot really usefully disable redirecting), so by calling /original-x, how to know what is the response schema?

The content schema of /new-x isn't a concern of /original-x.
All this is saying is the original-x has a different location that should be used instead.
This is the same behavior as HTTP now. The client will follow Location to /new-x and negotiate the content-type there.
See https://tools.ietf.org/html/rfc7538#section-3

But codegenerators do not behave like that, at least not that one I have tried. When I try to generate code for /original-x, there are only two options really possible for code to do: or do manual redirects and not automatic, returning simply the Location header for the /original-x call, leaving to the caller of generated code to then do another call to /new-x. Or, follow redirect automatically, but then code cannot really know to which endpoint it is going, or cannot have response be validated/parsed without some dynamic introspection.
So there should be more concrete way to say where is the redirect going, to which API endpoint (and not that just to some URI).

@mitar has a great point and maybe that's what the $ref idea earlier can solve for us.

I opened https://github.com/OAI/OpenAPI-Specification/issues/2512 for better handling of redirects (which can be a different thing than aliasing).

Just jumping back in here – I agree that an alias path is not the same thing as a redirect. A redirect implies the resource was originally there, and is now moved. Instead, we want a true alias, where both paths return the same value.
This is important for cases where one location might naturally spell a word differently than another, for example UK/Canada 'flavours' vs. 'flavors', or 'catalog' vs. 'catalogue'.
I'm not attached to one solution or the other, but I'll admit that the $ref solution feels mostly inline with the spirit of OpenAPI. There's just the operationId limitation mentioned in my very old SO post that would need to be addressed, and perhaps some documentation added to make this clear.
paths:
  /authorizations:
    get:
      description: Returns a list of authorizations
      responses:
        200:
          description: OK
  /authorisations:
    $ref: '#/paths/~1authorizations'

wondering if this feature was added.
i have a controller with 2 paths as below,
@PostMapping(path = {"/user/vehicle", "/user/car"})
public UserVehicle getUserVehicle(
But i want only "/user/vehicle" to be included in open api doc.

I've also wanted a solution for this, not so much for internationalized variants, but for normal API evolution. For example, a path is called x, but after feedback from your users, you (they) decide it should really be called y. To avoid a breaking change, it would be better to declare x is an alias for y (or y is an alias for x) and mark x as a deprecated operation. One could even hide or suppress the doc for x (in Swagger UI for example), but code generation or generated SDKs or other tools could route requests correctly (i.e. a generated handler/controller could return a 301 Moved Permanently, or simply call the new implementation).
Note that a simple $ref "solution" is less than ideal because one might want to define more than just reuse - marking an operation as deprecated or adding other x- extensions would be useful/necessary, but using $ref may prohibit that since a reference object may contain only the $ref property and no others.

I'm exactly in that situation right now. A feature request on an API that I'm working on requires me to rename a path.
So I need to first create an alias to that path, then tell the mobile app team to use that new path, and then remove the old path once the migration is completed.
But for the documentation part of this (that I do using Open API), I don't see any other solution rather than copy pasting the definition I've for the old path to create the new path.
Ideally, I'd want to rename my old path into the new path on my Open API JSON file, and then create a new entry with the old path make it refer to it as being an alias to the new path.
But yeah, looks like I can't do that :'(

I've also wanted a solution for this, not so much for internationalized variants, but for normal API evolution. For example, a path is called x, but after feedback from your users, you (they) decide it should really be called y. To avoid a breaking change, it would be better to declare x is an alias for y (or y is an alias for x) and mark x as a deprecated operation. One could even hide or suppress the doc for x (in Swagger UI for example), but code generation or generated SDKs or other tools could route requests correctly (i.e. a generated handler/controller could return a 301 Moved Permanently, or simply call the new implementation).
Note that a simple $ref "solution" is less than ideal because one might want to define more than just reuse - marking an operation as deprecated or adding other x- extensions would be useful/necessary, but using $ref may prohibit that since a reference object may contain only the $ref property and no others.

I'm exactly in that situation right now. A feature request on an API that I'm working on requires me to rename a path.
So I need to first create an alias to that path, then tell the mobile app team to use that new path, and then remove the old path once the migration is completed.
But for the documentation part of this (that I do using Open API), I don't see any other solution rather than copy pasting the definition I've for the old path to create the new path.
Ideally, I'd want, in my Open API JSON file, to rename my old path into the new path, and then create a new entry with the old path make it refer to it as being an alias to the new path.
But yeah, looks like I can't do that :'(

OpenAPI 3.1 allows defining reusable path item objects so one can define the full operation(s) in as reusable path items in #/components/pathItems, then reference them:
paths:
  /route/a:
    $ref: '#/components/paths/my-reusable-ops'
  /route/b:
    $ref: '#/components/paths/my-reusable-ops'
components:
  pathItems:
    my-reusable-ops:
      get:
        parameters: ....
        responses: ....
      put:
        parameters: ....
        requestBody: ....
        responses: ....

This is almost a solution for the narrower "exact alias" case - both paths will have identical operations, no copy/paste.
However, it falls short in at least two regards:

you cannot mark one operation as deprecated (or otherwise mark one as the "preferred" operation)
you cannot add an operationId to each operation that is used more than once because OpenAPI requires "The id MUST be unique among all operations described in the API." If there are more than one $ref to the defined path items, the operationId becomes duplicated. Boom. (Many tools and style guides require an operationId on each operation.)

A path item may also be a $ref to another path item. Since OpenAPI 3.1 allows a $ref reference object to have a summary and description, you can differentiate the two cases.
paths:
  /route/a:
    put:
      summary: Update an A thing (deprecated)
      description: Perform a full replacement of the representation of an A thing.

        Instead of this path, use the `get` operation on `/route/b`.
      $ref: '#/components/paths/my-reusable-ops/get'
  /route/b:
    put:
      summary: Update a B thing
      description: Perform a full replacement of the representation of a B thing.
        This replaces the `put` operation on `/route/a`.
      $ref: '#/components/paths/my-reusable-ops/put'
    get:
      summary: Fetch a B thing
      description: Return the representation of a B thing.
        This replaces the `get` operation on `/route/a`.
      $ref: '#/components/paths/my-reusable-ops/get'

However, you still cannot add an operationId or mark one as deprecated. Ideally one could do this:
paths:
  /route/a:
    put:
      summary: Update an A thing
      description: Perform a full replacement of the representation of an A thing.

        Instead of this path, use the `get` operation on `/route/b`.
      $ref: '#/components/paths/my-reusable-ops/get'
      deprecated: true

but reference objects cannot have such additional properties.

@DavidBiesack Thanks for the detailed answer !
For my usecase I ended up copy pasting because I want the deprecation to show on one of the 2 routes, but I'll remember that I can make exact aliases if I ever need to !

@DavidBiesack Thanks for the detailed answer !

So in recent JSON Schema (after 7, starting with draft-2019-09) they redefined $ref beyond its original standardization (which forbid extra properties next to it) and now they allow it. Maybe OpenAPI specification can follow the suit? See here: https://json-schema.org/understanding-json-schema/structuring.html#ref

@mitar OAS 3.1 already allows a $ref in an object to have siblings. OAS 3.1 no longer defines its reference objects as JSON Reference objects (as was done in OAS 3.0). OAS 3.1 also adopts JSON Schema 2020-12 which similarly has no such restriction - $ref is just a keyword.

Awesome. Then your comment:

but reference objects cannot have such additional properties.

Does not hold anymore?  What's doing that ~1?

got it, escape characters
https://swagger.io/docs/specification/using-ref/


Passed varibles not taking effect and breaking joystick functonality

I'm trying to add several axis, one of those is a joystick and it seems as though one of the axis is reversed. I would like to simply invert the axis but when I try and pass something like the below,
'''
joystick.add_input(
Axis(board.A3, invert=True)
)
'''
it seems to break the code and it refuses to work, I can add axis without any additional flags. I tried using the HOTAS example as a template and add the declarations as variables as described in the example but this also seems to break things. I have tried several different things but following the documentation has lead me to no joy. Any assistance is appreciated.

I think that, in trying to get it working, I was confused by the API call-out. From what I understand now, setting up the joystick axis (or buttons or hats) should look something like
Joystick.addinput(
Axis(board.A0, deadband=(some int) invert=True),
Ext.
)
I tried a bunch of other stuff to get it working like adding a virtual axis and inverting an analog input value, then using the joystick.axis[Axis.Z].source_value = int (joystick.axis[Axis.X].source_value × -1)
This example was pulled from your discussion thread from December of 2021 on multiple inputs having one output.
The failure mode for these things being wrong were basically just the hardware not being recognized by my computer as a joystick. I think there were bootout.txt error logs butbi don't have them at this point.
I did get things working on my end by using a different joystick with a y axis that wasn't inverted by default. Just a different peice of hardware but it would be nice to have the option to use my original joystick as it is nicer.


Update troubleshooting vagrant docs

Paired with @Rosa-Fox
We've moved around this file to try bring the more relevant content to
the top.
We've set a low review date of 2 months on this for a couple of reasons:

We're due to move offices so the Aviation House information will
become out of date
There's a lot of information that may be becoming redundant.

Your changes look good to me @chao-xian - I can't approve this now so either you can approve it or we get someone else to give it the ✅

Thanks @kevindew - I'll let you hit merge then 😄


Add blog post for CloudGripper talk

Added video link for post and brief description of what was discussed in the video.

Thank you Mike!


feat: expose controller and action in public request api

Adds the ability to access req.action (e.g 'index', 'show', etc.) and req.controller (e.g PostsController).

Current coverage is 85.86% (diff: 100%)

Merging #460 into master will increase coverage by 0.07%

@@             master       #460   diff @@
==========================================
  Files           170        170
  Lines          1816       1818     +2
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits           1558       1561     +3
+ Misses          258        257     -1
  Partials          0          0

Powered by Codecov. Last update e289ffc...ce969f7


Guild Leaderboard EXP

The guild leaderboard EXP doesn't update w/ the V9 experience amounts. It's been sitting at the pre-V9 experience.

Thanks for letting me know. Unfortunately, the API calls for guild data seem to be set to the pre v9 values, I have hotfixed this with web scraping for the top 10-15 guilds, however, due to the impending change to the web app, a permanent fix will likely be delayed until that update. Will look into

Guild Exp is now hidden to non-guild members, therefore these stats will not be public unless added with public API update


[#820] add checkstyle to streampipes-security-jwt

Purpose

Remarks

PR introduces (a) breaking change(s): <yes/no>
PR introduces (a) deprecation(s): <yes/no>

Close PR due to duplication #869


WebUI should escape slash characters when sent to server

Description
Several communities (CMS included) use slashes in the DIDs. When not escaped, these confuse REST APIs.
Steps to reproduce
Mayank and I reproduced this
Rucio WebUI Version
35.x
Additional Information
No response

Hello @ericvaandering!
Thank you for reporting this bug!
A pull request with an apparent fix has been attached and will be merged into master soon. I hope this resolves the issue!
Best regards,
Victoria


Mixins defined in an imported scss file aren't recognized in v1 beta

Problem
Apologies if I've missed something, literally started using v1 a few hours ago.
When I import a scss file that contains mixins, those mixins are not recognised when used.
Replication steps
Setup a structure like so:
└── src
   ├── assets
   │   └── styles
   │       ├── layout
   │       │   ├── theme.scss
   │       └── tools
   │       │   ├── mixins.scss

Inside mixins.scss declare a mixin:
@mixin redtext {
    color: red;
}

Inside theme.scss import mixins.scss and attempt to use it:
@import url('../tools/mixins.scss');
body{
    @include titletext;
}

Then run build:
yarn build
Output will show an error:
ERROR in ./assets/styles/layout/theme.scss
Module build failed: ModuleBuildError: Module build failed:
    @include redtext;
            ^
      No mixin named redtext

I know the import is working because when I add regular styles to it, they're included in the compiled file.

This is true for variables too, they are undefined outside of the file they are defined in.

Awesome thanks!
Maybe this could be included in the "transitioning to v1" docs that are being created.


Convenient design features

Hi!
Thanks for your project - it inspired me to create my :)
For starters, I would suggest you make the following function in a separate file, in order that they can be used in any script:
info () {
  printf "\n  [ \033[34m.\033[0m ] $1"
}

success () {
  printf "\n  [ \033[32mOK\033[0m ] $1"
}

fail () {
  printf "\n  [ \033[31mFAIL\033[0m ] $1"
  exit 1
}

question () {
  printf "\n  [ \033[33m?\033[0m ] $1 "
}

I think that I too can move a few functions in the individual libraries. The conclusion is that: We could try to create a joint project dedicated micro utilities written in bash :)
Let's do a modular bash! What do you think about this?

I'm back :fireworks:. I'll try to make your request finished as soon as possible. :smile:

:+1:

Please test colorize command now, it should work fine with printf.
Example:
printf "$(colorize "test" white bold blue)"


Task 157: Count vowels consonants

Vowels -> a,e,i,o,u
Consonants -> letters other than vowels

Thanks!
It's all good by me, except one issue: I think we can pick a more fine-grained category than "Answer Generation" (say, "Counting"). @swarooprm

yes, let's create a new category for counting @nrjvarshney


React write to chart with runTrackedMutations

Good afternoon,
I created a simple React plugin which retrieves entities from the information store via the geospatial search api. I can show those items on a OS map.
The next step I would archive is to add those entities also on the chart. I tried to implement the runTrackedMutations but without success, https://i2group.github.io/notebook-sdk/guide/tutorials/mutation-plugin.html.
Please can you help me with a working example of this functionality in React? Or maybe I should just create a 'normal' plugin, because I am not a React specialist:)

Hi Steven have you checked out the samples that go with the tutorials?
https://github.com/i2group/notebook-sdk/tree/main/samples
That has working code you can look at or run which might help.
Cheers

Hi Anthony,
I checked the examples, but there is no example how to add entities on the chart in React. In the React example they only show how to read selected entities from the chart. Entities are added to the chart only in the basic plugin in the entrypoint.js.
And I am struggling to convert that example into React.
for (const event of eventData) {
const startDateTime = mutations.valueFactory.createZonedDateTime(
event.dateTime,
getTimeZone(api.allTimeZones, event.timeZoneId),
false
);
      const eventRecord = mutations.addEntityRecord({
        itemType: eventEntityType,
        properties: {
          [eventTypePropertyType.id]: event.type,
          [startDateTimePropertyType.id]: startDateTime,
        },
      });

      const personSeenRecord = personLookup.get(event.wasSeen);

      if (personSeenRecord === undefined) {
        throw new Error(`Person matching ${event.wasSeen} is missing`);
      }

      mutations.addLinkRecord({
        itemType: observedLinkType,
        fromEnd: eventRecord,
        toEnd: personSeenRecord,
        linkDirection: 'with',
      });

      mutations.selection.add(eventRecord);
    }

HI @sdorresteijn88,
Where are you calling runTrackedMutation? I'd probably expect that to be called in response to a button click, in the React component?
e.g.
const handleClick = () => {
  api.runTrackedMutation((mutations) => {
    mutations.addEntityRecord(...);
  });
};

return <button onClick={handleClick} />

Hi Tim,
Thanks for your response. Finally I'll use it in the onCreate event of Leaflet Draw. But for now a simple working example where you can add a person to the canvas (https://github.com/i2group/notebook-sdk/blob/main/samples/mutation-plugin/entrypoint.js) would be great.
Attached the setup with just a button and the on click handler.
src.zip

Thanks for your suggestion. I'll add something to our backlog to enhance our React/Angular/Vue examples to show how mutation might be integrated with UI components.

That would be great! Can you also give an estimate in terms of planning? This is in connection with a demo that is planned soon, where I would like to show the power of the plugins.

I'm not able to give a time frame for when we'd be able to introduce new examples. The snippet I gave:

const handleClick = () => {
  api.runTrackedMutation((application, mutations) => {
    mutations.addEntityRecord(...);
  });
};

return <button onClick={handleClick} />

shows how you could call runTrackedMutation within a click handler. I'm not sure what else you require?

@sdorresteijn88 I've thrown together a quick example of using the mutation API in a react app, based on the existing SDK react example:
mutation-react-plugin.zip

Thanks a lot Tim, this is very helpful!


Decode nested json array

Hi I love Decodable but recently I encounter this situation with nested JSON.
JSON
{
    "cars": [
        {
            "name": "bmw",
            "passagers": [
                {
                    "name": "john",
                    "age": "23"
                },
                {
                    "name": "lucy",
                    "age": "23"
                }
            ]
        },
        {
            "name": "ford",
            "passagers": [
                {
                    "name": "błażej",
                    "age": "27"
                },
                {
                    "name": "marcy",
                    "age": "27"
                }
            ]
        }
    ]
}

Structs
struct Car:Decodable{
    var name:String
    var passagers:[Passager]
    static func decode(json:AnyObject) throws -> Passager {
        return try DirectionJson(
            name: json => "name",
            passagers: json => [Passager].decode //? error, so how?
        )
    }
}

struct Passager:Decodable{
    var name:String
    var age:Int
    static func decode(json:AnyObject) throws -> Passager {
        return try DirectionJson(
            name: json => "name",
            age: json => "age"
        )
    }
}

I was hoping that whole json would be decoded by single [Car].decode(json). But I'm not sure how I should handle [Passager] in Car.

It should be enough to write passagers: json => "passagers", and the type inference should figure it out.

That thank that works great. Again thank for you work.


Disable manage permissions when no rolebinding present for the default group

Closes: RHOAIENG-13091
Description
This PR aims to disable the Manage Permissions when model registry is in not ready state.

How Has This Been Tested?
Create a model registry
when Model registry is in progressing state, Manage permissions on the model registry table will be disabled.
Test Impact
Updated cypress test.
Request review criteria:

Self checklist (all need to be checked):

[x] The developer has manually tested the changes and verified that the changes work
[x] Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
[x] The developer has added tests or explained why testing cannot be added (unit or cypress tests for related changes)

If you have UI changes:

[x] Included any necessary screenshots or gifs if it was a UI change.
[ ] Included tags to the UX team if it was a UI/UX change.

After the PR is posted & before it merges:

[ ] The developer has tested their solution on a cluster by using the image produced by the PR to main

/retest


Argument Parsing with Docopt

Implemented parsing and render / update commands.
Moved formatter to separate module.

Fixes #2.


Having a script name random.py that imports numpy crashes the debugger

Title says it all, this is the window that I get (I cannot copy its content)

Having a script named anything that exists in the standard library is a great way to break Python. This is not unique to the debugger, and ultimately we can't reliably do anything better than fail.
You'll need to rename your script to something that is not an existing Python module, or anything that tries to use the random module will fail.


block queries on servicediscovery with consul

Fixes / New Feature #
block queries on servicediscovery with consul
issue #374
PollingInterval default is 10000 ms. May be someone should add this introduction in doc.
Proposed Changes
change query to block query could solve the problem of polling interval

@TomPallister DiskFileConfigurationRepositoryTests/should_return_file_configuration_if_environment_name_is_unavailable failed.


Warning: Quill Toolbar.getDOMNode(...) is deprecated. Please use ReactDOM.findDOMNode(instance) instead

__react 0.14.3 __
I got such error

The npm module has not been updated yet, you could install the latest with that fix via github:
npm i https://github.com/zenoamaro/react-quill

The latest doesn't have this fix either...

Looks like the repo does not have latest build in the dist folder, try to run npm run build

Here's the commit: https://github.com/zenoamaro/react-quill/commit/ff973605b78108fc22a7200ebea22b418ac562d2#diff-d24fe73020f46dde060f58786e987b61

Updates to the distributables are now being shipped with version v0.4.0, and published to NPM.


mp3blaster: Build with support for ogg vorbis

Motivation for this change
This package usually supports OGG but was built with only support for MP3 and WAV.
Adding libvorbis means that mp3blaster can now play OGG files.
Things done

[X] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[X] NixOS
[ ] macOS
[ ] other Linux distributions

[X] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[X] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[X] Tested execution of all binary files (usually in ./result/bin/)
[X] Fits CONTRIBUTING.md.

grahamcofborg build mp3blaster

Success on x86_64-linux (full log)
Partial log (click to expand)
shrinking /nix/store/bndf753lsknykhfqmpv0178sk1v3lmkw-mp3blaster-3.2.6/bin/splay
shrinking /nix/store/bndf753lsknykhfqmpv0178sk1v3lmkw-mp3blaster-3.2.6/bin/nmixer
shrinking /nix/store/bndf753lsknykhfqmpv0178sk1v3lmkw-mp3blaster-3.2.6/bin/mp3blaster
shrinking /nix/store/bndf753lsknykhfqmpv0178sk1v3lmkw-mp3blaster-3.2.6/bin/mp3tag
gzipping man pages under /nix/store/bndf753lsknykhfqmpv0178sk1v3lmkw-mp3blaster-3.2.6/share/man/
strip is /nix/store/97ahf6sg6mwnl98bksrjvbp4zb4yq44n-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/bndf753lsknykhfqmpv0178sk1v3lmkw-mp3blaster-3.2.6/bin
patching script interpreter paths in /nix/store/bndf753lsknykhfqmpv0178sk1v3lmkw-mp3blaster-3.2.6
checking for references to /tmp/nix-build-mp3blaster-3.2.6.drv-0 in /nix/store/bndf753lsknykhfqmpv0178sk1v3lmkw-mp3blaster-3.2.6...
/nix/store/bndf753lsknykhfqmpv0178sk1v3lmkw-mp3blaster-3.2.6

Success on aarch64-linux (full log)
Partial log (click to expand)
shrinking /nix/store/wky8yv55hbg7n5rwn78jdjc2862yk57l-mp3blaster-3.2.6/bin/splay
shrinking /nix/store/wky8yv55hbg7n5rwn78jdjc2862yk57l-mp3blaster-3.2.6/bin/mp3tag
shrinking /nix/store/wky8yv55hbg7n5rwn78jdjc2862yk57l-mp3blaster-3.2.6/bin/mp3blaster
shrinking /nix/store/wky8yv55hbg7n5rwn78jdjc2862yk57l-mp3blaster-3.2.6/bin/nmixer
gzipping man pages under /nix/store/wky8yv55hbg7n5rwn78jdjc2862yk57l-mp3blaster-3.2.6/share/man/
strip is /nix/store/ggjc1mfsicyq31xlg5d51hl9lc21wxnv-binutils-2.28.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/wky8yv55hbg7n5rwn78jdjc2862yk57l-mp3blaster-3.2.6/bin
patching script interpreter paths in /nix/store/wky8yv55hbg7n5rwn78jdjc2862yk57l-mp3blaster-3.2.6
checking for references to /build in /nix/store/wky8yv55hbg7n5rwn78jdjc2862yk57l-mp3blaster-3.2.6...
/nix/store/wky8yv55hbg7n5rwn78jdjc2862yk57l-mp3blaster-3.2.6


Header/Footer identify even/odd page

I need to know it the Header/Footer is on a even or odd page Number.
As far the Header/Footer doesn´t execute JS.
I have only placeholders for limited variables.
I need a suitable option that i can use different CSS on even/odd pages.
Maybe a automatic injected body class "even" / "odd" would help a lot.
Or are there any other possible solutions/workarounds?

Hey @ITspirit,
The header/footer feature is enabled by Chromium, I cannot add anything more than what’s currently available.
A workaround may be to create your own header/footer directly in the body and find a way to determine if a page is even using JS or else.


visable depricated use open

What does this PR do and why?
error was shown in my text editor notifing me that this prop is depricated, and to use open.
Screenshots or screen recordings
If applicable, provide screenshots or screen recordings to demonstrate the changes.
Acceptance checklist

[ ] I have evaluated the Approval Guidelines for this PR.
[ ] I have tested this PR in all supported browsers.
[ ] I have tested this PR in dark mode and light mode (if applicable).

Current dependencies on/for this PR:

main

PR #2047

PR #2048

PR #2049

PR #2089

PR #2090

PR #2091

PR #2092   👈

This comment was auto-generated by Graphite.


WIP - DRAFT - admin portal

Please ensure 1682 has been merged first - this PR is the UI changes only
Which issue this PR addresses:
#9777019 - List clusters in a table format
#9777031 - Display basic cluster info
What this PR does / why we need it:
PR introduces changes to the admin portal to display clusters in a list and basic information from the cluster document
Test plan for issue:
How did you test that this PR works?

Are there unit tests? Yes, portal API
Are there integration/e2e tests? No
If it is not possible to write automated tests, explain why and document how
to manually test and verify the feature.

Manually tested, partial phase 1 work complete. Feature is locked to specific region.

Is there any documentation that needs to be updated for this PR?
No documentation, tech debt to write in future ADO card

Can we split this please? It is going to be very difficult to review.

I think this is stale. I am going to close it


Windows support

Commit message contains summary of changes.
Failed checks are unrelated to Windows port - macOS tests are now enabled so errors came out.
This is not ready for merge yet, still few essential things are missing.

Error handling has been changed from snafu to thiserror so it would be good if you could rebase and change the error handling you added to thiserror. This is looking great so far btw.


Custom keybindings

Is your feature request related to a problem? Please describe.
I'd like to be able to port the keybindings I've customised, specifically to resize windows and to change their focus.
Describe the solution you'd like
A settings menu, configuration file (or instructions in the documentation if any of these already exist) on how to change the bindings.
Keybindings are on the QML files, and apparently you'd either have to read them dinamically somehow, or change the way the project is built (sorry but no experience with JS so might be saying something stupid).
Describe alternatives you've considered
Learning the new keybindings, which of course is not great.

System Settings > Shortcuts > KWin > Shortcuts named "Polonium"


Top banner no-links usecase

Top banner is currently showing irrelevant text when there is no hyperlink

Fixed in https://github.com/aemsites/hubblehomes-com/tree/issue-377 branch


Add integration tests for Appends

Add integration tests for Appends
Description
The native batch ingestion currently allows appending dynamically partitioned segments regardless of the original partitioning scheme. The new integration tests include the followings:

Append dynamically-partitioned segments to a dynamically-partitioned datasource
Append dynamically-partitioned segments to a hash-partitioned datasource. Compact hash-partitioned segments and dynamically-partitioned segments after append.
Append dynamically-partitioned segments to a range-partitioned datasource. Compact range-partitioned segments and dynamically-partitioned segments after append.

Some notes on the tests:

Note that the intervals of the appended segment overlap with those of original segments. This is done by using the same data for initial write and append.
Batch append itself is pretty well covered by unit tests (the unit tests mimic the distributed ingestion using multi-threaded ingestion testing framework (see AbstractParallelIndexSupervisorTaskTest). The integration tests focus on missing parts such as querying segments of mixed partitioning schemes.

Note: This PR also updates Integration Test's areSegmentsLoaded/getLoadStatus to use the new API to verify a datasource has the latest ingested data (https://github.com/apache/druid/pull/9965) instead of a Cluster wide loadstatus API. There are two benefits. First, we are now able to query status of a specific datasource, allowing multiple integration tests to be run concurrently. Second, we are able to take advantage of the new API's forceMetadataRefresh to make sure that we get latest published segment information (and compare/verify against the latest published segment information). This allows us to verify the segment count after the task finished. (Note that because of this, ITAutoCompactionTest.java test was also updated to verify segment count of only used and non-overshadowed segments).
This PR has:

[x] been self-reviewed.

[ ] using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)

[ ] added documentation for new or modified features or behaviors.
[ ] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
[ ] added or updated version, license, or notice information in licenses.yaml
[ ] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
[ ] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
[x] added integration tests.
[ ] been tested in a test Druid cluster.

LGTM 👍


pkg/openshift/rosa: new artifact directory field for create/delete clusters

Change
Previously you were able to provide a working directory field when creating/deleting clusters. This allowed the provider to write various files (e.g. aws vpc terraform files, cluster install/uninstall logs) to disk for later consumption
There may be certain files the provider may not wish to be archived and just temporarily written to disk. This commit introduces a new field to create/delete clusters options ArtifactDir. This directory is where artifacts can be written/saved off for later usage. While the WorkingDir is meant to be a temporary location and files are not archived. With this change WorkingDir will write files such as terraform files and ArtifactDir will store files such as install/uninstall logs. When this field is not provided, it defaults to the file systems temporary directory.

/cc @jbpratt

/lgtm


Add process_options hook for HTTPotion.Base

This hook is useful for API clients that need to use basic authentication. Such clients can inject their basic_auth option into the options list, like so:
defmodule Api do
  use HTTPotion.Base

  def process_options(options) do
    Dict.put(options, :basic_auth, { "user", "pass" })
  end
end

Thank you! Can't believe I missed that somehow :D


User's TX History (Activity) on Home Page

Story
As a user, I want to view my recent transaction history, so I can determine if my transactions are successful and detect unexpected activity (compromised account, accidental double transactions, etc).
Acceptance Criteria

[x] User can view most recent 5 transactions under "Activity" on home page

[x] Action Type (see Notes for list)
[x] Recipient's account name (if applicable)
[x] Amount of NEAR transferred (if applicable)
[ ] Timestamp

[x] User can select "View All" button under the Activity section, redirect to Explorer page (https://explorer.nearprotocol.com/accounts/accountName)
[x] Mobile ok
[x] Tested

Assets
Designs
Explorer Account Example
NOTES
Action List:

CreateAccount: "New account created:"
DeleteAccount: "Account deleted:"
DeployContract: "Contract deployed:"
FunctionCall: "Called method in contract: "
Transfer: "Transferred  N to <recipient":
Stake: "Staked  N"
AddKey : "Access key added for contract: " or "New key added for "
DeleteKey: "Key deleted"

Code from Explorer: https://github.com/nearprotocol/near-explorer/pull/102/files

Closed by #220


Screencast mode - settings for what to show

It would be nice to have control in settings to show non character (control keys) only vs show all. For example, showing CMD, CTRL, SHIFT, ESC, ENTER only vs showing all a, b, c, d .... z etc.

Would be great to have settings for the size and location as well. Maybe some custom css that can modify it to the users' liking?

Definitely . Without the possibility to filter characters, I wont use this very fun feature.

I'll work on this if no one else has!

What is stopping this from happening? The pull request was succesfully completed in February, and we are now in June.
I'd love for this to merge.

The PR was only started not completed.

Yeah I think @joaomoreno marked it as backlog. But as I wrote it I'd also love for it to merge :)

@mkenigs I sent you a code review back in March.

Sorry I missed that - don't see it, is it on the PR?

Added new setting screencastMode.onlyKeyboardShortcuts

Works nice.
Bloaaaat


/admin/ page issue

Hey there.
cool little app you have going on here. From a User perspective it may be wise to redirect to Login page from /admin/ when a user already exists (or adding a login button).
manually typing /admin/login will lose some users who don't know better ;)
Can't wait to see more!
-Ozzadar

Hi Ozzadar,
that (redirecting to /admin/login/) should happen. Is it not working for you?
 :)
Thanks

I'll take a look a little later today or tomorrow. Was setting it up on a coworker's box for her to try out and it wasn't redirecting properly.
For now I can tell you that the box runs Mac OS X (probably one of the latest versions) and after creating the user account, I had navigated back to /admin/ and it wasn't redirecting.
I'll get you more specific info after I look into it =)


Problem to install library

` pip install mrmr
Collecting mrmr
Using cached mrmr-0.9.2.tar.gz (18 kB)
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [10 lines of output]
Traceback (most recent call last):
File "", line 2, in
File "", line 34, in
File "C:\Users\anton\AppData\Local\Temp\pip-install-zngtmzxk\mrmr_a64358380c864590abc49eb48cf68abc\setup.py", line 9, in
from mrmr import version as mrmr_version
File "C:\Users\anton\AppData\Local\Temp\pip-install-zngtmzxk\mrmr_a64358380c864590abc49eb48cf68abc\lib\mrmr_init.py", line 5, in
from ._discretemrmr import *
File "C:\Users\anton\AppData\Local\Temp\pip-install-zngtmzxk\mrmr_a64358380c864590abc49eb48cf68abc\lib\mrmr_discretemrmr.py", line 31, in
from fakemp import farmout, farmworker
ModuleNotFoundError: No module named 'fakemp'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.`

@antoniotdelend The correct lib's name is mrmr_selection. So, if you run the command below, it should work:
pip install mrmr_selection


Some "xmlhttprequest/ajax" requests are treated as "other"

Steps to reproduce

Open filtering log
Go to https://vod.tvp.pl/video/korona-krolow,odc-184,41268622
Search in filtering log for ad.txt

Expected behavior

Request should be treated as xmlhttprequest.
Actual behavior

Request is treated as other.
Screenshot:

Your environment

AdGuard 7.0.2261.5797
Environment name and version: Chrome 72.0.3626.121
Operating system and version: Windows 10, v.1809, build 17763.348

That'd be not easy to solve btw, it's hard to detect XHR properly on the network level.


WASM - Non-trapping float-to-int conversion (3d)

Implement non-trapping float-to-int conversions
https://github.com/WebAssembly/design/pull/1089

@Cellule is this still planned for 1.8?


python310Packages.aioambient: 2023.10.1 -> 2023.12.0

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for python310Packages.aioambient is: Python library for the Ambient Weather API
meta.homepage for python310Packages.aioambient is: https://github.com/bachya/aioambient
meta.changelog for python310Packages.aioambient is: https://github.com/bachya/aioambient/releases/tag/2023.12.0
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 2023.12.0 with grep in /nix/store/gbp1qiw5nf0l2fs2srgk1vn9v30nyd4b-python3.10-aioambient-2023.12.0
found 2023.12.0 in filename of file in /nix/store/gbp1qiw5nf0l2fs2srgk1vn9v30nyd4b-python3.10-aioambient-2023.12.0

Rebuild report (if merged into master) (click to expand)

3 total rebuild path(s)

3 package rebuild(s)

First fifty rebuilds by attrpath
home-assistant-component-tests.ambient_station
python310Packages.aioambient
python311Packages.aioambient

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/gbp1qiw5nf0l2fs2srgk1vn9v30nyd4b-python3.10-aioambient-2023.12.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A python310Packages.aioambient https://github.com/r-ryantm/nixpkgs/archive/1ede548b027748858593e288636c89671e9de502.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/1ede548b027748858593e288636c89671e9de502#python310Packages.aioambient

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/gbp1qiw5nf0l2fs2srgk1vn9v30nyd4b-python3.10-aioambient-2023.12.0
ls -la /nix/store/gbp1qiw5nf0l2fs2srgk1vn9v30nyd4b-python3.10-aioambient-2023.12.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

2 packages built:

python310Packages.aioambient
python311Packages.aioambient

Maintainer pings
cc @fabaff for testing.

Result of nixpkgs-review pr 275143 run on x86_64-linux 1

4 packages built:

python310Packages.aioambient
python310Packages.aioambient.dist
python311Packages.aioambient
python311Packages.aioambient.dist


Add pygments qasm lexer and options for formatted output and saving to file

Summary
Adds a lexer for OpenQasm in pygments for highlighting, as well as allows for the option of formatted qasm output from circuit.qasm(formatted=True) and saving the qasm to a file circuit.qasm(filename="example.qasm").  This is needed for the circuit library but was added here as well since it is a nice feature.
An example:

Details and comments

I personally prefer the formatted output as default, but left as "raw" output for backward compatibility.

Currently fAiling due to a bad basicaer result that is unrelated


select2's not working on asp.net with masterpage

When I created normal page it worked fine
But when I use this under ContentPlaceHolder of masterpage it just doesn't work

This appears to be more of a usage question than a bug report or common feature request.  As we continue to keep the Select2 issue tracker geared towards these two areas, bug reports and common feature requests, we are redirecting usage questions to other communities such as the mailing list, IRC, or Stack Overflow.


Touch Gestures not having consistent input

Whenever a Touch Gesture input + any cursor movement happens (ie from a mouse or from having a pen on the same tablet) some Touch Gesture inputs are dropped.
Have only tested with Scroll bound to Paning so far.
This seems to be a linux only issue, the very limited testing I did on windows seemed to work as intended.

Pretty much all touch gestures are timing sensitives, which could be an issues if a pen & touch is used at the same time.
As for moving the mouse causing issues?
That is very odd, not something i or a couple other people (who ended up ghosting me when testing feedback was needed) noticed.
Will investigate, but maybe i should implement & provide a way in the app to test gestures and see when a gesture has started / bind was activated / touch was released (sometimes without any being involved)

Could you check if the same issue happens with single touch gesture using your pen.
(It should be a separate filter in the same tab and the setup knterface should be different)

Not sure If I did everything right. So I'll explain what I did.
Uninstalled all plugins but Scroll Bindings and Enhanced Out put Modes. extracted the contents of the plugin you linked into ~/.config/OpenTabletDriver/Plugins/Enhanced Output Modes. Ran the UX. Didnt have Pen Gestures so closed everything, redid extracting the plugin, and opened the OTD Daemon, GUI and your Touch Gestures UX. Enabled Pen Gestures and disabled Touch Gestures. Setup Scroll up bound to Pan UP and tested it. It worked for the first test, but Left click was still bound to the tip. Went to disable it and the scroll speed was a fraction of what it was before. I think I just closed and reopened the UX and added Scroll Down to Pan Down and it crashed
samelthecamel@Camel-Comp ~/OTDTesting $ ./Touch-Gestures.UX.Desktop Unhandled exception. System.InvalidOperationException: Operation is not valid due to the current state of the object.    at TouchGestures.UX.ViewModels.BindingsOverviewViewModel.OnSetupCompleted(Object sender, GestureAddedEventArgs e) in /home/runner/work/Touch-Gestures/Touch-Gestures/Touch-Gestures.UX/ViewModels/BindingsOverviewViewModel.cs:line 291    at TouchGestures.UX.ViewModels.GestureSetupWizardViewModel.OnSetupCompleted(Object sender, EventArgs e) in /home/runner/work/Touch-Gestures/Touch-Gestures/Touch-Gestures.UX/ViewModels/GestureSetupWizardViewModel.cs:line 156    at TouchGestures.UX.ViewModels.Controls.Setups.GestureSetupViewModel.OnSetupCompleted(GestureSetupViewModel sender) in /home/runner/work/Touch-Gestures/Touch-Gestures/Touch-Gestures.UX/ViewModels/Controls/Setups/GestureSetupViewModel.cs:line 200    at TouchGestures.UX.ViewModels.Controls.Setups.SwipeSetupViewModel.DoComplete() in /home/runner/work/Touch-Gestures/Touch-Gestures/Touch-Gestures.UX/ViewModels/Controls/Setups/SwipeSetupViewModel.cs:line 151    at CommunityToolkit.Mvvm.Input.RelayCommand.Execute(Object parameter)    at Avalonia.Controls.Button.OnClick()    at Avalonia.Controls.Button.OnPointerReleased(PointerReleasedEventArgs e)    at Avalonia.Reactive.LightweightObservableBase`1.PublishNext(T value)    at Avalonia.Interactivity.EventRoute.RaiseEventImpl(RoutedEventArgs e)    at Avalonia.Interactivity.Interactive.RaiseEvent(RoutedEventArgs e)    at Avalonia.Input.MouseDevice.MouseUp(IMouseDevice device, UInt64 timestamp, IInputRoot root, Point p, PointerPointProperties props, KeyModifiers inputModifiers, IInputElement hitTest)    at Avalonia.Input.MouseDevice.ProcessRawEvent(RawPointerEventArgs e)    at Avalonia.X11.X11PlatformThreading.RunLoop(CancellationToken cancellationToken)    at Avalonia.Threading.DispatcherFrame.Run(IControlledDispatcherImpl impl)    at Avalonia.Threading.Dispatcher.PushFrame(DispatcherFrame frame)    at Avalonia.Threading.Dispatcher.MainLoop(CancellationToken cancellationToken)    at Avalonia.Controls.ApplicationLifetimes.ClassicDesktopStyleApplicationLifetime.Start(String[] args)    at Avalonia.ClassicDesktopStyleApplicationLifetimeExtensions.StartWithClassicDesktopLifetime(AppBuilder builder, String[] args, Action`1 lifetimeBuilder)    at TouchGestures.UX.Desktop.Program.Main(String[] args) in /home/runner/work/Touch-Gestures/Touch-Gestures/Touch-Gestures.UX.Desktop/Program.cs:line 13 Aborted samelthecamel@Camel-Comp ~/OTDTesting $
somewhere in there my mouse was losing some functionally, think it didn't want to scroll while the Pen was touching the tablet but I don't remember the details.
Ill continue to test and come back with anymore notable results

Uninstalled all plugins but Scroll Bindings and Enhanced Out put Modes. extracted the contents of the plugin you linked into ~/.config/OpenTabletDriver/Plugins/Enhanced Output Modes

Odd, should only have to extract the release & overwrite files when necessary in the OTD.EnhancedOutputMode directory, then restart the service (Linux) or restart the daemon (Windows).
The exception you encountered only occur because the wizard somehow stopped being.

Modifying the Tip Binding at all provides the slower speed, using this site for my scroll testing

Abnormal to say the least.

remaking the Gestures fixed the slow scrolling speeds

This plugin stores gestures settings in a file named Touch-Gestures.json, it should be created in whatever directory OpenTabletDriver consider "AppData" :

$XDG_CONFIG_HOME/OpenTabletDriver,
~/.config/OpenTabletDriver

Could try backing it up, starting fresh and see if it occurs.

The same behavior I had with the touch is happening with the pen. But also happening with itself

If you can get a recording and see it here, or at least a recording of the recorded inputs, via OTD's Tablet debugger
(Tablet -> Tablet debugger), so i can throw it at the unit test and see what behavior comes out of it.

If you can get a recording and see it here

Accidentally streamed it instead of recording it, hopefully it still works out.
https://www.twitch.tv/videos/2288301273
The super slow speeds were when I was using the mouse at the same time as Pen Gestures
Pen at greater than 0 sens with Pen Gestures would be about half rate
I only had scroll up bound and had to reset the binding in the UX every time I changed the Sensitivity

Could try backing it up, starting fresh and see if it occurs.

Did not fix anything. fixing it is as simple as clicking the 3 dots then hitting apply then save and its good.

The only ways NextViewModel could not be the setup wizard are :

It might have crashed around the time I was making the down scroll binding. I don't remember exactly. But I do know that down scroll binding wasn't there when I opened it back up after the crash.

Pen at greater than 0 sens with Pen Gestures would be about half rate
Only way to get proper gesture input with the pen is by dropping the sensitivity to 0

It shouldn't be affected by pen sens, since it works using the raw coordinates provided by the tablet, and get triggered when a specific amount of lines is moved, no matter the area used.
Not exactly sure how to troubleshoot this at this point.
New Plans
Coming back to the original issue, Here is what I'm gonna add to my plans :

In-app, Gesture debugger, So i don't have to remote debug & place logpoints on every gestures,
Write more testing procedure,
Document the intended behavior of the UX,
Document the intended behavior of each gestures,
Investigate the scrolling behavior on more distros (Send Help), than just Raspi OS (Debian) & Mint (Ubuntu -> Debian).

Once the issue is identified, here is how I'm gonna proceed :

Fix Touch fix, do further testing, then request for testing from others,
Proceed the same way, for pen inputs.

There will be probably more to come, but for now, that's what i have in mind, that & the 5h of sleep, that is.
(US-like Schedule reset hit hard)

Investigate the scrolling behavior on more distros (Send Help), than just Raspi OS (Debian) & Mint (Ubuntu -> Debian).

I can distro hop through any you need testing done on. Is there anything other than just the scrolling behavior you would like me to test? I am fine with testing stuff unrelated to this if you like.
My current outline on what I plan to test is
Every major DE on whatever distro that implements them is most popular ie Gnome on Ubuntu
At least every Major distro ie Ubuntu, Fedora, Arch
That should leave me testing on
Ubuntu
Mint with Cinnamon
Fedora
openSUSE
Arch linux (prob just just with kde)
If Raspi OS works on regular hardware I can test that for you too
If there are any other distros you would like me to test on please let me know.
I will probably start testing on ether the 25th or the 2nd.

One of my install is Mint with cinamon, but it's possible things changed is newer versions, so prob worth investigating that

Finally managed to get a arch install working, and was unable to get the same behavior,
Will still proceed with the TODO whenever i have time as I'm preparing to start working in January.


Prepare 1.16.0-rc1 release

Merging this as the artifact is already published.

Merging this as the artifact is already published.


Dask client not running on Colab

Dask client is not running on Colab. My code is:
from dask.distributed import Client, progress
client = Client()
client

and the error is:
/usr/local/lib/python3.6/dist-packages/distributed/bokeh/core.py:57: UserWarning:
Port 8787 is already in use.
Perhaps you already have a cluster running?
Hosting the diagnostics dashboard on a random port instead.
  warnings.warn('\n' + msg)
tornado.application - ERROR - Multiple exceptions in yield list
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/tornado/gen.py", line 828, in callback
    result_list.append(f.result())
  File "/usr/local/lib/python3.6/dist-packages/tornado/concurrent.py", line 238, in result
    raise_exc_info(self._exc_info)
  File "<string>", line 4, in raise_exc_info
  File "/usr/local/lib/python3.6/dist-packages/tornado/gen.py", line 1069, in run
    yielded = self.gen.send(value)
  File "/usr/local/lib/python3.6/dist-packages/distributed/deploy/local.py", line 229, in _start_worker
    raise gen.TimeoutError("Worker failed to start")
tornado.gen.TimeoutError: Worker failed to start
---------------------------------------------------------------------------
TimeoutError                              Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/tornado/gen.py in callback(f)
    827                 try:
--> 828                     result_list.append(f.result())
    829                 except Exception as e:

33 frames
TimeoutError: Worker failed to start

During handling of the above exception, another exception occurred:

TimeoutError                              Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/distributed/deploy/local.py in _start_worker(self, death_timeout, **kwargs)
    227         if w.status == 'closed' and self.scheduler.status == 'running':
    228             self.workers.remove(w)
--> 229             raise gen.TimeoutError("Worker failed to start")
    230
    231         raise gen.Return(w)

TimeoutError: Worker failed to start

To reproduce the error please see the following Colab gist:
https://colab.research.google.com/gist/cornhundred/70524e9425db14f3adec9e86660aa313/trying_dask_on_colab.ipynb

Are you in contact with anyone from colab about this?

Not yet, but I'll raise the issue with Colab.

My guess is that they don't allow users to start processes.  You might try the following for now:
client = Client(processes=False)

Though there are performance implications of this, particularly when using GIL-bound computations.

@cornhundred, have you had a chance to try Matt's suggestion above?

Hi @jakirkham, yes adding the argument processes=False worked - see gist https://colab.research.google.com/gist/cornhundred/1fa599c4cb09cded2a598cc02ca3947b/trying_dask_on_colab_no_processes.ipynb
However, I have to read the documentation more to understand the consequences of not using proceses. Also, the Dashboard link is not working.

Glad to hear that worked.
Could you please raise another Colab issue about the Dashboard being inaccessible?

After talking with a colleague about the dashboard issue, they recommended using ngrok. This SO answer may also be useful for setting it up.
Would also be curious if jupyter-server-proxy works here. If not, it might be worth raising an issue on jupyter-server-proxy's tracker with whatever outcome is seen.

@cornhundred I suspect the dashboard being inaccessible is because printing the client info in Colab gives you a localhost address:
Dashboard: http://localhost:8787/status |

I've dug around for ways to expose it, and the simplest I've seen is using ngrok like so:
# expose Dask's status dashboard to a public URL
!wget https://bin.equinox.io/c/4VmDzA7iaHb/ngrok-stable-linux-amd64.zip
!unzip ngrok-stable-linux-amd64.zip

get_ipython().system_raw('./ngrok http 8787 &')

!curl -s http://localhost:4040/api/tunnels | python3 -c \
    "import sys, json; print(json.load(sys.stdin)['tunnels'][0]['public_url'])"

That should print a link which you can click and get the Dask Status Dashboard for your Colab instance.

@cornhundred, does this info help?

Hi @jakirkham, I made a gist with the changes suggested by @randerzander (https://colab.research.google.com/gist/cornhundred/7b6f535d279cc2005a13f6a6e21e00bd/trying_dask_on_colab_no_processes.ipynb#scrollTo=rQuhrN9ZxXLj), but I'm getting a "Web Page Blocked" error.
Access to the web page you were trying to visit has been blocked in accordance with company policy. Please contact the helpdesk x4HELP if you believe this is in error.
It sounds like this might be an issue with my work internet connection. I haven't tried this from my home computer yet.

Did you have a chance to try on your home computer? Am curious to know if it worked there.

Hi @jakirkham, yes the link worked when I ran the notebook off my cellphone's WiFi hotspot.

Is there anything left to do here, or can this be closed?

@mrocklin, I think it's resolved.
However, Colab does not appear to support processes (but this is a Colab issue).

Hi @jakirkham and @mrocklin, yes adding the argument processes=False worked - see gist https://colab.research.google.com/gist/cornhundred/1fa599c4cb09cded2a598cc02ca3947b/trying_dask_on_colab_no_processes.ipynb
However, I have to read the documentation more to understand the consequences of not using proceses. Also, the Dashboard link is not working.

I get
/usr/local/lib/python3.6/dist-packages/distributed/bokeh/core.py:57: UserWarning:
Failed to start diagnostics server on port 8787. [Errno 99] Cannot assign requested address
  warnings.warn('\n' + msg)
/usr/local/lib/python3.6/dist-packages/distributed/deploy/local.py:197: UserWarning:
Could not launch service 'bokeh' on port 8787. Got the following message:

[Errno 99] Cannot assign requested address
  self.scheduler.start(scheduler_address)

error in Colab now

from IPython.display import IFrame
IFrame(src='http://localhost:8787', width=900, height=600)

shows the UI though the client doesn't start

@Nithanaroy does colab restrict which ports procesess talk over?

That could be reason but I'm not sure. Is it possible to change the default 8787?

AIUI (reaffirmed by talking to a colleague offline) Colab doesn't support opening any ports. Maybe worth proding issue ( https://github.com/googlecolab/colabtools/issues/569 ) to let people know this is still an issue people are interested in?
So is running client = Client(processes=False) not working for you? If not, we might need to check that the dashboard is not setup here or figure out a way for you to disable it.

Ah sorry have you tried Client(processes=False, diagnostic_port=None)?
https://github.com/googlecolab/colabtools/issues/569#issuecomment-578392622

from dask.distributed import Client
client = Client(processes=False, diagnostic_port=None)

results in:
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-11-78f089029570> in <module>()
      1 from dask.distributed import Client, progress
----> 2 client = Client(processes=False, diagnostic_port=None)
      3 client

14 frames
/usr/lib/python3.6/asyncio/events.py in get_event_loop(self)
    600         if self._local._loop is None:
    601             raise RuntimeError('There is no current event loop in thread %r.'
--> 602                                % threading.current_thread().name)
    603         return self._local._loop
    604

RuntimeError: There is no current event loop in thread 'IO loop'.

What versions of Dask + Distributed are you using?

Edited my above comment to include versions and try seting both dashboard_address and diagnostic_port.

Interesting was not able to reproduce locally. Were you testing on Colab?

Ah sorry have you tried Client(processes=False, diagnostic_port=None)?
googlecolab/colabtools#569 (comment)

Thanks @jakirkham for the suggestion. Looks like it is not working on Google Colab with dask 2.15 as well. Created one with public access at https://colab.research.google.com/drive/1-gH4MYXDhj9CVBqjYdwWq12F9NSrTqLr

Any update on this? I'm getting the same error messages on Colab.


Button to clear permissions on new user pages

Trello: https://trello.com/c/c6TxIASX
In #2436 we ensured that the default permissions for new users were checked in the permissions checkboxes on when inviting new users (singly or in a batch). This is a big time-saver for some inviting users who frequently add these
permissions. However there's a subset of inviting users who set up new user accounts who don't need these default permissions (e.g. when setting up users who use the Licensing app).
This PR adds a "Clear selected permissions" button to the /users/invitations/new and /batch_invitations/:id/permissions/new pages to make it easier for those inviting users to complete their task.
Clicking the button clears the checked boxes and resets the "selected" counter for each app
https://github.com/alphagov/signon/assets/16707/570af1c2-a658-4003-8735-511ca5ef5bde

One question: Does it definitely make sense to have the button outside the form? And if so, would it make sense to set the aria_controls option (see these docs) for the component to point at the form?

Looking at this again I can see the button is inside the form (and even the relevant fieldset) - I must've got confused when I looked at the diff last week. I did still contemplate adding an id to the fieldset and setting the button's aria-controls attribute to point to it, but I decided it probably wasn't worth it since the button is so "near" the checkboxes that it controls.

I've just noticed some single quotes which have been accidentally added in the HTML, so I'm going to fix up the relevant commits.

I've just noticed some single quotes which have been accidentally added in the HTML, so I'm going to fix up the relevant commits.

Addressed in:

https://github.com/alphagov/signon/pull/2451/commits/196e09a3ff5288440ca552842c731ddeb64926f1
https://github.com/alphagov/signon/pull/2451/commits/00c42dab9915089b44024a3bf3d7eff080a2ee74

I'm happy to get this merged now!


added axios to dependencies

@virenb Sorry, this issue has been resolved https://github.com/MadlabsInc/mevn-cli/pull/51. I would encourage you to take any other issue which seems interesting for you :smile:


Return data type for download functions

Currently we are returning Uint8Array types from the download functions (and pssReceive) because it is suitable type for representing potentially binary data. However it is not easy to use and most people are not familiar with it. For example to convert it to string you have to use the TextDecoder class which is also something most developers are not familiar with (for illustration see #174).
Maybe it would be better to define a new type that could hold a Uint8Array for the binary data representation but would also provide conversion functions to text and json, similarly to the Response API used by fetch. For example:
interface Data {
   readonly bytes: Uint8Array
   text: () => string
   json: () => Record<string, any>
}

Then we could return this interface from our functions everywhere with the help of an implementation that takes a Uint8Array and creates this type from it.

The interface could also include a hex: () => HexString function that returns the data as a hex string.

Yup! I like it and I think it is really necessary.

Hmm should the hex() function return prefixed or non-prefixed hex? 😅

I would say non-prefixed. Prefixing with '0x' seems easier than .slice(2) to me. Alternatively we can provide another function that returns prefixed hex.

I have come up with two options for how to approach this. One will introduce breaking change and one not.

Wrap the bytes as @agazso proposed. Eq. returning instead of Uint8Array an object like he proposed. (breaking change)
Extend the Uint8Array object with the helpers functions like this:

export function wrapBytesWithHelpers(data: Uint8Array): Data {
  return Object.assign(data, {
    text: () => new TextDecoder('utf-8').decode(data),
    json: () => JSON.parse(new TextDecoder().decode(data)),
    hex: () => bytesToHex(data),
  })
}

What do you think?

I was thinking of this too and this is also good. And considering that it does not cause a breaking change I would go with it.

Hmm should the hex() function return prefixed or non-prefixed hex? 😅

I am all for non-prefixed, but you could add optional flag
hex: (prefixed = false) => `${prefixed ? '0x' : ''}${bytesToHex(data)}`

I have come up with two options for how to approach this.

Even though it is a breaking change, I would personally prefer option 1. But no strong prefference

Hmm should the hex() function return prefixed or non-prefixed hex? 😅

I am all for non-prefixed, but you could add optional flag
hex: (prefixed = false) => `${prefixed ? '0x' : ''}${bytesToHex(data)}`

That's possible to do, however then the return type will be ambiguous (e.g. HexString | PrefixedHexString), which is not a problem for Javascript users but it may be a bit more difficult to work with for Typescript users.

Ok, the only thing I am not sure if we won't introduce a memory leak with this approach of "wrapping" (it is applicable for both approaches), because they atm. rely on closure and I am not sure if the closure will be properly GCed. What do you think?

It's only a problem with GC when you have circular references. This case is equivalent having a class with the data passed in the constructor. Actually it would be also possible to do that, but I usually prefer a more functional approach.


plugin-cloud-storage: Update S3 URL generation

Currently, the plugin generates URLs like that:
https://s3.us-west-1.amazonaws.com/my-bucket/my-prefix/my-image.jpg

Notice how the bucket is specified as part of the path. S3 also supports specifying the bucket as part of the origin, like that:
https://my-bucket.s3.us-west-1.amazonaws.com/my-prefix/my-image.jpg

I think the latter is preferred, because all URLs in the Amazon console look like that. Also, this answer on Server Fault states that Amazon my sometimes redirect from the old deprecated form (that Payload uses) to the new one.
I think this should be changed:
https://github.com/payloadcms/plugin-cloud-storage/blob/e11a0fb28534b7d9fd0e91b51e7ab4df6cad664c/src/adapters/s3/generateURL.ts#L10-L14
If the new URL scheme is used, I think the bucket option can be dropped, because the endpoint will contain the bucket name already.

Moving this to feature request. We can continue the discussion there.


Wishlist of obs datasets

Adding observation to ADF.
What is this new feature?
Jesse is adding model to obs comparison. In this issue, we are collecting the list of datasets and variables we want to prioritize to be able to evaluate the model simulations.
SWCF

Dataset: CERES-EBAF Ed4.1
namelist of variable: SWCF
Location: /glade/work/brianpm/observations/climo_files/CERES_EBAF_Ed4.1_SWCF_climo.nc

LWCF

Dataset: CERES-EBAF Ed4.1
namelist of variable: LWCF
Location: /glade/work/brianpm/observations/climo_files/CERES_EBAF_Ed4.1_SWCF_climo.nc

FLNT

Dataset: CERES-EBAF
namelist of variable: toa_sw_all_mon
Location: /glade/work/brianpm/observations/climo_files/CERES_EBAF_Ed4.1_climo.nc

FSNT

Dataset: CERES-EBAF Ed4.1
namelist of variable: toa_sw_all_mon
Location: /glade/work/brianpm/observations/climo_files/CERES_EBAF_Ed4.1_climo.nc

PSL
Dataset 1

Dataset: MERRA
namelist of variable: PSL
Location: /glade/p/cesm/amwg/amwg_data/obs_data/MERRA_*.nc

Dataset 2

Dataset: ERA-Interim
namelist of variable: PSL
Location: /glade/p/cesm/amwg/amwg_data/obs_data/ERAI_*.nc

SST
We compare at different periods
pre-industrial

Dataset: HadISST/OI.v2 (pre-industrial: 1870-1899)
namelist of variable: SST
Location: /glade/p/cesm/amwg/amwg_data/obs_data/HadISST_PI_*

present day

Dataset: HadISST/OI.v2 (present day: 1999-2008)
namelist of variable: SST
Location: /glade/p/cesm/amwg/amwg_data/obs_data/HadISST_PD_*

PRECT
Global

Dataset: GPCP
namelist of variable: PRECT
Location: /glade/p/cesm/amwg/amwg_data/obs_data/GPCP_*.nc

Tropical

Dataset:  SSMI
namelist of variable: PRECT_OCEAN
Location: /glade/p/cesm/amwg/amwg_data/obs_data/SSMI_*

PREH2O

Dataset:
namelist of variable:
Location:

TGCLDLWP

Dataset: UWisc LWP Climatology
namelist of variable: TGCLDLWP_OCEAN
Location: /glade/p/cesm/amwg/amwg_data/obs_data/UWisc_LWP_*

SURF STRESS (TAUX and TAUY)

Dataset: Large-Yeager 1984-2004
namelist of variable: (TAUX and TAUY)
Location: /glade/p/cesm/amwg/amwg_data/obs_data/LARYEA_*

Next Steps

[ ] Step 1: agree on datasets.
Please review the set of observations suggested here.
If you have any additions/comments/feedback, leave a comment below.
If you agree with everything, leave a comment to say so.

[ ] Step 2:  Put the datasets in a single long-term location.
Where should the dataset lives?
For instance, somewhere in: /glade/p/cesmdata/cseg/inputdata or /glade/p/cesm/amwg or ???
Should we have a repo to outside people can access them?

[ ] Step 3: Rename datasets and add metadata as needed.
It is important to have metadata that indicates version, creator, etc.
A while back, we had started to require AMWG metadata for new any AMWG dataset.
Attempt for CESM metadata was discussed here

Next Steps

[ ] Step 1: agree on datasets.
Please review the set of observations suggested here.
If you have any additions/comments/feedback, leave a comment below.
If you agree with everything, leave a comment to say so.

[ ] Step 2:  Put the datasets in a single long-term location.
Where should the dataset lives?
For instance, somewhere in: /glade/p/cesmdata/cseg/inputdata or /glade/p/cesm/amwg or ???
Should we have a repo to outside people can access them?

[ ] Step 3: Rename datasets and add metadata as needed.
It is important to have metadata that indicates version, creator, etc.
A while back, we had started to require AMWG metadata for new any AMWG dataset.
Attempt for CESM metadata was discussed here

Would love feedback on next steps (@JulioTBacmeister, @swrneale, @andrewgettelman)

The list looks good.
For cloud 'microphysics' (CLOUD, TGCLDLWP, TAU, REFF) we probably want to be using the COSP MODIS simulator and associated observations. Especially for cloud fraction. We do have this data from the existing diagnostics, we just have to compare to different fields.
We could also add U, V, T, and probably compare to ERA-Interim (or ERA5). We should have this data already.

I think this list is good for a starting point. I agree with Andrew about using COSP diagnostics for some of this, including cloud cover (e.g., we should not compare CLDLOW to satellite "low cloud" products). Maybe these can be added incrementally?
While we are in transition to handling a larger and more diverse group of observations with intake-esm (https://github.com/NCAR/ADF/issues/102 , https://github.com/NCAR/ADF/issues/25), a stop-gap solution might be to make a combined, homogenized dataset out of this list. By this, I just mean take these handful of datasets and remap them to the FV1° grid (or whatever grid we want to use for evaluation for the rest of 2022), name them to the corresponding CAM variables, put a comment or something into each one's metadata to indicate the source and time-span, and put them all into climo files that match the ADF convention. That might make it easy to deal with them very similarly as we do with CAM cases within the ADF scripts. Maybe @nusbaume  would disagree though?
In the medium-term, I would suggest that we need to update a lot of these datasets. In some cases because there are new (hopefully improved) products (e.g., CERES Ed2.x -> CERES Ed4.1, ERAI -> ERA5). In other cases, just because the observations now cover a longer time period.  And I totally agree with next step numbers 2 & 3, finding a permanent place and getting good metadata is crucial.

Should one of the highest priorities be to easily swap in a new data set? In the short term, having an easy-to-find list of the files to use, that a user could change and ADF uses to determine which one to read in, should make that a lot easier. Are we thinking of putting this in the variable attributes information? If so, that would work too.
That unfortunately argues against a combined data set that would have to be remade every time any one changes. Although I think having an already-regridded version of each file would be valuable.

I second Dani's comment: better if it is incremental.
I think all that would be needed is if we had variable attributes for
obs_file_name
obs_var_name
obs_scl
Then as long as there was lat,lon,pressure (if needed) and a reasonable monthly time coordinate, I think the plotting codes could figure out how to load the observations for any variable and get them in the right units. Flexible, could be changed, some manual intervention, but I think that's fine. Also iterative.

@cecilehannay I presume this is for ANN,DJF->SON,JAN->DEC climo. datasets?
I think I agree with Brian, that we should grab updates to the datasets and put them mostly in the existing format so that Cecile could reasonably transition to ADP for CAM5 dev. simulations.
Do we want to make any attempt to overlap observational periods, or should we just grab the longest periods we can each time?
In terms of observational fields we should consult the https://climatedataguide.ucar.edu/ before we do some heavy lifting ourselves. A few things missing: Surface latent and sensible heats (not analysis based) and precipitable water (NCAP available 1988-2009  https://asdc.larc.nasa.gov/project/NVAP-M/NVAP_CLIMATE_Total-Precipitable-Water_1)
Rich

In case this helps, my general long-term plan for dealing with observations currently looks something like this:

There is an "official" Intake-ESM catalog that contains all of the (gridded) observational datasets that we want to compare the model against by default.  I believe if set-up properly this catalog can manage any sort of temporal resolution and spatial grid, but there will likely need to be fairly strict meta-data requirements for those observational files in order for the ADF to properly search for them in the catalog.  In general too I imagine a dataset wouldn't be added to this catalog until it was "blessed" by someone at AMP.

The ADF will also support the ability for the user to specify their own, non-official observational datasets, which can be done via the current variable meta-data YAML file.  I am happy if we want to require certain observational file features and meta-data, but I was planning to have the ADF basically accept almost anything and try its best to match the model data to that observational file.  This would allow a user to easily add their own observations while protecting the "official" observational data most users will want.

For the short-term I am basically going to implement option 2.  Then once that is working I can bring in the infrastructure needed for Intake-ESM, while at the same time we collectively agree on and update the observational datasets we want.  At that point I can create the "official" catalog, and we should then have both options available.

Good plan.  I like starting with option 2 and keeping it simple for now, so we can get going with existing data sets, and allow easy extensibility and minimal overhead. When option 1 is on line, we can start to migrate obs over. But that should happen later.  Thanks!

Hi All,
I just wanted to notify everyone possibly watching this thread that I have recently implemented model vs obs comparisons using the variable defaults file to specify what observational data set to use.  You can currently specify the observational file to use (either as just a file name or as a full path if it is located somewhere unique), the name of the observational data set (which will eventually be plugged into plot titles, webpages, etc.), and the name of the variable on the observational file that you want to use (so multiple observational variables can be located in a single file).
Currently the observations can be on any structured lat/lon grid you want, and the only new meta-data requirement is that the observations variable must have a units attribute (which is probably a good idea in general).
In terms of missing features, there is currently no way to deal with 3-D observational data, and the ADF assumes that the observations themselves are monthly climatologies with a time dimension of length 12 (one for each month).  Example files can be found on Cheyenne/Casper here (with credit going to @brianpm for the data files themselves):
/glade/work/nusbaume/SE_projects/model_diagnostics/ADF_obs
Of course I am hoping to remove all of these restrictions eventually, so if you have an observational data set you want to use that has a vertical dimension, or that has a different time dimension (e.g. seasonal or daily values) please let me know and I'll help with adding the necessary ADF functionality.
Thanks, and have a good weekend!

@nusbaume: I am a bit confused how to run versus obs.

Adding some notes from Jill Zhang at LLNL on the E3SM diagnostics and observations:
Thank you for your feedback! I looked a bit into the provenance of the AODVIS dataset, this observational composite dataset was derived based on the MACv1 (Max-Planck-Institute Aerosol Climatology) dataset from MPI. https://agupubs.onlinelibrary.wiley.com/doi/full/10.1002/jame.20035
In a recent effort to get in more aerosol diagnostics, I found a more recent version MACv2 available (https://www.tandfonline.com/doi/full/10.1080/16000889.2019.1623639), that I haven’t had a chance to integrate the new dataset.
All of our processed datasets are available publicly at https://web.lcrc.anl.gov/public/e3sm/diagnostics/observations/Atm/,
which contains 4 sub directories that supply data to E3SM diags:
/climatology         #include seasonal and annual mean climatology
/time-series         # include time-series datasets if available
/arm-diags-data  # time-series derived from ARM facilities
/tc-analysis         # tropical cycle tracks datasets
The AOD datasets is here: https://web.lcrc.anl.gov/public/e3sm/diagnostics/observations/Atm/climatology/AOD_550/
And as you pointed out, some datasets were grabbed from AMWG, including CRU, SSMI and COSP-OBS datasets. I only assembled the original time series from individual simulators, It would be great if we can get some help or work together for the COSP-OBS!
Thanks,
Jill

FWIW, I have processed the latest MODIS data into a first draft of a climo file for use in ADF. This is based on initial processing by @jshaw35. The file is here:
/glade/work/brianpm/observations/MODIS/climo/MCD06COSP_M3_MODIS.climo.200301-202012.nc
The variables are: CLMODIS (the histogram), CLTMODIS, CLHMODIS, CLMMODIS, CLLMODIS, CLDTHCK_MODIS (high, optically thick clouds), and cloud_mask (which won't be used in general).
Averaging interval is 2003-2020.
UNTESTED so if you see anything weird, just let me know and I can re-process.

@brianpm This is great! You beat me to it. I'd be happy to test the datasets.
My NCAR computer account was expired about two years ago. I'm submitting a new account request to get data from GLADE.

Okay, another first draft climatology, this time from CALIPSO GOCCP.
Original data from https://climserv.ipsl.polytechnique.fr/cfmip-obs/Calipso_goccp.html
I took the monthly data for cloud cover maps, cloud phase maps, and cloud fraction profiles and processed them to the monthly climatology. Simple averaging with nothing else going on. It is worth noting that no correction/adjustment is made for the South Atlantic Anomaly, so caution is advised. Global averages should probably mask out the affected region.
Files on glade:
/glade/work/brianpm/observations/clcalipso/climo/3D_CloudFraction330m_200606-202012_climo_CFMIP2_sat_3.1.2.nc
/glade/work/brianpm/observations/clcalipso/climo/MapLowMidHigh330m_200606-202012_climo_CFMIP2_sat_3.1.2.nc
/glade/work/brianpm/observations/clcalipso/climo/MapLowMidHigh_Phase330m_200606-202012_climo_CFMIP2_sat_3.1.2.nc

MACv2 aerosol is now processed.
/glade/work/chengzhu/analysis_data_e3sm_diags/MACv2/climatology
I extracted aod at 550nm, more aerosol optical properties are available if interested, they are available under MACv2/original_full_set. Limited testing has been done, for AOD55nm, the global mean matches values in the paper.
Not sure if the AMWG diags style, monthly and seasonal mean climo files still useful for ADF, but i think they should be easily to be converted to the 12 months based climo. My processing script is under MACv2/scripts.

ISCCP climatology. This is updated to use the ISCCP-H series data. The CTP-TAU histograms are not available in the monthly "basic" files. @Isaaciwd produced monthly mean files from the 3-hourly files. We validated that the values are close to the available monthly means; there is a slight discrepancy that appears to be attributable to the difference in the order of operations vis-à-vis when remapping from equal area to equal angle grids occurs. I took the derived monthly files and made this climo file:
/glade/work/brianpm/observations/isccp/climo/ISCCP-Basic.HGG.GLOBAL.10KM.climo.198307-201706.nc

This file contains:

FISCCP1_COSP : the CTP-TAU histogram (7x6)
MEANPTOP_ISCCP : mean cloud-top pressure
MEANTAU_ISCCP : mean cloud optical thickness
cosp_prs_bnds : bin bounds for pressure
cosp_tau_bnds : bin bounds for tau
CLDHGH_ISCCP : high cloud fraction
CLDMED_ISCCP : middle cloud fraction
CLDLOW_ISCCP : low cloud fraction
CLDTOT_ISCCP : total cloud fraction

These were renamed to match CAM's COSP outputs, but not much else was checked for consistency with CAM.

@brianpm thank you for the update. It is great that the coordinates/variable names are reformatted to match CAM. I'm wondering if you could share the script, I'm thinking to generate processing scripts that can write out two versions of data (the new ADF version and the AMWG seasonal mean version). Thank you!

@chengzhuzhang -- Yes, I can definitely share the script. I noticed that the CLDTOT_ISCCP values are higher than I expected. I'm going to try to check on that today (mainly I can't remember if I am supposed to skip the lowest optical depth bin). I'll confirm and remake the file if needed. I can put my script somewhere (maybe just in one of my github repos). I will try to get Isaac's script for doing the 3hr-to-monthly calculation, too.


Fix sed command

Hi, @aceoliver! Thank you so much for your PR 👍
Unfortunately, your fix will fail the installation on MacOS so I made a new PR for dealing with it: https://github.com/infinitered/ignite-ir-boilerplate-bowser/pull/45
It's because the sed command on MacOS, unlike the other POSIX OS takes a backup extension parameter when using -i flag.
Thanks!


Panic with HEAD requests when any other route contains a param

main.go
package main

import (
	"net/http"
	"log"

	"github.com/husobee/vestigo"
)

func main() {
	router := vestigo.NewRouter()
	router.Get("/:param/foo", func(w http.ResponseWriter, r *http.Request) {})

	http.Handle("/", router)
	log.Fatal(http.ListenAndServe(":9000", router))
}

Commands
go run main.go
curl -I localhost:9000/foo

Result
2016/06/22 18:11:42 http: panic serving <IP_ADDRESS>:36656: runtime error: invalid memory address or nil pointer dereference
goroutine 5 [running]:
net/http.(*conn).serve.func1(0xc820072100)
	/usr/local/go/src/net/http/server.go:1389 +0xc1
panic(0x728500, 0xc82000a130)
	/usr/local/go/src/runtime/panic.go:426 +0x4e9
github.com/husobee/vestigo.glob.func2.1(0x7fa51cd6c8c8, 0xc820075a00, 0xc8200dc000)
	/home/ernesto/goprojects/src/github.com/husobee/vestigo/handlers.go:41 +0x138
github.com/husobee/vestigo.glob.func6.1(0x7fa51cd6c8c8, 0xc820075a00, 0xc8200dc000)
	/home/ernesto/goprojects/src/github.com/husobee/vestigo/handlers.go:112 +0x3e7
github.com/husobee/vestigo.(*Router).ServeHTTP(0xc82000aef0, 0x7fa51cd6c8c8, 0xc820075a00, 0xc8200dc000)
	/home/ernesto/goprojects/src/github.com/husobee/vestigo/router.go:54 +0x52
net/http.serverHandler.ServeHTTP(0xc820072080, 0x7fa51cd6c8c8, 0xc820075a00, 0xc8200dc000)
	/usr/local/go/src/net/http/server.go:2081 +0x19e
net/http.(*conn).serve(0xc820072100)
	/usr/local/go/src/net/http/server.go:1472 +0xf2e
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:2137 +0x44e

If the route is replaced with router.Get("/foo", func(w http.ResponseWriter, r *http.Request) {}) there is no error. If it's replaced with router.Get("/bar", func(w http.ResponseWriter, r *http.Request) {}) the expected 404 error is returned.

@ernestoalejo: here is a PR with a fix to this problem -> https://github.com/husobee/vestigo/pull/24
This problem is due to the headHandler calling the get handler for that particular node.  If there is no handler, (because the node doesnt have a get handler) it tries to run the nil function.  Please let me know if this PR fixes this issue, and I will get it merged in.

I can confirm the change fixes the issue. Thanks for the quick patch!

https://github.com/husobee/vestigo/pull/24 has been merged with this fix.  Feel free to close this issue.


[Feature] Make Transaction: Typed2718

Component
consensus, eips, genesis
Describe the feature you would like
the transaction trait currently has its own fn ty function.
we can unify this with the newer Typed2718 trait by

remove fn ty from trait Transaction
require trait Transaction: Typed2718

Additional context
No response

:raised_back_of_hand:


Cannot generate a manufacturable .obj model

On macOS, I installed julia with brew install julia.
I modified the cat image to be black and white first, and save as a PNG:

I added the Images package to my global environment. Then ran julia create_mesh.jl cat_posing.png. It generates the following .obj model, as seen imported into MeshMixer:

Zipped and added here for reference:
cat_posing.png.obj.zip

Unless I'm missing something, this model doesn't appear to have the desired features that would produce a "magic window". I'd love to help but thought I'd first see if anyone else has had success generating a model successfully.

@timtomch thanks for the reply. How did you open the .obj file in Fusion 360 though? When I upload it and open it, it appears to be an empty mesh:


Update Twitter ID

Jenkins, test this please.

Jenkins CI test (for commit 6dbd14de5d2a81d2c8df58160b58c8494d3772df, target branch master) succeeded!

LGTM.


Fixed gif.GIF initialization

Go 1.5 has more variables in gif.GIF struct, using named variable initialization.

Thanks! Have merged the PR


Used in other Tilemap format

Hi vocksel, I'm creating a map editor for https://github.com/CleverRaven/Cataclysm-DDA , it is not using Tiled, so I have to copy-paste and modify some of your code to achieve this, just let you know I'm using your code, thank you!

Great! I hope this repo can be a valuable resource for your project :)


Improve documentation

Document the following areas of the CLI:

[ ] new
[ ] init
[ ] generate

[ ] component
[ ] class
[ ] directive
[ ] enum
[ ] interface
[ ] module
[ ] pipe
[ ] service

[ ] test
[ ] e2e

Hey guys!
I thought about this yesterday when I discovered that all the generate commands have many options. Any idea of where to start? README.md or a separate guide?
Cheers.

Still trying to figure that out for now, wanted to have a placeholder issue to track against.

It would be great to put more docs to https://cli.angular.io/ with similiar doc style which is used on angular.io

Next action item:

[ ] Move the WIKI content to /docs/wiki in the repo. The release should publish the wiki from git.

Following this updated doc to integrate Bootstrap with angular-cli results into following module conflicts:
jquery.js
tether.js
Details errors are as below:
**ERROR in multi scripts
Module not found: Error: Can't resolve 'C:\word\angular-demo\node_modules\jquery\dist\jquery.js' in 'C:\word\angular-demo\node_modules\angular-cli\models'
@ multi scripts
ERROR in multi scripts
Module not found: Error: Can't resolve 'C:\word\angular-demo\node_modules\tether\dist\js\tether.js' in 'C:\word\angular-demo\node_modules\angular-cli\models'
@ multi scripts**
I'm using below configuration:
angular-cli: 1.0.0-beta.20-4
node: 7.1.0(npm-4.0.3)
os: win32 x64
bootstrap : 3.3.7
Any help in this matter would be really appreciated!
Thanks,
Mitesh

@Meet91 I just verified my local app with no issues. Based upon your errors it seems like you're attempting to pull in more than just the bootstrap CSS as my guide suggests. If that's not the case please create a separate issue on this repo or post a question on StackOverflow. Make sure to either tag me in the issue and/or link to it here so I can find it. (I don't have a windows environment setup, so I'm testing on macOS v10.12.1)

@Meet91 see https://github.com/angular/angular-cli/issues/3202#issuecomment-262928724
@Brocco do you think this issue stems from a bad understanding of some configuration, or should I create a merge request to update to README, waiting for some later resolution

@noelmace I updated the angular-cli to beta.22-1 and error goes away and also tested with the bootstrap integration and all works great...Thank you so much for responding on such a short period!!!

@Brocco - can you elaborate a little bit on what you had in mind for Setup of global styles?

@jeremymwells global styles are the style files referenced via angular-cli.json this would be an overview of:

how to setup a reference here
an overview of the implications of setting up a style here versus within an individual component

This is not meant to be a complicated explanation, just an overview on how to achieve application-wide styling.

Added i18n to your list, @Brocco.

@Brocco added angular-cli.json to the list.

I would appreciate if angular-cli docs were constantly updated with minimum requirements of relevant dependencies, for each release.

I need to work on a serializer of the documentation for hte JSON Schema.

Considering this done. Expect more stories post 1.0.
@moniuch that's a great idea, but we're pretty lax on our requirements. The only relevant dependency is Angular and TypeScript :)

There is a mismatch to protrator.conf.json that the project line is not e2e but e2e/tsconfig.e2e.json from the generator output.
beforeLaunch: function() {
    require('ts-node').register({
      project: 'e2e/tsconfig.e2e.json'
    });
  },

Is there any reference to Setup of global styles that can be subscribed to on GitHub?

Found this in the docs https://github.com/angular/angular-cli/blob/master/docs/documentation/stories/global-styles.md, but I think what I meant by that was is there a way to add global Sass variables for use with components.  Otherwise, you're always importing a variables file, palette file, etc throughout your Sass files.


OAP-Spark  InsertIntoHadoopFsRelationCommand.scala

Reason:
The Changed code is because the related function in "FileFormatWriter" has been adjusted.
@@ -162,7 +162,7 @@ case class InsertIntoHadoopFsRelationCommand(
           partitionColumns = partitionColumns,
           bucketSpec = bucketSpec,
           statsTrackers = Seq(basicWriteJobStatsTracker(hadoopConf)),
-          options = options)
+          options = options)._2

resovled


Create Data Page

Please do work for this task in a branch called issue-XXX.

Created new page DataPage.jsx. Ran lint to check the code.


RFC 0003: WaitGroup

Provide a mechanism to wait on the execution of a set of operations distributed to a set of fibers.
Preview: https://github.com/crystal-lang/rfcs/blob/rfc-wait-group/text/0003-wait-group.md

Related discussion on the forum: https://forum.crystal-lang.org/t/waitgroup-implementation/4293

Really happy to see this! It feels very Go-centric as is. I think having to manually add fiber counts and decrement via wg.done are what make it feel that way.
@RX14's suggested helper method would help a lot. Could the API used by https://github.com/GrottoPress/pond be adapted to be as efficient as this implementation?
It would look like this:
def sliced_operation(wg, i)
  32.times do |j|
    wg << spawn do
      sub_sliced_operation(i, j)
    end
  end
end

wg = WaitGroup.new

16.times do |i|
  wg << spawn do
    prepare_slice(i)
    sliced_operation(wg, i)
  end
end

wg.wait

@mgomes Pond has an interesting approach. It's closer to nurseries in structured concurrency (keep a list of child fibers) than a mere counter though, and I feel like it would and up duplicating what such nurseries would do.
Looking at your example, I feel that the #spawn(&) helper would do just the same, and I prefer how abstract it is over #<<(fiber).

I resolved the unresolved questions as discussed above.