Change the CurseForge slug id to match the actual one

On curseforge the slug is more-chat-history whereas the modrinth and actual slug is morechathistory. this causes issues when marking compatibility, and is just generally confusing. should be a simple fix too!

can do, though i'm pretty sure I changed it in the opposite direction at some point in history. Out of curiosity, what tool doesn't let you use project IDs / different slugs when marking dependencies? I imagine something like REI would annoy it as it uses rei, roughly-enough-items and roughlyenoughitems.

it caused an error when i tried to publish a version of my mod chat patches that was previously incompatible with more chat history. that's no longer the case so it's not an issue for me anymore, but when the publishing plugin tried to run it couldn't find the curse forge link.
i hope that makes some sense lol!

yeah that's fair enough, was just curious. will change it when I get back to my PC after


Docs - Add Corrupted Files Page

Add a page to docs about corrupted files, why they occur, and what can be done to re-encode the tags

https://github.com/NickvisionApps/Tagger/releases/tag/2023.7.1-beta3


How can I limit one of the submodels of an ensemble to run on CPU only?

If I want to run a normal model on the CPU only, I can do this in the YAML configuration file:
profile_models:
  model_name:
    cpu_only: false

How can I achieve the same effect for one of the submodels in an ensemble model? I don't want all submodels to run on the CPU only but I want some of them to run on the CPU and others on the GPU.
Thanks.

@Alwahsh, that is a very good question. @nv-braf would be the person that knows for sure but he probably won't be available to reply until next week. I do see an internal ticket that implies this might not be supported yet.
May I ask are you running out of the 23.02 container? Or are you pulling from github?

Thank you!
I'm running out of 23.02 container but I had to change my configuration to use 23.01 version of Triton since the container of 23.02 doesn't seem to be out yet.
And I did manage to get the ensemble testing working. Thanks for providing it :pray:

This is currently not supported.

This is currently not supported.

Is cpu_only support planned? If so, when is it planned?

Yes, this will be supported as of 23.05 release, which should be by the end of the month (or you can grab it off of main). The new option is called --cpu-only-composing-models

@nv-braf There is no support for --cpu-only-composing-models in model_analyzer version 1.28?

Yes, there should be support for --cpu-only-composing-models in 1.28, which is the version in the 23.05 release.  Is it not working for you?

@tgerdesnv
I use model_analyzer locally and install it via pip:
pip install triton-model-analyzer

there is no functionality in the archive with --cpu-only-composing-models
[Model Analyzer] cpu_only_composing_models is not a valid yaml argument. Please be sure to use underscores and check spelling.
[Model Analyzer] Model Analyzer encountered an error: The Yaml configuration file is incorrect. Please check the logged errors for the specific options that are causing the issue.

But it works for me if you install through pip like this
pip install git+https://github.com/triton-inference-server/model_analyzer.git

You are correct. Unfortunately it appears that the feature didn't quite make it into 23.05.  As you observed, you can install off of main and it will work. Is that sufficient for you for now? It will definitely be in 23.06.

So-so, but I will wait. Thank you

This feature was released starting in 23.06. See documentation here.


Need a Live Preview of Not Yet Published Blog Posts

Right now the only way to see a blog post with all the site CSS applied is to publish it. It would be nice to have a way to see a post in full context before publishing.
There is a way to do that with Netlify, so we need to figure that out.

Preview builds are now enabled. I have updated the wiki page on blog posts accordingly.  Tagging @ProfYaffle just so he sees this (since I separated this out from another issue).

One other note. I re-saved the blog post about broken add-ons (I added a space at the end) so that it now has a preview.

Re-opening this. While the live preview looks great, there's a sync delay of some kind: I updated a draft post, saved it, and yet the preview still shows the old iteration.
It may be unavoidable, in which case some text on the preview button to remind people might be useful.
Perhaps unrelated, but - going back now to see if a few minutes has allowed the sync/build/whatever - I'm unable to even get a preview, instead getting a "Check for Preview" and refresh button that seem to do nothing.

That save got "caught" in a major rebuild, so it look almost 17 minutes to rebuild the preview.  Once we get things stable and in production, that shouldn't happen very often, but this time you saved that after I had pushed some changes that forced a full rebuild on the main site (which meant your separate temporary branch had to do the same thing).
I'll check to see if there's any way to customize what messages are shown when, but I think we may be stuck with what is there (although I can certainly submit an issue to the Netlify CMS github repo).  I'm getting ready to deploy some more changes that I'm hoping will speed build times, and that may help. But there is always going to be a 5 minute or so delay between saving a draft and getting a preview.

I stumbled on documentation on how to build custom live previews inside the CMS.  Once I stopped following the documentation and just started banging on things, I got a viable live preview done.  It's available to try in staging at:
http://kodi-tv-staging.netlify.app/admin
You probably shouldn't save any changes to the posts or add new ones, or those will get merged when we next merge staging into main. But you can at least edit exiting posts without saving and start new ones (again, without saving). The live preview is real time as you type, and it is exactly (well, 99% exactly) what you will see on the web site. There are two differences in the live preview.

The live preview shows the read time as 99 minutes. That is calculated by Gatsby at build time, so there is no way to access it in the live preview.
The live preview has no footer. We use a Gatsby static query to generate the footer list of sponsors, and, as with the read time, that query is run at build.  If the footer is included the live preview errors out.

There is an interesting bonus of this preview.  It is fully responsive. By resizing the preview pane you can see what the post will look like on other devices.  Here are some screenshots of that:
On a big screen with enough space for a full size preview:

On a laptop sized screen with enough space for mid-sized preview:

And one with the preview pane small enough that it emulates a mobile device:

Live preview for blog posts (and all other content types) are available in main now and published at https://kodi-tv.netlify.app.  I think this solves the problem much better than the incremental builds at save, so I have disabled those for now.  Unless we change it back, the site will only be rebuilt when a post is actually published.

Closing this then :)


Evaluation of Mozart

Very nice work, but I have a question in my mind. How should we evaluate the generated txt outputs, that is, with which method can we benchmark the OMR system?

We used the basic accuracy evaluation; The total number of predictions the model gets right considering that we had the true output of each testcase.


Add to CDNJS.com

Can you please add the distribution to cdnjs.com similar to every other react libraries. For example https://cdnjs.cloudflare.com/ajax/libs/react-relay/0.10.0/relay.min.js. Thanks in advance.

➤ ravilution commented:
I also tried using<EMAIL_ADDRESS>But there is a lot of dependency issues :-(


Revert "Implement minimal reflection probes. (#10057)"

Objective

Fix working on macOS, iOS, Android on main
Fixes #11281
Fixes #11282
Fixes #11283
Fixes #11299

Solution

Revert #10057

Fix for reflection probes should be pretty simple. I believe we just need to add a check if the render device supports the texture binding array feature, rather than a conditional cfg for non wasm platforms.


Document how to do pre-releases

this commit isn't signed off :)

@kensipe I better rush the merge before the DCO is enforced :D


updates

add automatic updates + changelogs

user can choose to ignore updates in config.json (when running bot, it wont show an update prompt)
on startup of the bot, git fetches from origin and if there's changes, prompt the user if they want to update the bot
if yes, update the bot with git and show the new changelogs from a changelog file

help wanted - dont know how to implement this in the best way, so if someone knows how updates are usually handled with github repo projects, please tell or feel free to implement this yourself.
wontfix - i'll do this maybe if the core elements of the project are done (database, dockerize project, more)

update checker in utils that runs on startup (await updatechecker before running bot / loading commands).
only run if user specified they want auto-updates. otherwise user needs to update manually themselves with git command.
util should console log the changes made.

Don't automatically update, to keep the bot stable. Already an update command with shell files


feat(templates): re-introduce external template management

Close #2088

This implementation is available via portainer/portainer:pr2119 (linux amd64 build only).


Fixes Partially #56

Fixes #56  . Partially
Changes proposed in this PR:

Homogenize Simple uses ETH data from Ocean, and linear models to mimic main2.py

FYI if you have "fixes #" in your PR, then github will automatically close the issue when you merge it into main. So I always use "towards #", which has no magic side effects from github upon merge, and is more compact. I'll update your PR accordingly now.


Data persists in DB after changing conditional field

Link to reproduction
https://github.com/payloadcms/public-demo/pull/25
To Reproduce
We should not store unwanted information in the collection after unchecking a conditional field.
Please check the following PR recently merged in the payload demo website that exemplifies this problem: https://github.com/payloadcms/public-demo/pull/25
An easy example to illustrate the problem can be accomplished with the following steps.

Create any conditional field. For example, a Show image? checkbox field that makes visible a field called media.
Add an image to the media field.
Make a call to the API to verify that the data is coming correctly.
Uncheck Show image? field to make the media field disappear.
Make a call to the API and you will notice that object with data of the media field is still present in the DB/api response.

Describe the Bug
We should not store unwanted information in the collection when the condition is not met.
It would be great to pass a configuration option wherever the condition is defined to enable this behavior.
Right now I am forced to do the following every time I define a condition in a component:
   admin: {
          condition: (_, siblingData) => siblingData?.showImage,
        },
        hooks: {
          beforeChange: [
            ({ siblingData, value }) => siblingData?.showImage  ? null : value),
          ],
        },

It would be great to have this out of the box, by doing something like:
   admin: {
          condition: (_, siblingData) => siblingData?.showImage',
        },
   removeValuesOnCondition: true

So whenever the condition is not true, it should not stored (reset) to the default empty value.
Payload Version
1.9.0

@tylandavis, you're right that we wouldn't want to run the condition on the backend as an extra step to then determine if a field should be unset or not.
I think the behavior needs to be baked into the field without adding additional config properties. If a user is editing a document on the admin panel and changes a conditional field so that another field that had a previous value is removed, then on save the property needs to be assigned null in the PATCH so that the backend can handle it without the need for a beforeChange hook or anything else.
Would that satisfy your concerns @creative-andrew?

That would be great and I think it should be the expected result. Namely, we shouldn't store extra data when a condition is not met and the user saves the document.


Add redirection to category pagination

I left some comments however this PR is reasonable :+1:

:cool:

:shipit:


Extension causes high cpu load

Issue Type: Performance
Extension Name: vscode-mocha-sidebar
Extension Version: 0.22.2
OS Version: Windows_NT x64 10.0.17134
VSCode version: 1.33.1

:warning: Make sure to attach this file from your home-directory:
:warning:C:\Users\midingoy\maty.vscode-mocha-sidebar-unresponsive.cpuprofile.txt
Find more details here: https://github.com/Microsoft/vscode/wiki/Explain:-extension-causes-high-cpu-load

hi I didn't saw any attached file BTW your link to vscode is incorrect
can you attach the file

Any answer or remedy for this issue?? I am facing the same

I Have the same issue and as I looked at Task Manager, I saw that the node processes  has around 25% CPU-usage.
Config:
"mocha.requires": [
"ts-node/register"
],
"mocha.files.glob": "test/**/*.test.ts",
"mocha.coverage": {
"enable": false
},
"mocha.parallelTests": 1


Docs-11836 download unified schema

@mauro-dentoni I left the instructions for downloading the schema with a 3rd-party API client, but if you want me to remove that I can. Wasn't sure if people would use it for automation or something

@IsaacEldridge I agree to leave both the unified schema download using the new button and the existing 3rd-party API client download. As you did, the new button instructions should go first, as it is our recommended method. Thanks!


newbranch

Added attention types
Added attention types.

Can we do several things:

Rename the files (there's no need for the JAX suffix, since this entire project is in jax)
Rename jax_attention folder to efficient_attention
Rename jax_sonic_mha to sonic_perf_lin_mha to make it more descriptive.
Can we also make each efficient attention mechanism its own folder within the efficient_attention folder?

Just trying to keep things a little neat since we're both working on this.
Lastly, you need to do an end-to-end test on the GPUs so that the entire thing works (Run pre-training for Perf/Lin/Sonic-Lin-Perf/Sonic-RFA-Lin, these are the important ones). Then we can go ahead and merge it.

Ok some changes we need to make to our efficient attention mechanisms (let's follow JAX's implementation and see if we get good performance. For now, only change SONIC-Perf-Lin (we will train this end-to-end and see if it performs well): https://flax.readthedocs.io/en/latest/_modules/flax/linen/attention.html#MultiHeadDotProductAttention):

Let's implement the linear transformations as Dense FFNs (see the link I posted). In doing so, we can use additional bias weights as well. It should be initialised with default_kernel_init from the library: flax.linen.linear.
Rather than doing a simple reshape at the end, feed it through another Dense FFN (see the link, line out = DenseGeneral( ...) again try to follow as closely to this as possible.

After you do this. Let's train SONIC-Perf-Lin end-to-end (for 120k steps pre-training, and the sweep on GLUE) and look at the results. If they are good, I think we are golden and can proceed to change the rest of the attention mechanisms as I said.


comment

This is Brillant

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: c646d8c7-4829-2567-c5c7-a9f526f7a569
Version Independent ID: 8ef7d393-8cdb-42b8-6db4-31633fdad102
Content: Touch and pen interaction terms - Microsoft Style Guide
Content Source: styleguide/a-z-word-list-term-collections/term-collections/touch-pen-interaction-terms.md
Product: non-product-specific
GitHub Login: @pallep
Microsoft Alias: pallep

Hey, thanks!
(I hope you're not being sarcastic ;)


fix: tailscale state issue

Description
If you with the current setup try to fetch a SSL cert with tailscale cert you get an error message around TailscaleVarRoot.
If you don't apply that state, it just perfectly works. It still even use the same directory.
Because of that issue also tailscale server https:8443 / HTTP://<IP_ADDRESS>:80 is also not working
Types of changes

[ ] Docs change / refactoring / dependency upgrade
[X] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Verification Process
try to fetch a cert with tailscale cert before and after that change.
You need to be in a directory that is writeable so for example /tmp/
Same apply to the tailscale serve
Release Notes

Contributing checklist

[ ] My code follows the code style of this project.
[X] I have read the CONTRIBUTING and LICENSE document.
[X] I fully agree to distribute my changes under Apache 2.0 license.

Sorry, but just out of your description I can't follow why removing the state argument would solve any problem. In fact putting the state variable to /etc/config was done on purpose so that after a reboot the state of tailscale is preserved. Furthermore, why does removing the state argument allow to use the cert' command while with state pointing to /etc/config` does not? So please elaborate a bit more and perhaps with as much technical details as possible.

@jens-maus thanks for checkin and feedback.

yes you are fully correct that remove does not work because the normal statedir /var/lib/tailscale is in the tmpfs area
adapted the PR and now it works just perfect on my side.

The main issue is if you just point to the state, tailscale has an issue with the statedir, maybe it points somewhere that is only readonly or don't have it.
Now with tailscale just having a proper statedir, it stores the state and also the generated certs,... in that dir.
If you want to test it for your self, just run:
tailscale serve https / http://<IP_ADDRESS>:80
and after that you have even perfect access via a nice valid SSL cert to your CCU from your tailnet.
This need further improvement before I can make a PR.

Thanks for your revised PR. However, I do have some more questions before I can consider accepting it.

According to the docs (https://github.com/tailscale/tailscale/blob/main/cmd/tailscaled/tailscaled.go#L168-L169) the -statdir parameter should point to an actualy directory and not to an actual state file. However, you PR introduces -statedir with a value that seems to point to a file.
In addition, the docs (https://github.com/tailscale/tailscale/blob/main/cmd/tailscaled/tailscaled.go#L168-L169) also state that if -statedir is used the default state file name is <STATEDIR>/tailscaled.state.

So IMHO it seems to look like that using -statedir /etc/config should be enough for the state, certs, etc. to correctly be created in the default /etc/config directory path where all other persistent config files are usually stored.
Or would it be even better/wiser to use -statedir /etc/config/tailscale to make sure that all necessary files are created in that particular subfolder and not directly in /etc/config? In that case I would propose to use
TAILSCALED_ARGS="-state /etc/config/${DAEMON}.state -statedir /etc/config/tailscale -socket /var/run/tailscale/${DAEMON}.sock"

That would make sure that the location of the state file is still the same like before, but the -statedir points to the /etc/config/tailscale directory path for additional files (SSL certs, drop files, etc...). If this is the case one would have to – however – hceck that /etc/config/tailscale will be automatically generated and that we don't need to create it in advance upon tailscaled startup.

@jens-maus I will change it in 5min to /etc/config/tailscale
As you can see in the statedir are multiple files and I think they should not pollute the /etc/config directory

The second thing is the service automatically creates the statedir if it does not exists with the following access rights:

@jens-maus I will change it in 5min to /etc/config/tailscaled/

Thx. In addition, I still think it would be good to also keep -state /etc/config/${DAEMON}.state to keep the *.state file outside of the /etc/config/tailscaled directory path so that existing installations keep on working since the previous state file naming / location was directory in /etc/config.

@jens-maus I will try that topic out and give you feedback. From "cleanless", I would more prefer to move the state file into that folder, because if you don't define the --state flag all files are always together in a folder.
We could add a "migration" script that just moves the state file into the folder, but this just creates more stuff in the init script, so this is also not nice. And I also understand that it make sense to be backward compatible.
I will share  my outcome if that works with both later.

@jens-maus I will try that topic out and give you feedback. From "cleanless", I would more prefer to move the state file into that folder, because if you don't define the --state flag all files are always together in a folder. We could add a "migration" script that just moves the state file into the folder, but this just creates more stuff in the init script, so this is also not nice. And I also understand that it make sense to be backward compatible.

Backward compatibility is IMHO more important here than "cleanliness". Especially since we are talking about just a single state file that will also be located in the /etc/config top-level directory path. So please apply my suggestion accordingly and refrain from adding any migration method to the init script, etc.

Thx, this was quick and I will merge this PR now. Thanks for your contribution!

@jens-maus I double checked it and it works fine.


[FP] When documentInternalElements is false in stylecop.json and it sees internal interface type.

I get warning SA1600 when the following code is seen in a project of mine at least within Visual Studio:
internal interface IBObject
{
}

Expected: No warning
Result: Warning is issued.

DocumentInterfaces=true takes precedence over DocumentInternalElements=false, so this will happen if DocumentInterfaces is not set to false. Is that the case for you?

DocumentInterfaces=true takes precedence over DocumentInternalElements=false, so this will happen if DocumentInterfaces is not set to false. Is that the case for you?

That is not in the json so I am guessing that it is by default true. I think in that case DocumentInterfaces should consider the internal elements though and should skip the internal ones if the DocumentInternalElements is false though.

Duplicate of #3100

The current implementation makes it possible to at least require documentation for interfaces, which seems reasonable to me. If you don't want to treat interfaces specially, then you can set DocumentInterfaces to false.
Which elements would you like it to require documentation for?

public interfaces only.


feat(call-home): added collector components

Signed-off-by: Vivek Kumar<EMAIL_ADDRESS>This PR adds metrics collector components for call-home. This requires a transmitter to work correctly.

@GlennBullingham -- @datacore-vkumar and I(@niladrih) will require permissions to be able to assign reviewers. Thanks.

try bors

bors try

bors try

try
Build failed:
* [continuous-integration/jenkins/branch](https://mayastor-ci.mayadata.io/job/Mayastor%20Extensions/job/trying/5/display/redirect)

@datacore-vkumar , you'll need to ignore the openapi, as seen here: https://github.com/openebs/mayastor-control-plane/blob/develop/.rustfmt.toml

bors try

bors try

bors try

bors try

bors merge

bors merge

bors merge

@datacore-vkumar in the future before the merge please squash commits such as fix(call-home): addressed review comments, and rebase from develop instead of merge.


Update to electron 1.4.x

Frameworks

V8 (52 => 53)
Chrome (52 => 53)
node.js (6.5.0 => no change)
libuv (1.9.1 => no change)

Fixes

[x] Accessibility: Selected tree item not read from screen reader (https://github.com/Microsoft/vscode/issues/13104)
[x] Crash when closing window with custom title (https://github.com/Microsoft/vscode/issues/15154)
[x] Windows: no output from CLI causing us to use cat.exe as workaround
[x] Unable to save CPU profiles (https://github.com/Microsoft/vscode/issues/13704)
[x] New fullscreen windows opens as tabs (https://github.com/Microsoft/vscode/issues/13550)
[x] Folder path truncated after resizing window (https://github.com/Microsoft/vscode/issues/14558)

Features

[ ] Can move webviews around (docs)
[x] Can leverage newly introduced ELECTRON_NO_ASAR environment variable for our processes
[ ] Could enable HC theme by default when detected in Windows (https://github.com/Microsoft/vscode/issues/15360)
New net module to run HTTP(s) requests through Chrome (docs)

Bugs

[x] Drag image looks completely broken now from the tree
[x] Need to verify AA works properly across OS

It's coming!

It is probably impossible to deny a request from the best closer this city has ever seen 👍

also has the macOS pasteboard API :)
https://github.com/electron/electron/pull/7719


Alias for icon send

Sorry but this issue is not about a new icon or bug... I just want know is it a good idea that add an alias submit for send?

BTW, how about add an alias wait for loader ?

@PeterlitsZo Welcome to the Lucide community. Thanks for this idea, I think that's a great idea, more tags helps searching icons.
Feel free to add those tags in the tags.json!


OpenAPI V2 documents do not properly render anyOf including nullable references

https://github.com/microsoft/OpenAPI.NET.OData/issues/33
https://github.com/microsoft/OpenAPI.NET.OData/issues/35
The following OpenAPI description does not render the complex object properly in V2
openapi: 3.0.0
info:
  title: asdads
  version: 1.0.0
paths:
  /:
   get:
     responses:
       200:
         description: ok
         content:
           application/json:
             schema:
               type: object
               properties:
                 complex:
                   anyOf:
                     - $ref: '#/components/schemas/complex'
                     - nullable: true
components:
  schemas:
    complex:
      type: object
      properties:
        name:
          type: string

@darrelmiller - not sure if this is related, but it looks like all sibling properties do not get serialized when $ref
"definitions": {
  "AResource": {
    "description": "ARM tracked top level resource.",
    "type": "object",
    "properties": {
      "id": {
        "description": "Resource ID.",
        "type": "string",
        "readOnly": true
      },
      "name": {
        "description": "Resource name.",
        "type": "string",
        "readOnly": true
      },
      "type": {
        "description": "Resource type.",
        "type": "string",
        "readOnly": true
      },
      "location": {
        "description": "Resource location.",
        "type": "string",
        "x-ms-mutability": [
          "read",
          "create"
        ]
      },
      "tags": {
        "description": "Resource tags.",
        "type": "object",
        "additionalProperties": {
          "type": "string"
        }
      },
      "properties": {
        "$ref": "#/definitions/AResourceProperties"
===================NO OTHER PROPERTIES=====================
      }
    },
    "x-ms-azure-resource": true
  },
  "AResourceProperties": {
    "description": "Resource Properties",
    "type": "object",
    "properties": {
      "prop1": {
        "type": "string"
      },
      "prop2": {
        "description": "prop2 description",
        "type": "string"
      }
    }
  }
}

My OpenAPI schema has description and extensions (see the 'NO OTHER PROPERTIES' section), but those aren't generated. Is this related?

From the OpenAPI 3.0.0 specification
This object cannot be extended with additional properties and any properties added SHALL be ignored.

https://spec.openapis.org/oas/v3.0.3#fixed-fields-18


python310Packages.rpyc: disable failing test

Motivation for this change
Fix build (https://hydra.nixos.org/build/163433233)
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 154678 run on x86_64-linux 1

11 packages built:

flexget
pwntools (python39Packages.pwntools)
python310Packages.angr
python310Packages.angrcli
python310Packages.angrop
python310Packages.pwntools
python310Packages.rpyc
python39Packages.angr
python39Packages.angrcli
python39Packages.angrop
python39Packages.rpyc


一直连接失败，回调onConnectFailure()

Whoops!Lost connection to http://localhost:8090/ircloud

The server needs to add this
    @Override     public void registerStompEndpoints(StompEndpointRegistry registry) {         registry.addEndpoint("/ircloud").**setAllowedOrigins("*")**.withSockJS();     }


Style/DateTime tweaks

As per the discussion in #6189, I'm proposing my alternative. Even if closed, it could still be useful for people who want to keep this code as a custom cop, so I'm opening the PR anways :)

Before submitting the PR make sure the following are checked:

[x] Wrote good commit messages.
[ ] Commit message starts with [Fix #issue-number] (if the related issue exists).
[x] Feature branch is up-to-date with master (if not - rebase it).
[x] Squashed related commits together.
[ ] Added tests.
[x] Added an entry to the Changelog if the new code introduces user-observable changes. See changelog entry format.
[x] The PR relates to only one subject with a clear title
and description in grammatically correct, complete sentences.
[x] Run rake default or rake parallel. It executes all tests and RuboCop for itself, and generates the documentation.

CI is green now after a rebase.

I think you can also expand a bit the rationale section of the cop and also add there was this is disabled by default.

@bbatsov I tried to improve the docs, not sure if it's what you had in mind.

This cop is enabled again in 0.60.0.

Yup, sounds like a bad rebase of https://github.com/rubocop-hq/rubocop/pull/6293.

Yup, sounds like a bad rebase of #6293.

I'm trying to resolve this in #6451.


27 standarize get transaction

Description
Get Transaction method has been standardized with the following structure
{
    "tranction":{
         "blockHash": "XXX",
         "fee": yyy,
         "data": {
             ...
         },
         "transfers":[
              {
                   "from":"XXX",
                   "to":"XXX",
                   "amount": yyy,
                   "fee":yyy
              }
        ]
    }
}

Where:

blockHash: Block hash where the transaction is stored
fee: Transaction fee
data: Info about the transaction. This filed contains the blockchain transaction info
transfers: Collection of transfers done in the transaction where

from: Origin address
to: Receipt address
amount: Amount of corruncy transfer
fee: Transfer fee

Fixes #27
Dependencies (if any)

Type of change

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[x] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?

Get Transaction Method test updated for ETH in test_eth.py
Get Transaction Method test updated for BTC in test_btc.py

Test Configuration:
Operating system (output of sw_vers): macOS 12.0.1
Related PR or Docs PR

Docs PR related #
Other PR related #
Good practices to consider

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[ ] Any dependent changes have been merged and published in downstream modules

This PR should be reviewed by @bridgedragon


Utility Notes

Utility AI
The principal around this is to be able to describe these things in an clear and expandable way (hopefully generic)
In the majority of cases theres a few identifiers that prescribe the most value

How much value the action has (damage, healing)
Will the action reduce enemy count
The speed of the action (improvement for later - see bottom) ie: BasicAttack faster than Ultimate or Sunball

questions:

how to value weight a decisive action like killing the boss? just improve its value by alot?
is killing two enemies better than breaking a lock?
how to we value certain locks for enemies?

appraisals: (base)
basic attack
skills
combos
ultimates
items
modifiers (multi)
damage modifiers
damage modifiers override
controller specific actions? (ChromaticApparation)
filters (remove)
live mana count
dead status
mp status
hp status (redbar means heal, maybe in reasoner context)
target (priority)
memory knowledge based modifiers (multi?)
will action prevent an enemy turn?
will action delay an enemy turn?
will damage kill enemy
will damage end the fight
will aoe damage kill enemy
will aoe damage end the fight
questions:

since damage has a range (rand 0-3 on base) do we even consider worst case at all times? future improvement to include a % chance of killing with range?

fight specific modifiers
damage phases - some way to set in the controller than the enemy is taking increased damage?

this may just be temporary damage modifiers overrides
can probably say if damage_modifiers_overrides.len() > 0 in the above case
holding ult
holding combo
holding live mana?

how do value overcapping (will_attack_overcap_live_mana) that type of thing
does it make sense to spend 1 on the highest appraisal if possible to avoid overcaps? configurable?

fight specific mechanics

maybe hash enemy guids into a value we can use for unique encounter ids
identify uuid (enemies have unique ids as well as guids) in list to determine reasoner execution context.

priority targets
questions:

does this take a reasoner execution context and modify it based on values during the fight
should we make this an fsm for complex boss fights?

Utility Helper
have live configurable values for utilty/reasoner execution context and a button to regenerate on the fly so these values can be tested at runtime
Enemy Improvements (Later)
Enemy Utility AI

Checking enemy attacks and calculating enemy damage while things are casting?

Note: we'll likely need to describe these in code anyways for the timed hit handlers and block button presses
unless i can find a way to determine this properly in realtime.

will enemy damage kill a player (casting check or basic attack check if turns left is a value)

if so, use healing item?

Skill casting time (Later)
When deciding an action, we should account for the time it takes an action to complete. Its far better to collect this information during actual runs instead of manually timing things

for this we can log out a parsable format that includes the action name and the time it went from execution to ready again; there will be variance due to jump in (which we may be able to consider)

q: how to we not include logs that kill enemies to prevent upper variance?

struct Controller {
    reasoner_execution_context: Option<ReasonerExecutionContext>
}

struct ReasonerExecutionContext {
    filters: Vec<ContextFilter>,
    modifiers: ContextModifiers,
    settings: ContextSettings,
}

// This can be generic
trait ContextFilter {
    fn valid(action: Action, context: ReasonerExecutionContext) -> bool;
}

impl ContextFilter for LiveManaFilter {
    fn valid(action: SoSAction, context: ReasonerExecutionContext) -> bool {
        !context.settings.hold_live_mana &&
        actions.appraisal.player.live_mana >= action.appraisal.live_mana_cost
    }
}

impl ContextFilter for ManaFilter {
    fn valid(action: SoSAction, context: ReasonerExecutionContext) -> bool {
        !context.settings.hold_mp &&
        actions.appraisal.player.mp >= action.appraisal.mp_cost
    }
}

impl ContextFilter for DeadFilter {
    fn valid(action: SoSAction, context: ReasonerExecutionContext) -> bool {
        !action.appraisal.player.is_dead
    }
}

struct ContextSettings {
    hold_live_mana: bool,
    hold_mp: bool,
}

some pseudo code for filters maybe

Another thought:
I like generating all the possible actions, then  filter, then maybe run a calculate function to actually calculate scores for efficiency.

Doing it that way with the filters is one way. Another is to not generate things that are invalid moves (moving the "possibility" check to when creating the Appraisals). I think we did it that way in the old TAS.

My thought is (and its probably short sighted), is that using filters like this instead of doing it in appraisal generation can help avoid contaminating the core utility ai stuff with assumptions about Sea of Stars things.
I'll try thinking through things from that angle as well...

Ensure spell locks are calculated in Combat AI


Move kernel/soc/arch/drivers to new logger

Still pending fix for #10026

this obsoletes #9358 #9361 #9359

also obsoletes #9428 and #9425

obsoletes #9429 as well

@nashif : isn't the new logger a bit immature still?

no, afaik it is ready to go and we need to move everyone to the new logger in 1.14.

@nashif rather than duplicate my work (my pull-requests were clearly marked with the appropriate tag - see: https://github.com/zephyrproject-rtos/zephyr/labels/area%3A Logging) you could have cherry-picked my patches. There was not much work left as I was waiting for the v1.13 release and the pull-requests logging: add template for log configuration and log: make name param explicit to be merged to clean my pull-requests.
You might actually have cherry-picked my patches but remove my credit (see https://github.com/zephyrproject-rtos/zephyr/pull/9429#issuecomment-422175587)...
If you are not aware there is actually a command named git cherry-pick for that purpose.
I did this boring work to help the Open Source project to have the logging subsystem to be merged into v1.13 as it was initially planned and also to not need my other pull-request that I think is needed in v1.13 without the new logging subsystem.

@Olivier-ProGlove

@nashif rather than duplicate my work

I was not duplicating your work, the changes I have done were inclusive and covered everything in the tree, not selected architectures or drivers. Many of you PRs were stale and needed attention and I did not want to cherrypick something that was not ready and that could cause confusion on my part due to the old state they are in.

I used the new API for LOG_MODULE_DECLARE
I used the Kconfig template
I covered ALL architectures and files
I applied a mew generic domain scheme
...

I apologise if that caused any discomfort or confusion, that was not the intention.
Thank you for you contributions and we value your input.

Codecov Report

Merging #10029 into master will increase coverage by <.01%.
The diff coverage is 7.14%.

@@            Coverage Diff             @@
##           master   #10029      +/-   ##
==========================================
+ Coverage   52.58%   52.58%   +<.01%
==========================================
  Files         213      213
  Lines       26148    26150       +2
  Branches     5636     5636
==========================================
+ Hits        13749    13752       +3
+ Misses      10145    10144       -1
  Partials     2254     2254

Impacted Files
Coverage Δ

kernel/init.c
72.58% <ø> (ø)
:arrow_up:

drivers/crypto/crypto_tc_shim.c
58.08% <0%> (-0.44%)
:arrow_down:

drivers/crypto/crypto_mtls_shim.c
67.1% <22.22%> (+0.43%)
:arrow_up:

subsys/logging/log_core.c
71.18% <0%> (+1.12%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 780c1c2...981987e. Read the comment docs.

@nordic-krch any idea why samples/drivers/crypto is failing here?

@jarz-nordic @nordic-krch @Olivier-ProGlove review please


GPII-3327: Add gcp-stackdriver-exclusion.

Requires https://github.com/gpii-ops/exekube/pull/12.
If anyone has pointers on how to make the filter more readable by spreading it across multiple lines, let me know :).
I manually verified that the relevant kubelet log lines are no longer added to Stackdriver. Stackdriver also provides stats on how much is excluded. I wonder if we can / should have automated tests to verify that exclusion rules are working correctly?
I started with one module per exclusion. It is simple, but might not scale very well. I think I'll switch to one module that accepts a Map (or List of Maps) of exclusion rules and manages them all together. Does this sound right?

LGTM,
I didn't found anything useful to split the lines. It seems that there are some issues open related to such functionality.
https://github.com/hashicorp/hcl/issues/211
https://github.com/hashicorp/terraform/issues/8210
https://github.com/hashicorp/hcl/issues/144

@natarajaya

Some of the exclusions will stop working if we decide to use Stackdriver Beta.

Is this a note for the future, or do you think we should target Stackdriver Beta now? What do I need to do to enable the Beta-style naming -- do you have code that does this for your monitoring work that you can share?

@stepanstipl

I would consider using a bit more generic filters, like on severity basis for example.

I did a single calico rule to make sure I could match on that log correctly. I was planning to add additional rules for other useless lines.
I thought throwing away all INFO-level logs would be too aggressive, but maybe that was wrong. I'll take another look at what's in Stackdriver.

Thanks everyone for your comments! Sorry for the delay in replying to them. I think I've got them all now.
The next batch of commits is mostly restructuring to use one module accepting a map. See also related update to https://github.com/gpii-ops/exekube/pull/12.

I prefer this way of supporting multiple entries via map. 👍

LGTM and +1 for using maps of exclusions instead of one module per exclusion.

@stepanstipl

I would consider using a bit more generic filters, like on severity basis for example. Taking the calico one and changing it to following:
resource.type="container"
resource.labels.cluster_name="k8s-cluster"
resource.labels.namespace_id="kube-system"
logName:/logs/calico-typha
textPayload:[INFO]

I don't think it will help performance if we have additional filters on cluster_name or namespace. Also, we want to exclude these logs even if we add (or change the name of) a k8s cluster to each environment (e.g. for vault or locust) or if Calico moves to a different namespace. Am I missing anything?
As discussed here, I'm open to but uncertain about using container_name instead of logName.

seems to return lot of "crap" I would probably get all rid of.
vs
NOT textPayload:[INFO]

stil returns a lot of suddently much more interesting stuff.

Looking at Calico logs minus all INFO-level logs returns nothing for the cluster I spun up this morning. There are ~25 lines from other clusters I've run over the past month or so, but these are the only non-INFO Calico logs I see:
Unable to decode an event from the watch stream: http2: server sent GOAWAY and closed the connection; LastStreamID=40009, ErrCode=NO_ERROR, debug=""
Anyway, my concern about throwing away all INFO-level logs from Calico is what happens when we have a weird networking problem to debug and now we have basically no Stackdriver logs from the networking layer.
Some possible mitigations:

A "weird networking problem" may produce logs above INFO-level that help us see what is happening
Calico logs won't be in Stackdriver, but they will be available (for a limited time, as long as the Node where Calico was running doesn't disappear) via something like kubectl -n kube-system logs -l k8s-app=calico-typha.
"Weird networking problems" at the Calico layer may be so weird that we'll quickly give up trying to debug them, blame Google, and/or switch to Flannel or Weave :p.

I would appreciate additional input on this question. Perhaps it warrants a Parking Lot item.

It seems that the severity (at least in Calico case is there, just not parsed correctly).

Severity works for me, including inequalities like severity<=INFO.

@mrtyler Regarding Stackdriver Beta naming changes – I think we should leave this out of scope of this PR for now. I made a note for myself to update resource names in exclusions if this merges first.

@mrtyler

I don't think it will help performance if we have additional filters on cluster_name or namespace. Also, we want to exclude these logs even if we add (or change the name of) a k8s cluster to each environment (e.g. for vault or locust) or if Calico moves to a different namespace. Am I missing anything?
As discussed here, I'm open to but uncertain about using container_name instead of logName.

Please don't focus on individual specific filters in the example, this was about the moving away from prefix matched logName. I agree with leaving out cluster name is probably good idea, I would probably keep in namespace, but no strong opinion here.

seems to return lot of "crap" I would probably get all rid of.
vs
NOT textPayload:[INFO]

stil returns a lot of suddently much more interesting stuff.

Looking at Calico logs minus all INFO-level logs returns nothing for the cluster I spun up this morning. There are ~25 lines from other clusters I've run over the past month or so, but these are the only non-INFO Calico logs I see:
Unable to decode an event from the watch stream: http2: server sent GOAWAY and closed the connection; LastStreamID=40009, ErrCode=NO_ERROR, debug=""
Anyway, my concern about throwing away all INFO-level logs from Calico is what happens when we have a weird networking problem to debug and now we have basically no Stackdriver logs from the networking layer.
Some possible mitigations:

A "weird networking problem" may produce logs above INFO-level that help us see what is happening
Calico logs won't be in Stackdriver, but they will be available (for a limited time, as long as the Node where Calico was running doesn't disappear) via something like kubectl -n kube-system logs -l k8s-app=calico-typha.
"Weird networking problems" at the Calico layer may be so weird that we'll quickly give up trying to debug them, blame Google, and/or switch to Flannel or Weave :p.

I would appreciate additional input on this question. Perhaps it warrants a Parking Lot item.

It seems that the severity (at least in Calico case is there, just not parsed correctly).

Severity works for me, including inequalities like severity<=INFO.

Beware of difference between NOT textPayload:[INFO] vs severity<=INFO filters - this gives very different results. The Severity, in case of Calico logs, is there, but in the textPayload and not correctly propagated into the severity field.

@natarajaya

Regarding Stackdriver Beta naming changes – I think we should leave this out of scope of this PR for now. I made a note for myself to update resource names in exclusions if this merges first.

Great, thanks.
@stepanstipl

moving away from prefix matched logName

Ok. I have added exact matches on logName everywhere (for consistency with kubelet logs, which have logName but not container_name).

Beware of difference between NOT textPayload:[INFO] vs severity<=INFO filters - this gives very different results.

Aha! And 😬 !
 severity:  "INFO"
 textPayload:  "2018-09-20 15:42:15.942 [FATAL][5] daemon.go 305: Received SIGTERM, shutting down

I will use textPayload instead of severity, as you recommended. Thanks for following up on this.
@mrtyler

my concern about throwing away all INFO-level logs from Calico is what happens when we have a weird networking problem to debug and now we have basically no Stackdriver logs from the networking layer.

Some possible mitigations:

Thinking on this more (and with no additional comments from the team), I will throw out anything from Calico marked [INFO].

With three targeted and one broad exclusion, the Stackdriver stream for my dev cluster looks pretty good! I created https://issues.gpii.net/browse/GPII-3397 for the last noisy thing I see, a couchdb error that looks like a misconfigured health or liveness check. I'll check on the log again in the morning.
Note for interdiffers: there is a commit where I moved some things around to prepare for adding export rules in GPII-3324. You may wish to exclude that one.
Pending feedback, I think this work is ready for merge.
(There is also an update to https://github.com/gpii-ops/exekube/pull/12)

Looks good.
I'm wondering if the actual rules definitions shouldn't move to gcp/modules/gcp-stackdriver-exclusion/main.tf? Instead of having to repeat them per environment.
Personally I don't think this should be different per env (and if, there's always option to override the default).

I'm wondering if the actual rules definitions shouldn't move to gcp/modules/gcp-stackdriver-exclusion/main.tf? Instead of having to repeat them per environment.

Good idea.

@mrtyler @stepanstipl I took me a while to go thru the discussion and evaluate all proposed options, but looks like I found another "hybrid" alternative :)
It still uses magic __var.project_id__ string in variable declaration, because Terraform does not allow interpolations in defaults, but then it does substitutions in the same module using template_file resource.
Similarly to Stepan's alternative, mine handles all substitution magic within same gpii-infra's module, which means we don't have to carry it into exekube's module. But, IMO, it is much easier to read and understand than hardcore magic with Terraform locals.

@natarajaya Thanks for the alternative. It looks like this is the same as the first string replacement strategy, except it has an extra step (template_file). I don't see what the extra step gets us. Am I missing something?
Another disadvantage of the hybrid approach is it puts the burden for __project_id__ special handling on the user of the module. I think other exekube projects that adopted gcp-stackdriver-* will have the same need that we do for injecting ${var.project_id} into their filters at runtime. Am I off-base here?
I'm happy to have more discussion on this but my vote is still for Remain... with the current string replacement solution. :)

@mrtyler Ok, I thought what we wanted here, beside more readable code on gpii-infra's end, is to make corresponding exekube code more dry :D
I understand the point about shifting burden on the users of the module, but at the same time – maybe it is the right thing to do and we should not think for them, there may be different circumstances and different needs...
Also, looks like while we having one GKE project per Stackdriver account, there is no need to specify project_id in exclusions. I tried to use substring operator to filter kubelet-healthz-200 logs, and it worked just fine.
Probably worth another discussion?

I understand the point about shifting burden on the users of the module, but at the same time – maybe it is the right thing to do and we should not think for them, there may be different circumstances and different needs...

Unless users need to use the literal string __var.project_id__ in their filters, they will be unaffected by the string replace feature.

Also, looks like while we having one GKE project per Stackdriver account, there is no need to specify project_id in exclusions. I tried to use substring operator to filter kubelet-healthz-200 logs, and it worked just fine.

Discussed in https://github.com/gpii-ops/gpii-infra/pull/114#discussion_r217734836.

Discussed in #114 (comment).
Ok, in this case LGTM! 👍

Maybe I wasn't clear before, but I have a bigger problem with where the replacement is done, rather than the replacement method itself.
@natarajaya thansk for this, I like to see varous ways to go around TF's HCL language limitations :)
I see this as two different things:

Where we do the replacement: I'm much more in favour to leave any magic in our modules and don't push it down to exekube's modules. Personally I hate when something does some non-standard magic that I don't expect (for many reasones - difficult to debug, unexpected behaviour, my use case typically differs to one of others).

How we do the replacement: I still slightly prefer not using the __var.project_id__, but if it stays in our module, it's leass of an issue imho.

I wrote up conclusions from our last discussion:
https://github.com/gpii-ops/exekube/pull/12#issuecomment-425732814


Remove error of incorrect use of vkEnumeratePhysicalDevices

The validation layer throws an error if vkEnumeratePhysicalDevices is called w/non-NULL pPhysicalDevices if pPhysicalDeviceCount is not been previously retrieved by a previous call to the same function.
I don't think there's any justification for this in the spec. It may be good practice, but the spec allows you use vkEnumeratePhysicalDevices to retrieve a subset of the devices (and that VK_INCOMPLETE is returned if you didn't retrieve them all) so I don't think the validation layer should complain about this.
(See https://www.khronos.org/registry/vulkan/specs/1.0/xhtml/vkspec.html#devsandqueues-physical-device-enumeration)

Agree that it doesn't need to be an error. Main concern with that check was user requesting devices when non were available, but as spec indicates that case should be handled correctly by loader and  VK_INCOMPLETE returned.
My initial thought is to keep the message but demote it to WARNING, as inexperienced users may just enumerate 1 device without querying the count, and the warning is a good indicator that they may want to actually query the count and handle cases in their app with more (and less) than 1 device.

OK. I'm fine with just demoting it to a warning. I can update the MR.

Fixed by PR #632.


[Feature request] RSS/Atom feed

Hello,
would you consider adding RSS/Atom feed to the site, to track new and updated games?
Thanks!

Any reaction?

It's nice idea. If I have time I will give it a try. Any external contribution in this regard is of course welcome.

Nothing happened, looks iomprobable, closing.


UI font size and assignment is inconsistent

Certain UI elements have varied font sizes and font assignment on different platforms, and just in general.
Incomplete list of examples:

On Linux, the menu bar and context menu use the system font under KDE, UI font under GNOME.
On macOS, the find dialog labels use the system font and the combo boxes use Open Sans
On macOS, the status bar, dock widget titles, and most pop up dialogs use the system font instead of Open Sans
On Windows and Linux, the button font /might/ be larger? Need to confirm.

I need to come back and update this once I've done some more observation.
Somewhat related: #2422

Should be fixed as of ~2.3.2810.


Support overriden configuration in ckpmanager

with this extra CLI argument added to Checkpoint Manager, it becomes possible to override Checkpoint Manager configuration via -D system properties that are specified to Heron API server process during deployment time.
A useful use case is to switch heron.statefulstorage.classname to a different stateful storage backend, such as DlogStorage when deployed on Kubernetes cluster, without having to build customized Heron docker image.

@nwangtw


Getting errors because typings aren't used

Sometimes I get errors when opening a definitions file and closing it. Wether the definitions are @typings or custom.
Closing and reopening one file makes the errors disappear again for all the open files.
I think it also happens when editing regular .ts files
I'm using typescript_tsdk with version 2.4.1. I don't see any errors on the console.

Seems it's the same problem as in #594. Only the imports paths seem to stop working sometimes.


Grant launch permission for multiple recent AMIs

🗣 Description
This pull request modifies the post-Packer code to grant launch permission for several of the most recent AMIs.  Previously launch permission was granted only for the most recent AMI.
This entails:

Adding an input variable to specify X, the number of most-recent AMIs.
Modifying main.tf to grant launch permission for the X most recent AMIs.
Modifying the output to return the resulting array of cisagov/ami-launch-permission-tf-module objects.

💭 Motivation and context
This is a necessary change because assessments can go on for several months, and cisagov/guacscanner needs to be able to view AMI information in order to create Guacamole connections.  If Guacamole is re-deployed mid-assessment, then cisagov/guacscanner may need to re-create connections for old AMIs.
It is important to grant full launch permissions for these historical AMIs, since some assessment types will want to launch additional instances using the same AMI from which their original instances were launched.
Resolves cisagov/cool-system-internal#47.
🧪 Testing
I applied the new code to the staging Terraform workspace and verified that it created the expected resources.
✅ Checklist

[x] This PR has an informative and human-readable title.
[x] Changes are limited to a single goal - eschew scope creep!
[ ] All future TODOs are captured in issues, which are referenced in code comments.
[x] All relevant type-of-change labels have been added.
[x] I have read the CONTRIBUTING document.
[x] These code changes follow cisagov code standards.
[x] All new and existing tests pass.

I am requesting to merge this PR outside of our usual Kraken run since it is necessary for the current UAT exercise.


Enable easy S3 access

Lots of people have trouble accessing S3 from their Flintrock clusters.
#90, which is about accessing S3 from Flintrock clusters, is the most visited issue on this project. A related issue, #88, which is driven by the same problem, is the second-most visited issue on this project. Two recent guides that go over how to use Flintrock -- this one and this one -- take time to address the same issue.
This PR attempts to address this common problem by 1) setting better defaults that enable Spark on Flintrock clusters to seamlessly access data on S3, and 2) by providing instructions in the README on how to make use of these new defaults.
I tested this PR by launching several clusters in a variety of configurations. I was able to seamlessly access S3 in all cases.
It seems to be working well, but I would like to get some feedback from people who have hit this issue in the past to make sure I'm headed in the right direction here:

cc @PiercingDan and @AlexIoannides, who recently authored some guides that cover this issue in Flintrock.
cc @marcuscollins, @hhbyyh, and @ereed-tesla, who reported this issue or discussed it in #90 and #88.

I know this PR may be too late for some of you, since you may have moved on or come up with your own workaround. So no hard feelings if you are not interested. And of course, if anyone else reading this would like to chime in with their feedback that would also be helpful.
If you would like to install Flintrock directly from this PR (assuming you are running Python 3.4+), you can do that with this:
pip install git+https://github.com/nchammas/flintrock@easy-s3-access

Fixes #90.

Looks good to me 👍

Hello,
Sounds great - will take a look on the weekend!
Alex

On 25 Jan 2017, at 02:25, Nicholas Chammas<EMAIL_ADDRESS>wrote:
Lots of people have trouble accessing S3 from their Flintrock clusters.
#90 https://github.com/nchammas/flintrock/issues/90, which is about accessing S3 from Flintrock clusters, is the most visited issue on this project. A related issue, #88 https://github.com/nchammas/flintrock/issues/88, which is driven by the same problem, is the second-most visited issue on this project. Two recent guides that go over how to use Flintrock -- this one https://github.com/PiercingDan/spark-Jupyter-AWS/ and this one https://alexioannides.com/2016/08/18/building-a-data-science-platform-for-rd-part-2-deploying-spark-on-aws-using-flintrock/ -- take time to address the same issue.
This PR attempts to address this common problem by 1) setting better defaults that enable Spark on Flintrock clusters to seamlessly access data on S3, and 2) by providing instructions in the README on how to make use of these new defaults.
I tested this PR by launching several clusters in a variety of configurations. I was able to seamlessly access S3 in all cases.
It seems to be working well, but I would like to get some feedback from people who have hit this issue in the past to make sure I'm headed in the right direction here:
cc @PiercingDan https://github.com/PiercingDan and @AlexIoannides https://github.com/AlexIoannides, who recently authored some guides that cover this issue in Flintrock.
cc @marcuscollins https://github.com/marcuscollins, @hhbyyh https://github.com/hhbyyh, and @ereed-tesla https://github.com/ereed-tesla, who reported this issue or discussed it in #90 https://github.com/nchammas/flintrock/issues/90 and #88 https://github.com/nchammas/flintrock/issues/88.
I know this PR may be too late for some of you, since you may have moved on or come up with your own workaround. So no hard feelings if you are not interested. And of course, if anyone else reading this would like to chime in with their feedback that would also be helpful.
If you would like to install Flintrock directly from this PR (assuming you are running Python 3.4+), you can do that with this:
pip install git+https://github.com/nchammas/flintrock@easy-s3-access
Fixes #90 https://github.com/nchammas/flintrock/issues/90.
You can view, comment on, or merge this pull request online at:
https://github.com/nchammas/flintrock/pull/180 https://github.com/nchammas/flintrock/pull/180
Commit Summary
enable easy S3 access
add note about Hadoop 2.7+
File Changes
M .travis.yml https://github.com/nchammas/flintrock/pull/180/files#diff-0 (1)
M README.md https://github.com/nchammas/flintrock/pull/180/files#diff-1 (32)
M flintrock/config.yaml.template https://github.com/nchammas/flintrock/pull/180/files#diff-2 (4)
M flintrock/core.py https://github.com/nchammas/flintrock/pull/180/files#diff-3 (81)
M flintrock/flintrock.py https://github.com/nchammas/flintrock/pull/180/files#diff-4 (13)
M flintrock/services.py https://github.com/nchammas/flintrock/pull/180/files#diff-5 (59)
M flintrock/templates/hadoop/conf/core-site.xml https://github.com/nchammas/flintrock/pull/180/files#diff-6 (2)
M flintrock/templates/hadoop/conf/hdfs-site.xml https://github.com/nchammas/flintrock/pull/180/files#diff-7 (2)
A flintrock/templates/spark/conf/spark-defaults.conf https://github.com/nchammas/flintrock/pull/180/files#diff-8 (1)
M flintrock/templates/spark/conf/spark-env.sh https://github.com/nchammas/flintrock/pull/180/files#diff-9 (2)
M tests/conftest.py https://github.com/nchammas/flintrock/pull/180/files#diff-10 (30)
A tests/test_core.py https://github.com/nchammas/flintrock/pull/180/files#diff-11 (29)
Patch Links:
https://github.com/nchammas/flintrock/pull/180.patch https://github.com/nchammas/flintrock/pull/180.patch
https://github.com/nchammas/flintrock/pull/180.diff https://github.com/nchammas/flintrock/pull/180.diff
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/nchammas/flintrock/pull/180, or mute the thread https://github.com/notifications/unsubscribe-auth/AFsSZvas4bul_yAnvXEzqhpxuy2B23Fdks5rVrKcgaJpZM4LtB5L.

Alex Ioannides
|| 42a Guernsey Grove || Herne Hill  ||  London  ||  SE24 9DE ||
||<EMAIL_ADDRESS>|| 07903 356 154 ||

does this also support loading application jar from s3 ?

I have just tried launching job from application jar from s3 using flintrock from branch easy-s3-access , i have got following exception
java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.s3a.S3AFileSystem not found
Only way i got this working by adding jar to SPARK_CLASSPATH,
https://github.com/pragnesh/flintrock/commit/48c0e16ba0bcf558c707bec51a65f622c7ab4402
i know SPARK_CLASSPATH is deprecated since Spark 1.0, but this is the only way it works.

@pragnesh - I'm not familiar with launching jobs in the way you describe. Can you share an example of the command that triggers the error you reported?

@nchammas for example i have uploaded jar spark-examples_2.11-2.1.0.jar to "example_bucket" on s3,
if tried to launch job org.apache.spark.examples.SparkPi using following command then I got  "Class org.apache.hadoop.fs.s3a.S3AFileSystem not found" exception
spark-submit
--deploy-mode cluster
--class org.apache.spark.examples.SparkPi
--master spark://hostname:6066
s3a://example_bucket/spark-examples_2.11-2.1.0.jar
This command only work if i add s3 related jar to SPARK_CLASSPATH, This scenario supported with emr.

Hey guys,

Glad to see easier S3 access being implemented.

I remembered that I got some pretty  unintelligible errors when trying to access S3 files with Spark 2.0.1 on Hadoop 2.7.2, 2.6, 2.5. Had to revert to 2.4 and a custom version of Spark built against 2.4. Referenced this issue SPARK-7442.
This is detailed in my guide under Using Flintrock
It seems that the above will work with Spark 2.0, 2.1 on Hadoop 2.7.3. I will try it when possible,

Danny

Side note: Reading zipped csv files from S3 didn't work for me if anyone has any luck with that

@pragnesh - If you try this command with Spark 2.1 and Hadoop 2.7, does it work for you?
spark-submit \
  --packages org.apache.hadoop:hadoop-aws:2.7.3
  --deploy-mode cluster \
  --class org.apache.spark.examples.SparkPi \
  --master spark://hostname:6066 \
  s3a://example_bucket/spark-examples_2.11-2.1.0.jar

Hey @PiercingDan - Yup, this PR is supposed to address precisely the problem you documented in your guide, and which I also reported in SPARK-7442. 👍

Side note: Reading zipped csv files from S3 didn't work for me if anyone has any luck with that

Did this work using the approach you documented in your guide but not with this PR? Or are you reporting a separate issue? If it's a separate issue that you think may be related to Flintrock, I suggest opening a new issue. I'll take a look.

@nchammas I followed the method in my guide. I suspect the issue is unrelated to Flintrock, it is most likely spark-csv related.

@nchammas I have just tried the way you ask me to test it is still failing with same exception,
java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.s3a.S3AFileSystem not found

Unfortunately --packages does not work with --deploy-mode cluster : see SPARK-12559

Unfortunately --packages does not work with --deploy-mode cluster : see SPARK-12559

That looks like actual issue for "java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.s3a.S3AFileSystem not found" error.

looks like SPARK-10789 also talk about same issue.

Thanks for the references @dm-tran and @pragnesh. Looks like the application jar issue is something that should be left for future work.
Btw @pragnesh, I edited your comment to use triple backticks to format long blocks of code. It looks better. 👍
If there are no other questions or concerns about this PR, I will merge it in tonight or tomorrow.


Keeper setting

[x] Top tabs title is little bold make it as per design (for eg look at title keeper setting or App Backup

Verified this issue on keeper app v 1.2.0(319)


Fix the bug that Lab0 is allowed when user manually delete the labNumber attribute in json file.

If user deletes a Lab's LabNumber in json file now and restart the app, the app will still accept the json file and display Lab0 as a tag, which is an unexpected behaviour.
This fix prevents the above to happen and start with an empty ProgrammerError instead.

Codecov Report

Merging #589 (3cdebc1) into master (a5f1eee) will increase coverage by 1.83%.
The diff coverage is 52.18%.

@@             Coverage Diff              @@
##             master     #589      +/-   ##
============================================
+ Coverage     59.88%   61.72%   +1.83%
- Complexity      541      630      +89
============================================
  Files           100      104       +4
  Lines          2064     2252     +188
  Branches        222      261      +39
============================================
+ Hits           1236     1390     +154
- Misses          736      744       +8
- Partials         92      118      +26

Impacted Files
Coverage Δ

...n/java/seedu/programmer/commons/core/Messages.java
0.00% <ø> (ø)

...main/java/seedu/programmer/logic/LogicManager.java
38.46% <0.00%> (-8.42%)
:arrow_down:

...grammer/logic/commands/DashboardCommandResult.java
100.00% <ø> (+100.00%)
:arrow_up:

...ogrammer/logic/commands/DownloadCommandResult.java
100.00% <ø> (ø)

...u/programmer/logic/commands/ShowCommandResult.java
36.36% <ø> (+3.03%)
:arrow_up:

...programmer/logic/commands/UploadCommandResult.java
100.00% <ø> (+100.00%)
:arrow_up:

src/main/java/seedu/programmer/model/Model.java
100.00% <ø> (ø)

...n/java/seedu/programmer/model/student/ClassId.java
85.71% <ø> (+28.57%)
:arrow_up:

...ain/java/seedu/programmer/model/student/Email.java
80.00% <ø> (ø)

...ogrammer/model/student/QueryStudentDescriptor.java
85.71% <ø> (ø)

... and 63 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3e33619...3cdebc1. Read the comment docs.


chore(npm): update deps

What:  update dependecis
Why:
How:
Checklist:

[ ] Documentation
[ ] Tests
[x] Ready to be merged
[ ] Added myself to contributors table

Coverage remained the same at 100.0% when pulling a8cc592a5a1edf710a59474bce34238e4dbda466 on npm-update into ccf76eecc27d330ee9a3322c5b96953f16dc541e on master.


Mention log.html if long message is cut

Relevant error messages may be hidden by Robot if the output is verbose. The current warning is [ Message content over the limit has been removed. ], but it's not obvious that Robot does actually preserve the full output in log.html. I think it would be helpful to change the warning to something like [ Message content over the limit has been removed, see log.html for the full output. ]
I can make a PR to fix this, just wanted to make an issue first in case there are concerns.

I like the idea, but there are some problems with it:

The exact same message tests have is shown both in the log and in the report. The note should be worded so that it works in both situations. For example, the proposed "see log.html for full output" would be pretty confusing in the log.html file itself. Do you have ideas for a more generic message?
The log file name can be configured and thus using hard coded "log.html" isn't a good idea. This is easy to fix to just use term "the log file" instead.
Multiple tests are going to be broken due to this change. Fixing them is likely somewhat annoying but shouldn't be a blocker.

Notice also that our plan is to change showing long messages so that instead of cutting them, they'd be collapsed and the full message would be available behind some kind of a show the full message link or button (#4931). That change is currently planned for RF 8.0, but it doesn't have too high priority and it may be moved forward. Enhancing the note about cutting the message now would thus have benefits even if the note would be removed altogether in the future.

The exact same message tests have is shown both in the log and in the report. The note should be worded so that it works in both situations. For example, the proposed "see log.html for full output" would be pretty confusing in the log.html file itself. Do you have ideas for a more generic message?

That's a good point! Maybe something like [ Removed excess output, see relevant Keyword section in log.html for the full message. ] There's probably a better name than "Keyword section".

The log file name can be configured and thus using hard coded "log.html" isn't a good idea. This is easy to fix to just use term "the log file" instead.

I'm using log.html because Robot has four different output files and it's not immediately obvious to a beginner. Where is --log set? I'm guessing it'd be possible to obtain whatever value that is.

Multiple tests are going to be broken due to this change. Fixing them is likely somewhat annoying but shouldn't be a blocker.

I'd say that's the tests working as intended! :) I'd be willing to write the PR anyways so I can take on that work.

Notice also that our plan is to change showing long messages so that instead of cutting them, they'd be collapsed and the full message would be available behind some kind of a show the full message link or button (https://github.com/robotframework/robotframework/issues/4931).

That would be very nice! But yeah I agree that tweaking the truncation warning is more immediately helpful. A dropdown also wouldn't be helpful for the terminal output, which is actually why I opened this issue in the first place.


QUESTION regarding "making the bike faster"

I am looking to buy a super73 and I am in EU. You mention that the app can only do what is already built into the official app... But from my understanding the EU versions (specifically Sweden) cannot be put into unlimited/throttle/offroad mode where it goes faster than 25 km/h in the official app...?
Does this mean that the superduper app wont be able to do that either?
Just want to make sure I can go faster than 25 km/h when spending 2000 EUR...
Also new here so didnt know where to ask questions.

The settings for different modes are hidden in the official app based on your location and bike model. The bike itself is not locked, they are all the same bikes really.
Our apps show all the settings so you can switch to any US or EU mode.
(I believe superduper has US modes, walker73 has EU and US)
Here is a list of the modes with the speeds:
https://sites.google.com/view/super73-reverse-engineering/documentation?authuser=0#h.lne9cvtdnbbp

I see, thanks for the quick reply!
You don't by any chance know if all models (EU and US) have the loose connector for throttle upgrade or is that exclusive to US?

Throttles are not allowed on the street but it's perfectly legal to use one on private property so they wouldn't remove the connector.
They should all have it. In fact S73 sells "offroad" kits with throttles and different handle bars both in the EU and US and make it very clear that they are for private property only.

I cant seem to find those kits on their official website, only third party websites. However they do look oem like you say, however they also all say that they are only compatible with certain models... (S2, RX och ZX) that could however be in regards to the handlebar and not the throttle. Once again thank you for replying :)

Yeah probably different connectors and handlebars on the older modes.

Hi @RipppX
The Super73 is basically capped to 25km/h for legal reasons (in EU)
If your bike is the model S2, RX, ZX models, you are able to use SuperDuper app or the Super73 official app.
The throttle should be an optional add-on when purchasing the e-bike, which I highly recommend getting it.
You can proceed without it, but then you are purely relying on Pedal Assists to get you past 25km/h
Within both apps, there are settings where you can change it to go faster than 25km/h

Other models of the Super73 which are considered out-dated (slower). Some have throttles, but is only capped by it's motor compacity and don't have any Bluetooth settings.

If you don't want the handlebars, you can get the same throttle on AliExpress or amazon for a few bucks btw, don't get ripped off, it's a standard widespread connector. 😉
@Bamuel I get notifications for your issues and unsubscribing doesn't seem to be working so I guess I'll keep lurking around  😂😂

Thanks guys!


HI ADMIN THERE IS THIS BUG IN THE ALL MENU SECTION THE ALLMENU DOESN'T APPEAR

bot on and other menus function normally but in the all menu section the menu does not appear


Implement complex type usages and properties

A ComplexTypeUsage represents the reference to a complex type on an entity type or on another complex type. It contains the ComplexProperty instances that represent the properties of the complex type and their facets in this usage.
/cc @AndriySvyryd (But anyone can review...)

Agreed. We probably should review it with others--I'll make a note to talk about it when we discuss in the design meeting.


Using argument dim > 2 during encoding implies weird output during the decoding

If you use the dim argument of the Encoder.encode method and then decodes, the output contains again just 2 dimensions but with more points at the end with weird values. This problem appear in the version 1.1.1
Quick example:
import geojson
import geobuf
l = [(1,2,3),(4,5,6)]
geobuf.decode(geobuf.encode(geojson.LineString(l),6,3))
The output of this encode - decode is:
OrderedDict([('type', 'LineString'),              ('coordinates', [[1.0, 2.0], [4.0, 5.0], [7.0, 8.0]])])
It also ouputs a weird result if dim is 4 or 5.

Didn't had the same issue, in my case the decode wouldn't even work with an IndexError but I used more complex data than a LineString.
I attempted a patch here which seems to solve your issue: #50
Just tested your case with it returns me
OrderedDict([('type', 'LineString'), ('coordinates', [[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])])
And without: OrderedDict([('type', 'LineString'), ('coordinates', [[1.0, 2.0], [4.0, 5.0], [7.0, 8.0]])])
(Didn't tried with other dimenssions.)


Add version check for Minikube

Signed-off-by: Prathamesh Desai<EMAIL_ADDRESS>

@ozer550 Hi Prathamesh, Pls make sure that all the commits are GPG signed and not just the last few.

@ozer550 Hi Prathamesh, Pls make sure that all the commits are GPG signed and not just the last few.

There was some issue with my initial setup fo gpg keys that lead to unsigned commits. Will have all my commits signed from now on, sorry for the inconvenience!


NUTCH-2573 Suspend crawling if robots.txt fails to fetch with 5xx status

add properties

http.robots.503.defer.visits :
enable/disable the feature (default: enabled)
http.robots.503.defer.visits.delay :
delay to wait before the next trial to fetch the properties
(default: wait 5 minutes)
http.robots.503.defer.visits.retries :
max. number of retries before giving up and dropping all URLs from the given host / queue
(default: give up after the 3rd retry, ie. after 4 attempts)

handle HTTP 5xx in robots.txt parser
handle delay, retries and dropping queues in Fetcher

Stop queuing fetch items if timelimit is reached

re-queuing items where the robots.txt request returned a 5xx
redirects (http.redirect.max > 0) or
outlinks (fetcher.follow.outlinks.depth > 0)

In a first version, I forgot to verify whether the Fetcher timelimit (fetcher.timelimit.mins) was already reached before re-queuing the fetch item. This caused very few fetcher task to end up in an infinite loop. In detail, this happened:

fetcher thread starts fetching an item and requests the corresponding robots.txt. Ev., the server responds slowly.
fetcher timelimit is reached, all fetcher queues are flushed
robots.txt response "arrived". Because it's a 5xx the fetch item is re-queued and the fetch is delayed for 30 min. (custom configuration).

Then steps 1 and 3 are retried until the max number of retries is reached. But this was fixed and I've also made sure that redirects or outlinks are not queued if the timelimit is reached.

Hi @lewismc, done: updated metrics wiki page (hitByTimeLimit is already documented), added Javadocs and renamed the counter to follow the naming convention of the other robots_* counters. Also renamed the method ("timelimitReached" -> "timelimitExceeded").

Looks like it failed on Javadoc generation @sebastian-nagel


pkg/nixpath: make FromString receive non-absolute paths

The whole module layout was a bit messy - store paths had to be passed in primarily with the store dir prefix, even though it's not possible to have a different prefix, at least with the way it's currently structured.
There were Absolute() helper functions which helped with concatenation of strings, but no actual validation, which also were not really used outside of tests. The comments were also wrong, as no cleaning of store paths actually did happen.
Some library code (like pkg/derivation) even started implementing their own String() methods.
Clean this up a bit - have NixPath's FromString() receive a non-absolute path, and have an additional FromAbsolutePath() method that can be used if you have an absolute path.
Move from nixpath.Absolute() to a Absolute() function defined on a specific NixPath struct.
This moves things closer to how they look in the corresponding nix_compat::store_path::StorePath rust codebase.

As @zimbatm mentioned, nixpath.String() suddenly changing behaviour without warning is annoying and a footgun.
I renamed the package to storepath entirely, which will force users to explicitly upgrade (and look at the docstrings for the new package), hopefully keeping the blast radius smaller.
It's not perfect, but better than silently returning different strings on a go mod update.


Links flash white on ios when thumbing/scrolling through app.

Acceptance Criteria

[ ] Upon scrolling in app, when touching different sections(links) default white "flash" should not occur.

Context
Currently, when you thumb through the app on ios, anytime that you touch a link, the whole element flashes white.

We noticed that this issue is absent on an Android Alpha build through the Play store.

Ahhhh now I understand this issue. Yes, we are utilizing React Native's TouchableOpacity native feedback mechanism on most touchables.
We can either turn down the affect, or not use native feedback at all. Thoughts? @burkeshartsis any other ideas?
In an ideal world, I'd like to see a scale effect applied to cards when you tap on them in iOS, similar to what you see in the new app store, but that's something we could do down the road.

So from a UX perspective some kind of tap feedback is good. It's reassuring for the user and provides a consistent response to user tap interaction app wide. This is especially useful when tappable objects don't have any other discernible CTA. I could see how TouchableOpacity might be a bit much and would agree with @conrad-vanl that a slight scale effect would be great. Each native platform has it's own UX feedback pattern (Android has a ripple effect with an optional haptic vibration). My recommendation would be to not remove/change TouchableOpacity unless we have a suitable replacement for it.

I think my "easy win" suggestion was just to turn down the affect of the opacity a bit? I'd agree it's a little much now and distracting

@conrad-vanl @burkeshartsis FWIW, We have a build on alpha in the Play Store that @FrankGrand has used, and there seems to be zero touch response.
I agree that there needs to be some type of feedback and scaling the current touch response is a good start.

@IsaacHardy yep, android is supposed to be using it's native touch response, which shows the material design wave animation. It's either set to white, and white on white === no response, or it's showing the wave under the card instead of on top of it.
Either way, touch responses on android react native, like most things android + react native, are a bit behind and more inconsistent. We can only hope to make it better over time :) (and, fwiw, although low priority, I believe this is something @burkeshartsis wanted to look at IRC)

Might also be able to improve with this prop: https://facebook.github.io/react-native/docs/touchablewithoutfeedback.html#delaypressin


WIP: Well-kown URI

Using /.well-known/core for discovery

(Ehem, sorry for the linting error in my suggestion.)

Coverage increased (+28.0%) to 33.578% when pulling 79686bd53f6471341f06a0f466be6b84a8cb3ed9 on well-known-uri into 2e96675e96e5e95c15e75c46600ca2be8f162419 on main.

Coverage increased (+28.0%) to 33.578% when pulling 79686bd53f6471341f06a0f466be6b84a8cb3ed9 on well-known-uri into 2e96675e96e5e95c15e75c46600ca2be8f162419 on main.

Coverage increased (+28.0%) to 33.578% when pulling 79686bd53f6471341f06a0f466be6b84a8cb3ed9 on well-known-uri into 2e96675e96e5e95c15e75c46600ca2be8f162419 on main.


Error: 'dart:ffi' can't be imported when compiling to Wasm.

Unable to compile to Wasm because of ffi package. I don't use ffi directly but other packages I am using relies on it. I am assuming ffi package doesn't support wasm compilation yet!
Error: 'dart:ffi' can't be imported when compiling to Wasm.
                     import 'dart:ffi';

@BirjuVachhani Then either the package doesn't work on the web or the package uses conditional imports that need to be fixed. Please see the exact dependencies that led to the import of dart:ffi and see if any of them is relying on conditional imports. If so, then they may need to be fixed (e.g. most common issue: import ... if (dart.library.html) needs to be replaced with import ... if (dart.library.js_interop) ... - as dart:html is only available in dart2js but not dart2wasm - see https://github.com/dart-lang/sdk/issues/56358)
Adding FFI support on the web is a long standing issue that is tracked here: https://github.com/dart-lang/sdk/issues/46690


Sync to Slim\Http in 4.x

Pick up all the changes to Slim\Http in Slim 4 by copying the code from Slim 4.x into this repository.
Notes:

I've kept the createFromEnvironment to createFromGlobals rename
I've deleted the FactoryDefault factory class.
I've added PHPCS.
I've added PHP 5.6 testing to Travis.
I've removed completely base path - we need to handle this in Slim, not Slim-Http

Coverage increased (+7.6%) to 98.197% when pulling e609c8a9cdfea3b03f47066a01022ad6aa702412 on akrabat:replace-with-slim4x-version into 7407c9ebf1011ef634b28d5bd2c54bede9db04d2 on slimphp:master.

In theory this looks pretty good.

Coverage increased (+7.7%) to 98.207% when pulling 9c485cd15be47ea1d1b604436e235e335a6639ed on akrabat:replace-with-slim4x-version into 7407c9ebf1011ef634b28d5bd2c54bede9db04d2 on slimphp:master.

Coverage increased (+7.7%) to 98.207% when pulling 556b6285a6d3eed842e5772effcaa76801dbc4fd on akrabat:replace-with-slim4x-version into 7407c9ebf1011ef634b28d5bd2c54bede9db04d2 on slimphp:master.


$content and $type

Serializing of collections results in a funny schema. For instance I serialized an array of strings, here is my result.
"{
\"$content\":[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\",\"8\",\"9\"],
\"$type\":\"System.String[]
\"}"

This output is not up to standards and does not work with other json serializers (specifically the ones running on my webserver). Is there any way I could turn off this functionality ? Specifically it should look like this.
[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\",\"8\",\"9\"]

Just an update. In my 'api wrapper' class I included some logic to extract the $content value.

        /// <summary>
        /// Serializes the object into json
        /// </summary>
        /// <typeparam name="T"></typeparam>
        /// <param name="value"></param>
        /// <param name="prettyJson"></param>
        /// <returns></returns>
        public static string Serialize<T>(T value, bool prettyJson = false)
        {
            fsData data;
            fsResult r = Internal.TrySerialize<T>(value, out data);

            if (r.Failed)
                throw r.AsException;

            if (data.IsDictionary && data.AsDictionary.ContainsKey("$content"))
                data = data.AsDictionary["$content"];

            if (!prettyJson)
                return fsJsonPrinter.CompressedJson(data);

            return fsJsonPrinter.PrettyJson(data);
        }

I probably won't be able to take a closer look for a few more days, but what does your serialize call invocation look like? Ie, what is typeof(T) equal to?

Ive tried it with two objects. The first was a string array (string[]). I was passing a set of ID's to my db. The second was this DemoObject (similar scenario).
 // [StorageTable("DemoObject")]
        class DemoObject
        {
           // [StorageIdentity]
            public string Id { get; set; }
            public string String { get; set; }
            public int Number { get; set; }
            public Color Color { get; set; }
            public Vector3 Vector { get; set; }

            private static int Counter;
            public static DemoObject Create()
            {
                return new DemoObject
                {
                    Id = (Counter++).ToString(),
                    String = UnityEngine.Random.value.ToString(),
                    Number = UnityEngine.Random.Range(0, 10),
                    Vector = UnityEngine.Random.insideUnitSphere,
                    Color = new Color(UnityEngine.Random.value, UnityEngine.Random.value, UnityEngine.Random.value)
                };
            }
        }

What is the actual Serialize invocation look like?

Serializer Class
using System;

namespace FullSerializer
{
    /// <summary>
    /// Static instance for easy serializing
    /// </summary>
    public static class JsonSerializer
    {
        /// <summary>
        /// Static Serializer Instance
        /// </summary>
        public static readonly fsSerializer Internal = new fsSerializer();

        /// <summary>
        /// Serializes the object into json
        /// </summary>
        /// <param name="value"></param>
        /// <param name="type"></param>
        /// <returns></returns>
        public static string Serialize(object value, Type type)
        {
            // serialize the data
            fsData data;
            Internal.TrySerialize(type, value, out data).AssertSuccessWithoutWarnings();

            // emit the data via JSON
            return fsJsonPrinter.CompressedJson(data);
        }

        /// <summary>
        /// Serializes the object into json
        /// </summary>
        /// <typeparam name="T"></typeparam>
        /// <param name="value"></param>
        /// <param name="prettyJson"></param>
        /// <returns></returns>
        public static string Serialize<T>(T value, bool prettyJson = false)
        {
            fsData data;
            fsResult r = Internal.TrySerialize<T>(value, out data);

            if (r.Failed)
                throw r.AsException;

           // hack fix
           // if (data.IsDictionary && data.AsDictionary.ContainsKey("$content"))
           //     data = data.AsDictionary["$content"];

            if (!prettyJson)
                return fsJsonPrinter.CompressedJson(data);

            return fsJsonPrinter.PrettyJson(data);
        }

        /// <summary>
        /// Deserializes the Json into an object
        /// </summary>
        /// <typeparam name="T"></typeparam>
        /// <param name="json"></param>
        /// <returns></returns>
        public static T Deserialize<T>(string json)
        {
            if (string.IsNullOrEmpty(json))
                return default(T);

            fsData data;
            fsResult fsFailure1 = fsJsonParser.Parse(json, out data);
            if (fsFailure1.Failed)
                throw fsFailure1.AsException;

            T instance = default(T);

            fsResult fsFailure2 = Internal.TryDeserialize<T>(data, ref instance);

            if (fsFailure2.Failed)
                throw fsFailure2.AsException;

            return instance;
        }

        /// <summary>
        /// Deserializes the Json into an object
        /// </summary>
        /// <param name="json"></param>
        /// <param name="type"></param>
        /// <returns></returns>
        public static object Deserialize(string json, Type type)
        {
            if (string.IsNullOrEmpty(json))
                return null;

            // step 1: parse the JSON data
            fsData data = fsJsonParser.Parse(json);

            // step 2: deserialize the data
            object deserialized = null;
            Internal.TryDeserialize(data, type, ref deserialized).AssertSuccessWithoutWarnings();

            return deserialized;
        }
    }
}

Use
void Awake(){
            var strings = new[] {"0","1", "2","3"};
            var errorJson1 = JsonSerializer.Serialize(strings);
}

Output
"{
\"$content\":[\"0\",\"1\",\"2\",\"3\"],
\"$type\":\"System.String[]
\"}"

Should Look Like
[\"0\",\"1\",\"2\",\"3\",\"4\",\"5\",\"6\",\"7\",\"8\",\"9\"]

When I run the test case I get the expected output of ["0","1","2","3"]
Here's the entire file:
using System;
using FullSerializer;
using UnityEngine;

/// <summary>
/// Static instance for easy serializing
/// </summary>
public static class JsonSerializer {
    /// <summary>
    /// Static Serializer Instance
    /// </summary>
    public static readonly fsSerializer Internal = new fsSerializer();

    /// <summary>
    /// Serializes the object into json
    /// </summary>
    /// <param name="value"></param>
    /// <param name="type"></param>
    /// <returns></returns>
    public static string Serialize(object value, Type type) {
        // serialize the data
        fsData data;
        Internal.TrySerialize(type, value, out data).AssertSuccessWithoutWarnings();

        // emit the data via JSON
        return fsJsonPrinter.CompressedJson(data);
    }

    /// <summary>
    /// Serializes the object into json
    /// </summary>
    /// <typeparam name="T"></typeparam>
    /// <param name="value"></param>
    /// <param name="prettyJson"></param>
    /// <returns></returns>
    public static string Serialize<T>(T value, bool prettyJson = false) {
        fsData data;
        fsResult r = Internal.TrySerialize<T>(value, out data);

        if (r.Failed)
            throw r.AsException;

        // hack fix
        // if (data.IsDictionary && data.AsDictionary.ContainsKey("$content"))
        //     data = data.AsDictionary["$content"];

        if (!prettyJson)
            return fsJsonPrinter.CompressedJson(data);

        return fsJsonPrinter.PrettyJson(data);
    }

    /// <summary>
    /// Deserializes the Json into an object
    /// </summary>
    /// <typeparam name="T"></typeparam>
    /// <param name="json"></param>
    /// <returns></returns>
    public static T Deserialize<T>(string json) {
        if (string.IsNullOrEmpty(json))
            return default(T);

        fsData data;
        fsResult fsFailure1 = fsJsonParser.Parse(json, out data);
        if (fsFailure1.Failed)
            throw fsFailure1.AsException;

        T instance = default(T);

        fsResult fsFailure2 = Internal.TryDeserialize<T>(data, ref instance);

        if (fsFailure2.Failed)
            throw fsFailure2.AsException;

        return instance;
    }

    /// <summary>
    /// Deserializes the Json into an object
    /// </summary>
    /// <param name="json"></param>
    /// <param name="type"></param>
    /// <returns></returns>
    public static object Deserialize(string json, Type type) {
        if (string.IsNullOrEmpty(json))
            return null;

        // step 1: parse the JSON data
        fsData data = fsJsonParser.Parse(json);

        // step 2: deserialize the data
        object deserialized = null;
        Internal.TryDeserialize(data, type, ref deserialized).AssertSuccessWithoutWarnings();

        return deserialized;
    }
}

public class MyTest : MonoBehaviour {
    public void Awake() {
        var strings = new[] { "0", "1", "2", "3" };
        var errorJson1 = JsonSerializer.Serialize(strings);
        Debug.Log(errorJson1);
    }
}

Ok, I think I understand what is going on. It looks like the serializer likes to include the complex schema when the type is weak. See my example below.
public class MyTest : MonoBehaviour
{
public class MyComplexTest
{
public object Inner2 { get; set; }
}
public void Awake()
{
    var strings = new[] { "0", "1", "2", "3" };

    var tester = new MyComplexTest();
    tester.Inner = strings;

    var errorJson1 = JsonSerializer.Serialize(tester);
    Debug.Log(errorJson1);
}

}
The result was "{"Inner":{"$content":["0","1","2","3"],"$type":"System.String[]"}}"
Is there any way I could turn this functionality off ?

I don't have any plans for wholesale disabling emitting type information (since if you disable emitting types / handling inheritance, then you also probably want to disable cycle support)
Would something like
public class Model {
    [fsProperty(DoNotEmitTypeMetadata = true)]
    public object Member;
}

work?
Not saying that's what I would implement, just thinking.

Thanks for working with me. Your awesome.
That might do the trick. So long as I can pass 'standard' json to my web-services, I am happy. Dynamo specifically is very fickle over its json.
If I get some time this week. I will take a look into the code to see If I can find an elegant solution.

I have the same issue with "$content" information showing up in JSON when I think it should not. I've created a failing test case along with the serialization class I'm using here:
https://gist.github.com/robertwahler/a17cf3a990207212344d
The first of the two tests in JsonSerializerGenericTest.cs fails, the test comments show the emitted JSON so you don't actually have to run the test to see the issue.

Hey robertwahler
Take a look at my solution. It is somewhat hackish, but works
https://github.com/NVentimiglia/Unity3d-Databinding-Mvvm-Mvc/blob/master/Foundation Databinding/Assets/Plugins/JsonSerializer.cs

Okay, thanks. I think I know what is causing this (here if you're interested). I'll take a look into fixing it when I get a chance.

@robertwahler Thanks for the test case! It nicely identified the issue. I've also committed it to the repo.


Add mesos-master versions 0.25.0 and 0.26.0 for Centos 7 and Ubuntu 1…

Hi --
I added support for mesos-master version 0.25.0 and 0.26.0 for both CentOS 7 and Ubuntu 14.04.
Thanks!
Kevin

Hi -- Any updates for this, https://github.com/mesoscloud/mesos-slave/pull/3, or https://github.com/mesoscloud/marathon/pull/2 ?

Hm, this repos have not seen much activity lately, is this EOL?!
:+1: to @kevinschoon PR