Duplicate methods printed . JSON document has duplicates as well.

I have a simple spring MVC app that has a few rest services. Swagger works great and the only thing it appears to do wrong is print each api twice for the same http method.
I looked at the JSON document it pulls for each API and they have duplicates too. SO I dont think it's the UI that has the bug, but rather the Swagger-spring MVC plugin.

This api array contains an object with this "path":"/rest/vendor/create" twice and the same way for others that are repeated.
"apis":[
{
"path":"/rest/vendor/create",
"description":"create",
My Spring MVC gradle build has this.
compile 'com.mangofactory:swagger-springmvc:1.0.0'
compile 'com.fasterxml.jackson.core:jackson-databind:2.5.0'

Any help is appreciated . Thanks!

@pprabhu3430 Could you please try with 1.0.2 and report back?

I got 1.0.2 and its the same exact issue. Duplicates are seen.
Also I notice that the response object doesnt correctly populate sometimes. In the below case a List of VendorType is returned. The VendorType is a hibernate entity, not sure if that causes any conflict. Use APIModel annotations doesnt help. When the response is not a hibernate entity, it seems to work fine and shows the response structure correctly. The entity correctly has getters and setters defined for the properties. However they are capital case properties to accomodate a legacy system we have which serializes all JSON object properties to capital case.

Ok I got the latest Swagger UI and the issue doesnt exist anymore. I guess they have a way of removing duplicates if present in the api definition. No idea. Putting this out there in case it helps someone.

@pprabhu3430 Could you confirm if the dupes exist in the api-docs json?

I have a screenshot of the json doc returned for my vendor api.

As you can see there are duplicates in the Returned JSON structure.

@pprabhu3430 any updates on this? I'm closing this issue as not reproducible. Feel free to reopen if its still not fixed

swagger-springmvc
0.8.8
I am having this issue and it is returning the same operation multiple itmes

Can some one help me how to fix it


DO should accept SelfIp with mixed allow-service: default and ports.

That type of config is valid for BIG-IP:

net self /Common/<IP_ADDRESS> {
address <IP_ADDRESS>/16
allow-service {
tcp:443
default
}
traffic-group /Common/traffic-group-1
vlan /Common/external
}
But DO doesn't have way to process that. It can take only:
string ("default", "all", "none"), or array of ['tcp:80', 'tcp:8080', 'udp:8088']

Thank you for your feedback. I have added this to our internal product backlog as AUTOTOOL-2845.

This issue was resolved as part of the 1.29.0 release of Declarative Onboarding.


More detail would be great!

We saw one of the CosmosDB Product Managers use a similar architecture at a recent meetup and it looked great, I'm struggling to find more code on it, so a well done example could be helpful!

Document details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 25c5d5cf-09ea-ed8f-11d3-b8c8e8a8c2b9
Version Independent ID: 5050d13f-4976-58a2-2a49-0a1f43b67a9f
Content: IoT using Cosmos DB - Azure Solution Ideas
Content Source: docs/solution-ideas/articles/iot-using-cosmos-db.md
Service: architecture-center
GitHub Login: @adamboeglin
Microsoft Alias: pnp

We have a full IoT reference architecture that might help: https://docs.microsoft.com/en-gb/azure/architecture/reference-architectures/iot/
there is code supporting that linked at the bottom (in github)
directly here: https://github.com/mspnp/iot-guidance

@JRhodes95  For now I will proceed with closure of this and If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.


Implement integration tests for Invenio

Prepare Travis tests for lastest versions of pure Invenio (>=3.2):

[x] The build & tests should be tried in the following environment:

Python 3.8
PostgreSQL 9.6
ElasticSearch 7

[x] For now, it is sufficient to test the build by running something inside invenio shell

[x]  Tests should be in a separate shell script run_tests.sh, called from travis manifest

[x] When the test succeeds, Travis should generate a requirements.txt artifact from pip freeze

@mesemus could you please review?


More on EC2 instance status checks and events

Some coverage in the EC2 section on instance status checks (system reachability, instance reachability) and maybe Events, e.g. "Hey, 200 of your instances are going to be rebooted in 2 days!"

What status checks are and what they mean
What kinds of notifications happen and why, and how to plan for them

Occasionally, an instance will have issues that AWS automatically checks -- System and Instance reachability.  A system check failure is due to a case where either the actual machine instance or networking or other problem occurs and must be resolved by AWS.  Instance reachability errors occur if the operating system is failing, memory or disk space is exhausted; and typically you can intervene to resolve.
In many cases, unexplained instance reachability checks are a precursor to a hardware failure, so should be addressed immediately.  This problem becomes more common for a given instance generation over time since hardware is likely to fail more frequently.  Make sure you have good systems in place to recognize the AWS emails, and set up specific CloudWatch monitoring for critical systems.
Tip/Trick: if you are seeing system reachability failures, this can be a sign that it's time to upgrade your hardware to a more modern generation.
Often, Instance Check failures can be resolved with an instance reboot (restart OS on same hardware), or restart (different hardware, but configured the same).  The AWS console will eventually force a reboot in cases where you are not able to log into the instance.
It is important to pay careful attention to emails sent by AWS, as there are several cases where planned events occur.  These will also display in the AWS EC2 Console in the Events tab.  Cases include occasional required maintenance of the underlying hardware, during which you'll typically get advanced notice.  More recently, OS or Xen vulnerabilities will result in a very short window of perhaps several days.  The event will typically be a reboot, but may also be a restart.
Tip/Trick: because these events can occur very rapidly, and sometimes with no notice, you should ensure that your system will survive a reboot.  Common errors include services that are not configured to start automatically -- when making changes to configuration, be sure to test in advance.
Danger/Warning: Instance Store "ephemeral" drives will survive reboot, but will not survive restart; use EBS, S3, or EFS mounts when practical to ensure that data is not lost.
Tip/Trick: AWS is built around the assumption that machines are replaceable; where possible, configure your instances to meet this assumption.  Some systems such as Solr, MongoDB, or Hadoop tend to use instance types such as hs1 and d2 because they have terabytes of instance store drives that are cheaper and faster than EBS.  Always use EBS-backed instances for boot and configuration information, as that will survive.  If you must store data on instance store drives, make sure that your system knows how to detect and automatically mount and format drives at boot time if not present. While Hadoop and other systems will attempt to replicate data, this is costly and increases risk.
Warning/Danger: it is common for an entire family of machines to need reboot, often due to security issues. You should be prepared to respond quickly if an uncontrolled reboot by AWS will affect your system.


Replace @CheckForNull with @NonNull

According to codingstyle: https://github.com/uhafner/codingstyle/blob/13767f1ef6a77a844899905839b2c6418b33e027/codacy/checkstyle.xml#L145

@uhafner I don't understand why @CheckForNull is not allowed here in the coding style, to avoid Null pointers exist? Also, if we follow this, before (say String) when we return a null, now we are going to return an empty string?

@uhafner I don't understand why @CheckForNull is not allowed here in the coding style, to avoid Null pointers exist? Also, if we follow this, before (say String) when we return a null, now we are going to return an empty string?

Neither CheckForNull is used quite a lot in Jenkins =/

There are multiple annotations to allow null as value. I'm using the general edu.umd.cs.findbugs.annotations.Nullable. In order to have only this single annotation in my plugin I added the architecture check. The problem with @CheckForNull is that it is not analyzed by error prone and NullAway.

The best thing is to avoid null as valid return value if possible.

There are multiple annotations to allow null as value. I'm using the general edu.umd.cs.findbugs.annotations.Nullable. In order to have only this single annotation in my plugin I added the architecture check. The problem with @CheckForNull is that it is not analyzed by error prone and NullAway.

as far as I understand spotbugs doesn't use Nullable but does use CheckForNull,
cc @oleg-nenashev

The best thing is to avoid null as valid return value if possible.

absolutely

Spotbugs does not report a warning if one uses ´Nullable´. It is only reported in IntelliJ and ErrorProne. I think I can remove the check from Checkstyle and move it to my ArchitectureTests. Then it is not flagged here...

I saw both edu.umd.cs.findbugs.annotations.Nullable and http://findbugs.sourceforge.net/api/edu/umd/cs/findbugs/annotations/CheckForNull.html, the reason we are still using it is that javax.* annotations have licenses issues and we can't find better replacements?
And for the situation to return a String, is it better to return an empty string than a null pointer?

I'm not sure if I understand your last comment. SpotBugs defines three annotations

edu.umd.cs.findbugs.annotations.Nonnull
edu.umd.cs.findbugs.annotations.Nullable
edu.umd.cs.findbugs.annotations.ChecksForNull

The other old annotations with the same name but the javax.* package should not be used anymore.

And so for the situation to return a String, is it better to return an empty string than a null pointer? But it seems return a null pointer gives the users a hint that some problems during the calling....

Rather then returning null the method that wants to return null should throw an exception (and not the consumer of the method).

Sorry, some problems with typing on my last comment, just saw annotations like edu.umd.cs.findbugs.annotations.Nullable are marked as deprecated, see http://findbugs.sourceforge.net/api/edu/umd/cs/findbugs/annotations/Nullable.html

They are not @Deprecated anymore. Spotbugs reverted that since the JSR305 is dead (see https://github.com/spotbugs/spotbugs/issues/130).

You need 4.x of spotbugs, older versions of Jenkins pull in 3.x as a transitive dep, I think of you make it an explicit dependency it’ll work

I created a PR for NullAway: https://github.com/uber/NullAway/pull/397
So I think it is save to continue to use SpotBugs @CheckForNull annotation. I'll remove the import check from the CheckStyle rules.

Thanks! Nice work

Great! Thanks.


fix incorrect variable reference in rf-scikit Snowflake example

This PR fixes the following error in one of the snowflake notebooks:
E           Exception: Non-zero exit code (1) from pod j-testd-examples-gpu-61459954e99644409f8f6917b52bf252-866fd97gp:
E           [NbConvertApp] Converting notebook nyc-taxi-snowflake/rf-scikit.ipynb to python
E           [NbConvertApp] Writing 13911 bytes to nyc-taxi-snowflake/rf-scikit-fb763d16-c78f-4b47-8255-05b38d390f79.py
E           ---------------------------------------------------------------------------
E           AttributeError                            Traceback (most recent call last)
E           ~/project/examples/nyc-taxi-snowflake/rf-scikit-fb763d16-c78f-4b47-8255-05b38d390f79.py in <module>
E               151 columns = [x[0] for x in taxi.description]
E               152 taxi_train = pd.DataFrame(taxi.fetchall(), columns=columns)
E           --> 153 taxi.columns = taxi.columns.str.lower()
E               154
E               155
E
E           AttributeError: 'SnowflakeCursor' object has no attribute 'columns'

In examples-gpu/nyc-taxi-snowflake/rf-scikit.ipynb, the variable taxi is used to refer to a Snowflake query result, but there are some parts of the notebook that reference it as if it was a pandas DataFrame.
How I tested this
I ran this notebook in a new examples-gpu project on app.internal, which uses the latest Saturn images.

This looks different than what's in main (I made some fixes a couple weeks ago)


docs(examples): add simple example

This is an example of a GrAMPS server with no data sources. Still need to add additional examples (data sources from npm, local data sources, mixed).

Examples should move to the https://github.com/gramps-graphql/gramps repo instead.


OpenShift SDN Connectivity is broken

Not exactly an Operator issue, but Cryostat+OpenShift related.
There are connectivity issues between Cryostat and target pods for intra-cluster JMX connections when using OpenShift SDN, where some large proportion of network requests fail.

This bug is caused by JDP autodiscovery, which we don't need in Kubernetes environments. See: https://bugzilla.redhat.com/show_bug.cgi?id=1979733


fix(lib): getting .git from parent dir

Closes #72

Codecov Report

Merging #74 into master will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff          @@
##           master    #74   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files           3      3
  Lines         123    126    +3
=====================================
+ Hits          123    126    +3

Impacted Files
Coverage Δ

lib/getGitFolder.js
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a579a3d...ed34139. Read the comment docs.

find-parent-dir was already in yarn.lock, since husky from devDependencies uses it.
https://github.com/kentcdodds/validate-commit-msg/blob/master/yarn.lock#L1097
I wonder why library has lockfile, isnt it ignored by package users? Looks like just yet another file to maintain

Having same packages for every contributor does not guarantee that package will work for users, I think that having some diversity would be better for contributors
I know it might be some risks from having different versions inside dependencies, but since lockfile has same risks but for for users, we should rely on semver (and avoid package who do not)


NVLOG to prevent unused var

clang 5 fails when NVLOG set because resource is unused
Signed-off-by: Daniel Hochman<EMAIL_ADDRESS>

@htuch I noticed this in my local env. We don't use NVLOG anywhere in testing. On our release build we use NVLOG but with gcc and not clang.


Depth map

Hi,
I found in the "Depth Images" under "Outputs' structure details", it stated, "Depth values are scaled between 0 and the specified Depth range value".  I have the following questions:

How exactly is the scaling done?
Would it be different for '.png' and 'exr" representations?
Besides, would it be possible to get the absolute distance of the object to the camera from (1). the depth image from EasySynth (2) other depth shaders?

Hi,
I can see the wording is not the best. You can see the structure of the depth shader (material) in ContentBrowser, at All/Plugins/EasySynth Content/PostProcessingMaterials/M_PPDepthImage. You can change its functionality to fit your needs.

The shader has access to absolute depth values (in cm), but as we want to save them as valid (displayable) images, which have a defined range (PNG: [0, 255], EXR: [0.0, 1.0]), there needs to be an upper distance limit that is mapped to the maximum color intensity.

Linear (not radial) depth values are clamped to the number of meters specified in the Depth range parameter and then divided by the same parameter to achieve the [0.0, 1.0] range.
Our shader always outputs a value between 0.0 and 1.0. The engine automatically scales this value to [0-255] if png is the chosen output format.
I just tried and it seems that you can bypass all calculations in the M_PPDepthImage and connect the SceneDepth parameter directory to Emissive Color. This way, the shader output will have a range way outside [0.0, 1.0] and the results will be:

EXR images: contain absolute depth values in centimeters, and appear as completely white when opened in an image viewer
PNG images: invalid, pixels farther than 255 cm are "clipped" to 255, they are completely white

In order to get the distance from the camera instead of the distance from the camera plane, something like this is needed:


Make black a mandatory test

A clean attempt at #5

Next step is isort --profile black .

Thanks for the help.


Roll engine 7d3e722349e7..6a8782fe0162 (10 commits)

https://github.com/flutter/engine/compare/7d3e722349e7...6a8782fe0162
git log 7d3e722349e7923115f0b8800022a08c8dd795b3..6a8782fe0162fa4bce5458c92dda4a3ea280825b --no-merges --oneline
6a8782fe0 Roll src/third_party/skia 93eeff578b08..0e0113dcbd9a (9 commits) (flutter/engine#9662)
791143f17 ExternalViewEmbedder can CancelFrame after pre-roll (flutter/engine#9660)
d637f291a External view embedder can tell if embedded views have mutated (flutter/engine#9653)
ceee3d733 Roll src/third_party/skia 38ae3f42fec1..93eeff578b08 (1 commits) (flutter/engine#9659)
d75729006 Roll src/third_party/skia febc162c7898..38ae3f42fec1 (3 commits) (flutter/engine#9658)
58133abce Roll src/third_party/skia 3de5c6388142..febc162c7898 (2 commits) (flutter/engine#9657)
29342dd30 Roll src/third_party/skia 1e2cb444e0c1..3de5c6388142 (8 commits) (flutter/engine#9656)
b5473560d Pipeline allows continuations that can produce to front (flutter/engine#9652)
8306ee68c Move the mutators stack handling to preroll (flutter/engine#9651)
511b9f23a Roll src/third_party/skia 215ff3325230..1e2cb444e0c1 (12 commits) (flutter/engine#9650)
The AutoRoll server is located here: https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md
If the roll is causing failures, please contact the current sheriff (jsimmons@google.com), and stop
the roller if necessary.

Trybots failed. These were the failed builds: https://cirrus-ci.com/task/6054167045472256

Commit queue failed; closing this roll.


fix MHA with avx512

Type of Change
bug fix
Description
fix MHA avx512 accuarcy on CLX and ICX
Expected Behavior & Potential Risk
the expected behavior that triggered by this PR
How has this PR been tested?
how to reproduce the test (including hardware information)
Dependency Change?
any library dependency introduced or removed

TODO: a). perf check b). remove omp

performance on SPR has no change.

performance on 13900K (Linux)
===================before====================================
model_print_timings:        load time =  5039.76 ms
model_print_timings:      sample time =    13.87 ms /    32 runs   (    0.43 ms per token)
model_print_timings: prompt eval time =  5039.36 ms /  1024 tokens (    4.92 ms per token)
model_print_timings:        eval time =  2378.82 ms /    31 runs   (   76.74 ms per token)
model_print_timings:       total time =  7438.79 ms
===================after====================================
model_print_timings:        load time =  3938.19 ms
model_print_timings:      sample time =    15.15 ms /    32 runs   (    0.47 ms per token)
model_print_timings: prompt eval time =  3937.72 ms /  1024 tokens (    3.85 ms per token)
model_print_timings:        eval time =  2135.32 ms /    31 runs   (   68.88 ms per token)
model_print_timings:       total time =  6095.11 ms

Currently, in docs/fused_attention.md the Runtime ISA is mentioned as AMX_BF16 . I guess with this PR the text should be changed to AVX512F or newer .

Currently, in docs/fused_attention.md the Runtime ISA is mentioned as AMX_BF16 . I guess with this PR the text should be changed to AVX512F or newer .

@bil-ash  Title changed. This PR changes the global threading. docs/fused_attention.md is out of date, AVX512F and AVX2 are all ready for use.


feat(commands): added Git hook argument handling

Type: feature
Scope: commands
Related issues: fixes #133
What is this?
Added functionality to propagate Git hook script invocation arguments to Lefthook commands in the form of positional shorthands such as {1} for the first argument, {2} for the second, etc.
Also introduced {0} as the equivalent shorthand of {1} {2} ... (space joint string of individual Git hook arguments) so all arguments can be passed down to command calls with one shorthand regardless of the actual argument count.
Why?
To improve script vs. command feature parity and make commands able to use Git hook arguments directly.
Some Git hooks pass down arguments to the hook scripts to specify what is happening exactly.
An example is commit-msg where the file containing the proposed commit message is passed down as a positional argument - because it can be .git/COMMIT_EDITMSG, .git/GITGUI_MSG or another source.
In such cases this argument is best to be passed transparently to commands by Lefthook as well in order to be able to define simple commands using the exact proposed commit message or its file from the argument to do their work.

Awesome! Will look at soon.


RuntimeError: glLinkProgram() failed:

self.cpp_wrapper = _get_plugin().RasterizeGLStateWrapper(output_db, mode == 'automatic', cuda_device_idx)
RuntimeError: glLinkProgram() failed:
Fragment info
0(2) : error C7528: OpenGL reserves names starting with 'gl_': gl_PrimitiveID
(0) : error C2003: incompatible options for link

You may try to edit this file "nvdiffrast/common/rasterize.cpp" with remove all "in int gl_PrimitiveID" ,  and it will work.....

Also, if you come from https://github.com/NVlabs/eg3d make sure that you clone the current version of nvdiffrast to the repository.
The EG3D repo currently links to an old branch of Deep3DFaceRecon, which recommends using git clone --recursive. This, however, clones an old version of nvdiffrast to Deep3DFaceRecon/nvdiffrast, which produces the same error you mentioned.
To fix this I uninstalled nvdiffrast, removed the folder and cloned the current version via git clone https://github.com/NVlabs/nvdiffrast .
See here for more info.


Remove version: 360.360Chrome.X version 21.0.1082.0

[Automated] It returns code over 400 in all urls
Microsoft Reviewers: Open in CodeFlow

URL: https://down.360safe.com/cse/360csex_21.0.1082.0.exe
Status Code: 200
(Automated message - build 897)

Close with reason: Package still available.;


NOT mask?

It'd be nice to have the ability to use json-mask to say what fields should NOT be returned.  For instance, if I have some JSON where every field except one should be included.
Is this possible currently?  I didn't see anything in the API docs for it.

That would be a great feature to have. I'd love a pull request for it, if you're interested.
Maybe something that looks like this:
mask({p: {a: 1}, z: 1}, '!p')  // {z: 1}

And in case when there are both, the not should take precedence.

:+1: I've been using this with koa-json-mask and just started using it for body parameter filtering, so I may take a crack at it.
On another note, I've used strong_parameters with Rails, and one thing I found useful was the ability to require parameters and throw an error rather than just permitting/denying them. If this sounds interesting, what kind of syntax do you think might make sense?

@jordansexton go for it.
Re. throwing errors: I'm not sure I completely understand (perhaps having not used strong_parameters). Do you mean instead of masking values, this lib would be used as a validator to check whether an object matches a certain structure?

Not instead of masking values, but in addition to. Here are the three behaviors (not mutually exclusive):
// existing behavior (inclusion)
> mask({ firstName: 'Foo', lastName: 'Bar' }, 'firstName')
{ firstName: 'Foo' }

// first proposed behavior (exclusion)
> mask({ firstName: 'Foo', lastName: 'Bar' }, '!firstName')
{ lastName: 'Bar' }

> mask({ firstName: 'Foo', lastName: 'Bar' }, '!firstName,!lastName')
{ }

// second proposed behavior (requirement)
> mask({ firstName: 'Foo', lastName: 'Bar' }, 'firstName!')
{ firstName: 'Foo' }

> mask({ firstName: 'Foo', lastName: 'Bar' }, 'firstName!,lastName!,middleName!')
Error: field "middleName" required

// combined behaviors
> mask({ parent: { name: 'Parent', children: [{ name: 'Child' }] } }, 'parent!/!name,children(!name)')
{ parent: { children: [{}] } }

> mask({ parent: { name: 'Parent', children: [{}] } }, 'parent!/!name,children(name!)')
Error: field "parent/children(name)" required

This makes sure that the output doesn't have anything you don't want, but also that it definitely has things you do want. What do you think?

Coming from strong_parameters' builder-style DSL, the form might be:
require(:parent).permit(:children).require(:name)

Not only is this less succinct, but it also lacks exclusion, and passing arrays and hashes in is composable, but totally unreadable compared to json-mask. params uses this type of DSL with only/except/require, but doesn't seem to support array or hash values.

Is doing input validation a use-case for json-mask?  It seems like more of
a library for customizing output, not validating input.
On Fri, Mar 27, 2015 at 6:15 PM, Jordan Sexton<EMAIL_ADDRESS>wrote:

Coming from strong_parameters' builder-style DSL
http:///rails/strong_parameters#require-multiple-parameters, the form
might be:
require(:parent).permit(:children).require(:name)
Not only is this less succinct, but it also lacks exclusion, and passing
arrays and hashes in is composable, but totally unreadable compared to
json-mask. params http:///vesln/params uses this type of DSL with
only/except/require, but doesn't seem to support array or hash values.
—
Reply to this email directly or view it on GitHub
https://github.com/nemtsov/json-mask/issues/8#issuecomment-87106008.

While it's certainly most commonly used for customizing output (as in koa-json-mask), I would make the argument that customizing (and optionally validating) isn't outside the scope.

It doesn't seem like it would take much in terms of technical burden or performance to do accomplish both. Once compiled, this could be memoized (optional; not currently done). It requires adding support for ! before a symbol to the lexer, which is looked up in a simple hash, adding a flag around here for the next token, then checking for the flag in here.

Item 1 would need to be done anyway to support the first proposed !exclusion use case. If the default behavior is whitelisting props, it makes a certain degree of sense to provide for blacklisting. This could be expanded to support the require! use case by peeking at the stack and checking if the ! is before or after the token currently being evaluated.

Both are new features that don't break backwards compatibility.

It wouldn't be easy to extend json-mask to support the behavior without forking the project, because it requires modifying the compiler, which is efficient but not built for extensibility.

There is no node project I could find which currently does this well, and using ! imposes no new complexity for a user of json-mask, as the feature is just an extension to the mask's syntax.

Anyway, all this doesn't mean that this feature definitively falls in json-mask's scope, but to me it suggests there's a niche that could be filled and there are some people trying to do this with other solutions that aren't as powerful or succinct. What's are your thoughts against?

My thought was that there are other packages that do validation better,
with more options.  For example, https://github.com/Atinux/schema-inspector
Regardless I think this discussion has veered off track from the intended
issue, which was just to add a NOT mask, not to examine the entire scope of
the library :)
On Fri, Mar 27, 2015 at 7:39 PM, Jordan Sexton<EMAIL_ADDRESS>wrote:

While it's certainly most commonly used for customizing output (as in
koa-json-mask), I would make the argument that customizing (and optionally
validating) isn't outside the scope.

It doesn't seem like it would take much in terms of technical burden
or performance to do accomplish both. Once compiled, this could be memoized
(optional; not currently done). It requires adding support for ! before a
symbol to the lexer, which is looked up
http:///nemtsov/json-mask/blob/master/lib/compiler.js#L44 in a
simple hash, adding a flag around here
http:///nemtsov/json-mask/blob/master/lib/compiler.js#L89 for the
next token, then checking for the flag in here
http:///nemtsov/json-mask/blob/master/lib/filter.js#L21.
2.
Item 1 would need to be done anyway to support the first proposed
!exclusion use case. If the default behavior is whitelisting props, it
makes a certain degree of sense to provide for blacklisting. This could be
expanded to support the require! use case by peeking at the stack and
checking if the ! is before or after the token currently being evaluated.
3.
Both are new features that don't break backwards compatibility.
4.
It wouldn't be easy to extend json-mask to support the behavior
without forking the project, because it requires modifying the compiler,
which is efficient but not built for extensibility.
5.
There is no node project I could find which currently does this well,
and using ! imposes no new complexity for a user of json-mask, as the
feature is just an extension to the mask's syntax.
Anyway, all this doesn't mean that this feature definitively falls in
json-mask's scope, but to me it suggests there's a niche that could be
filled and there are some people trying to do this with other solutions
that aren't as powerful or succinct. What's are your thoughts against?
—
Reply to this email directly or view it on GitHub
https://github.com/nemtsov/json-mask/issues/8#issuecomment-87121925.

Agreed, I'm not talking about full type validation though. Just simple sanity validation ("Can I expect this deep prop to exist in this object?"). And schema-inspector is so verbose; there's no way to make it anywhere as succinct and expressive as json-mask. I use square/lgtm for full validation at the model layer, but it's pretty heavy for the resource layer I use json-mask at.
Anyway, I like your proposed feature.

@jordansexton apologies for not commenting on this sooner, I was out for the weekend. I agree with @loudwinston, json-mask isn't the best library for validation.
Even if we add the bang ! to the library that would be just key existence validation; which is only one type. For me, that isn't sufficient. When I validate I want to be thorough, so that anything downstream wouldn't need to re-validate. So, instead of just checking whether a key exists, I tend to check that it isn't empty or null, that it is an integer between 2 and 42 for example or that a key is boolean or a non-empty array. A library I use often for this type of validation is joi. It's designed for validation, and supports all of these use-cases via a fairly succinct api.
But if you're really interested in implementing validation with json-mask syntax, let's create a new library that would support:

json-mask syntax +
core types: validate(obj, 'address/city:string, isAdmin:boolean, config:object, friends:array')
defined types:

define('Person', 'name:string, age:number')
validate(obj, 'person:Person')

custom types:

define('EmptyString', function (value) { return '' === value })
validate(obj, 'name:EmptyString')

or / and: userInput:string|number, userInput:string&NotWhitespace
optional values: name:string, age:number? (name is required, age is optional [strict mode])
required values: name:string!, age:number (name is required, age is optional)

So, I hope you're still interested in implementing the NOT :)

The value of checking for the existence of properties at the resource layer is not safety and robustness but speed and ease of understanding. json-mask is fast and wouldn't be made appreciably slower by the check, and the syntax is easy to grep.
And, as I said, I was just looking for a property existence check in the syntax of json-mask, not type validation. I strongly prefer to place all type validation in the model layer and not the resource layer, closest to where they may be persisted, and there are libraries that do this asynchronously and with many features already. I have validators for all my models, so I'm not looking to recreate them, or implement them at the HTTP resource layer, as many are necessarily specific to the persistence mechanism (unique and foreign keys if persisted to the DB; file size and name if persisted to the FS) and should be somewhat abstracted away from the language of REST resources which often represent composite objects (Stripe's API is a great example of this).
While blacklisting properties makes some sense to be in scope as a counterpart to whitelisting, I always prefer to whitelist incoming data anyway rather than blacklist. It's safer both in the immediate sense of being strict about what you accept, and also safer for other developers who are working on the project and who may add properties to the model which could then be unknowingly output or allowed to be input at at the resource level.
Because of all this, I don't have much use for the blacklisting feature, so perhaps @loudwinston will come back to it. And perhaps what I'm thinking of isn't really the best use case for json-mask anyway. It was interesting poking around in the compiler, and while I thought it would make sense to do it with similar syntax, I respect your goal of keeping the library's focus tight.

Hi guys.
The NOT mask would be very interesting. Is anyone working on it already? I'm asking just to avoid doing duplicate work... I could give it a try if nobody is currently doing it...
Thanks!

Go for it!

:+1:


How to run two versions of the example application at the same time?

Thanks for all your work on this great project.  I'm testing out ecs-plugin to do blue-green deployments by running two versions of the example application at the same time, and eventually updating the DNS to point to the later version's load balancer.
This works great:
docker ecs compose --project-name=example up

however when I run
docker ecs compose --project-name=improved_example up

I get the error:
FrontendTCP5000TargetGroup already exists in stack

What's the simplest approach I can pursue so I can run both versions of the application at the same time, with hopefully no changes to the compose file itself?

That's unfortunate, I should have noticed this as I use to run multiple time the same stack in parallel to run tests while another stack is shutting down.
We actually don't have to give TargetGroups a Name, better just drop this.

see https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-elasticloadbalancingv2-targetgroup.html#cfn-elasticloadbalancingv2-targetgroup-name, we should just not set this attribute in CloudFormation stack


Skip unlinked files when creating tar partitions

Fixes #228. cc @fdr
Continuing this iteration when the exception is raised leads to an
unknown state where et_info holds the information of the previous
iteration.
This leads to re-uploading the previous file whenever an unlinked file
is processed.
It is OK (the comment above the code states it too) to skip and let the
WAL replay those.
Minimal example:
└─$  cat test.py
import errno
import os
import tarfile

paths = [
    "/home/leo/test", # This path exists and plays a regular PG file
    "/blah", # This path does not exist
]

bogus_tar = tarfile.TarFile(os.devnull, 'w', dereference=False)

for p in paths:
    try:
        t = bogus_tar.gettarinfo(p)
    except EnvironmentError, e:
        if (e.errno == errno.ENOENT and e.filename == p):
            print "boo %s" % p

    print t

└─$  python test.py
<TarInfo 'home/leo/test' at 0x7f68ee205750>
boo /blah
<TarInfo 'home/leo/test' at 0x7f68ee205750>

Committed in https://github.com/wal-e/wal-e/commit/6be466c4150738e0e368a46978b2c4aaa2ae75bb


retaining image repo names in kbld packaging workflow

When you package. I wonder why this does not do repo name to repo name map. Instead, it pushed into the same repo
https://github.com/k14s/kbld/blob/master/docs/packaging.md
``Images will be imported under a single new repository docker.io/dkalinin/app1. You are guaranteed that images are exactly same as they are referenced by the same digests in produced YAML configuration (though under a different repository name).```

how would you avoid naming conflicts between various image names (eg lots of things could have a image name called "webhook")?
i would also question need for retaining names since humans shouldn't be looking at image refs manually but rather rely on a tool to tell them what images are in use.

@cppforlife I thought this is based on the hash. So no conflicts?
My simply use case is that. I have many k8s manifests depends on registry 1. I want to package them locally and upload to the other offline/internal registry. And I can template/replace my manifest with the new registry url

I thought this is based on the hash. So no conflicts?

the digest does identify image uniquely however it's confusing that unrelated projects may end up using same repo name (e.g. my-private-reg.com/dkalinin/webhook@sha256:..., in this case, whose webhook are we talking about) since their name could be the same. with current behaviour since all images referenced by config are uploaded to a single repo, you can for example delete that repo when you no longer need that project (without guessing which images to delete).

My simply use case is that. I have many k8s manifests depends on registry 1. I want to package them locally and upload to the other offline/internal registry.

that's exactly how kbld package/unpackage is meant to be used for.

And I can template/replace my manifest with the new registry url

instead of manually templating things we expect people to rely on kbld to update image references with new urls. docs/packaging.md shows that as an example. any reason why that doesnt work for you?

cool I got your point @cppforlife basically using kbld and the manifest.lock (one generated during unpack) to interoperate the manifest files again.

that's right.

closing. question was answered it seems.


Allow for user defined prometheus job template

In lieu of providing a way to generate prometheus job configuration based on chef-search based discovery we should allow for users of this cookbook to define their own prometheus job template.

Added here: https://github.com/rayrod2030/chef-prometheus/pull/13


Correct Test 1.14 DM26 Enable Bit Outcomes

Correct Test 1.14 DM26 Enable Bit Outcomes
Enable status settings are incorrectly identified as errors in part 1 test 14.
Details provided below in explanation to CARB.

Correct Test 1.14 DM26 Enable Bit Outcomes
Dear Dan and Paul,One of the reasons I’ve been struggling with DM26 is the overly complex language in Part 1 Test 14. This was complicated by incorrect outcomes, which I identified to the software developer yesterday as shown below.  My conclusion is that the text for unsupported monitors, as identified in DM5, needed to be simplified.  The revised text will provide the same outcomes intended by the current language, which Allen Lyons and I wrote about 8 years ago.  The current language for part 1 test 14 does not evaluate enabled bit content.  This I believe to be correct.  The examples for enable bit values, given in J1939-73 5.7.29, illustrate test conditions driven by manufacturers’ implementations that cannot be reasonably controlled in the J1939-84 text. 1. Monitor enable status for the current driving cycle. This bit shall indicate when a monitor is disabled in a manner such that there is no likely or reasonable way for the driver to operate the vehicle to allow the monitor run. Typical examples are:• Engine-off soak not long enough (e.g., cold start temperature conditions not satisfied)• Monitor maximum time limit or number of attempts/aborts exceeded• Ambient air temperature too low or too high• BARO too low (high altitude)The monitor shall not indicate “disabled” for operator-controlled conditions such as rpm, load, accelerator position, minimum time limit not exceeded, etc. 2. Monitor completion status for the current driving/monitoring cycle. Status shall be reset to “not complete” upon starting a new monitoring cycle. Note that some monitoring cycles can include various engine-operating conditions; other monitoring cycles begin after the ignition key is turned off. Some status bits on a given vehicle can utilize engine-running monitoring cycles while others can utilize engine-off monitoring cycles. Resetting the bits to “not complete” upon starting the engine will accommodate most engine-running and engine-off monitoring cycles; however, manufacturers are free to define their own monitoring cycles. NOTE: DM26 bits shall be utilized for all non-continuous monitors which are supported, and change completion status in DM5. If a non-continuous monitor is not supported or always shows “complete,” the corresponding DM26 bits shall indicate disabled and complete. DM26 bits may be utilized at the vehicle manufacturer’s discretion for all continuous monitors which are supported with the exception of the bit that shall always show CCM (comprehensive component monitoring) as enabled for spark ignition and compression ignition engines. I’ll see if I can’t detangle the part 2 DM29 language soon.Sincerely Yours,Eric Swenson ————— Text to software developer below.Dear ...,When I read this, I believe that the enable bit for a monitor supported in DM5 is permitted to say disabled or enabled for the current cycle in DM26. The failure criteria for unsupported monitors, does prohibit the enabled setting. Disable is prohibited for the CCM enable bit (bit 3 of SP 3303).<IP_ADDRESS> Fail Criteria a. Fail if any response for any monitor supported in DM5 by a given ECU is reported as “0=monitor complete this cycle or not supported” in SP 3303 bits 1-4 and SP 3305 [except comprehensive components monitor (CCM)].  b. Fail if any response for each monitor not supported in DM5 by a given ECU is not also reported in DM26 as “0=monitor complete this cycle or not supported” in SP 3303 bits 5-7 and [also reported in DM26 as] “0=monitor disabled for rest of this cycle or not supported” in SP 3303 bits 1 and 2 and SP 3304. [23]c. Fail if any response from an ECU indicating support for CCM monitor in DM5 reports “0=monitor disabled for rest of this cycle or not supported” in SP 3303 bit 3.  d. Fail if any response indicates number of warm-ups since code clear (WU-SCC) (SP 3302) is not zero.  e. Fail if any response indicates time since engine start (SP 3301) is not zero. f. Fail if no OBD ECU provides DM26. a-c are summarized below
| DM5 Support | DM26 Enable | DM26 Complete | Outcome
-- | -- | -- | -- | --
<IP_ADDRESS>.a | Supported | Any | Complete | Fail
<IP_ADDRESS>.b | Not Supported | Any | Not Complete | Fail
<IP_ADDRESS>.b | Not Supported | Enabled | Any | Fail
<IP_ADDRESS>.c | CCM Supported | Disabled | Any | Fail
 So when I look at  07:17:42.1013 Global DM26 Request07:17:42.1088 18EAFFF9 [3] B8 FD 00 (TX)07:17:42.1111 18FDB800 [8] 00 00 00 37 80 12 E0 1EDM26 from Engine #1 (0): Warm-ups: 0, Time Since Engine Start: 0 seconds Vehicle Composite of DM26:    A/C system refrigerant     not enabled,     complete    Boost pressure control sys     enabled, not complete    Catalyst                   not enabled,     complete    Cold start aid system      not enabled,     complete    Comprehensive component        enabled,     complete    Diesel Particulate Filter  not enabled, not complete    EGR/VVT system                 enabled, not complete    Evaporative system         not enabled,     complete    Exhaust Gas Sensor         not enabled, not complete    Exhaust Gas Sensor heater  not enabled, not complete    Fuel System                    enabled, not complete    Heated catalyst            not enabled,     complete    Misfire                        enabled, not complete    NMHC converting catalyst       enabled, not complete    NOx catalyst/adsorber      not enabled, not complete    Secondary air system       not enabled,     complete FAIL: <IP_ADDRESS>.a - Engine #1 (0) response for a monitor Diesel Particulate Filter in DM5 is reported as supported and is reported as not enabled by DM26 responseFAIL: <IP_ADDRESS>.a - Engine #1 (0) response for a monitor Exhaust Gas Sensor in DM5 is reported as supported and is reported as not enabled by DM26 responseFAIL: <IP_ADDRESS>.a - Engine #1 (0) response for a monitor Exhaust Gas Sensor heater in DM5 is reported as supported and is reported as not enabled by DM26 responseFAIL: <IP_ADDRESS>.a - Engine #1 (0) response for a monitor NOx catalyst/adsorber in DM5 is reported as supportedI’ve come to the conclusion that the disabled prohibition was applied in a. to the supported monitors (as defined in DM5) in error. Here the cited supported monitors properly show ‘not complete’ for a.I find that my prior acceptance of these and similar error messages was a mistake.Do you agree?My recommendation is to rewrite a-c as a-d by splitting and simplifying the word salad in b.<IP_ADDRESS> Fail Criteria a. Fail if any response for any monitor supported in DM5 by a given ECU is reported as “0=monitor complete this cycle or not supported” in SP 3303 bits 1-4 and SP 3305 [except comprehensive components monitor (CCM)].  // After a reset (here DM11), all completed monitors are reset to not complete.b. Fail if any response for each monitor not supported in DM5 by a given ECU is also reported in DM26 as “1=monitor not complete this monitoring cycle” in SP 3303 bits 5-7.// Unsupported monitors will never need to run and are ‘de facto’ complete.c. Fail if any response for each monitor not supported in DM5 by a given ECU is also reported in DM26 as “0=monitor enabled for this monitoring cycle” in SP 3303 bits 1 and 2 and SP 3304. [23]  // Unsupported monitors will never run and cannot be enabled.d. Fail if any response from an ECU indicating support for CCM monitor in DM5 reports “0=monitor disabled for rest of this cycle or not supported” in SP 3303 bit 3.  // For now, CCM is always to be shown as complete.I think you will agree these are easier to read.Best Wishes,Eric

The existing text is technically correct, but too hard to read.