[CognitiveServices - HealthInsights] SDK for JS

Packages impacted by this PR
Health Insights CancerProfilingRest and ClinicalMatchingRest (new packages).
Issues associated with this PR
[Cognitive Services - Health Decision Support] REST API Review · Issue #5731 · Azure/azure-rest-api-specs-pr (github.com)
Describe the problem that is addressed by this PR
What are the possible designs available to address the problem? If there are more than one possible design, why was the one in this PR chosen?
Provide a list of related PRs (if any)
[Cognitive Services - Health Insights] CADL revision for public preview by asaflevi-ms · Pull Request #22990 · Azure/azure-rest-api-specs (github.com)

cc  @Azure/dpg-devs

/azp run prepare-pipelines

/azp run js - healthinsights - mgmt - ci

API change check
APIView has identified API level changes in this PR and created following API reviews.
azure-rest-health-insights-cancerprofiling

cc  @Azure/dpg-devs

/azp run prepare-pipelines

The CI is failing are because of two reasons:

there's some lint errors of the code. you can run rushx lint --fix in each project folder.
there's some mismatch in the test recodings. I think you can use record mode to rerun the test and then submit the changes of recodings folder to fix it.

@reutgross you can put the in progress link to the https://github.com/Azure/azure-sdk-for-js/blob/main/eng/ignore-links.txt so that the CI can pass. But I don't recommend to put the samples related links here.

API change check
APIView has identified API level changes in this PR and created following API reviews.
azure-rest-health-insights-cancerprofiling azure-rest-health-insights-clinicalmatching

@joheredi, as this is the first preview of HealthInsights SDK, I think the signoff of the APIView is required for the SDK release.
Could you help to take at the APIView or would you suggest someone else to signoff?


context.params.slug is not an array if U provide top level route

🐛 Bug Report
const { render } = await getPage({
      route: "/somerandomurl",
    });

results in context.params.slug being a string "somerandomurl" but if U use "/some/thing" as a route it will be an array
Expected behavior
Slug should be an array even if there is only one element

Hi @tomaszrondio!
What's the pathname of the relevant page file?

Hi @tomaszrondio!
What's the pathname of the relevant page file?

I have [...slug].jsx that catches pretty much all routes.

I have [...slug].jsx that catches pretty much all routes.

Found the issue. Current implementation has some intrinsic limitations which requires a bit of time and refactoring. I'll try to work on it this weekend.

Found the issue. Current implementation has some intrinsic limitations which requires a bit of time and refactoring. I'll try to work on it this weekend.


raylib: support macOS

Description of changes
Allows raylib to be built on Darwin systems.

Things done

Built on platform(s)

[x] x86_64-linux
[x] aarch64-linux
[x] x86_64-darwin
[x] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[x] sandbox = true

[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 263836 run on aarch64-darwin 1

2 packages built:

raylib
raylib-games


Fix wrong precedence check for double arrow operator

Purpose

Fixes #23621

Approach

Update parsing of right associative expressions by lowering the precedence

Samples

N/A

Remarks

N/A

Check List

[x] Read the Contributing Guide
[ ] Updated Change Log
[ ] Checked Tooling Support (#)
[ ] Added necessary tests

[x] Unit Tests
[ ] Spec Conformance Tests
[ ] Integration Tests
[ ] Ballerina By Example Tests

[ ] Increased Test Coverage
[ ] Added necessary documentation

[ ] API documentation
[ ] Module documentation in Module.md files
[ ] Ballerina By Examples

Can you add a simple parser-test too?
Refer the last section in this doc on 'how to': https://docs.google.com/document/d/1Wv1FJ6JJbGqje6bXSBuJI6fNPbikFd0CreS83JO93lk/edit#heading=h.tedq9q3qfv0f

Can you add a simple parser-test too?
Refer the last section in this doc on 'how to': https://docs.google.com/document/d/1Wv1FJ6JJbGqje6bXSBuJI6fNPbikFd0CreS83JO93lk/edit#heading=h.tedq9q3qfv0f

Added in the latest commit.


At the beginning of each new game-round the correct players have the small and big blind values deduced from their score and added to the pot.

Closed in commit: 236f74a333977cc6dc0c5824c7535abde7b33244


nil_dirichlet condition variable on input tensor.

The objective
I am trying to accurately model the heat system of a fridge and I was wondering if it is at all possible to make the boundary condition act conditionally on whether the fridge door is open or closed since these would be different boundary conditions ofcourse.
class Heat2D(TimeDependentProblem, SpatialProblem):

    # Define these yourself
    LENGTH_X = 82
    LENGTH_Y = 70
    DURATION = 2284

    output_variables = ['u']
    spatial_domain = Span({'x': [0, LENGTH_X], 'y': [0, LENGTH_Y]})
    temporal_domain = Span({'t': [0, DURATION]})

    def heat_equation_2D(input_, output_):
        '''1'''
        # c is thermal diffusivity, variates for different materials so google
        c = (0.01/torch.pi) ** 0.5

        du = grad(output_, input_)
        ddu = grad(du, input_, components=['dudx','dudy'])
        return (
            du.extract(['dudt']) -
            (c**2)*(ddu.extract(['ddudxdx']) + ddu.extract(['ddudydy']))
        )

    def nil_dirichlet_x(input_, output_):
        '''2 and 3'''
        du = grad(output_, input_)
        u_expected_boundary = 0.0
        return du.extract(['dudx']) - u_expected_boundary

    def nil_dirichlet_yL(input_, output_):
        '''4'''
        du = grad(output_, input_)
        u_expected_boundary = 0.0
        return du.extract(['dudy']) - u_expected_boundary

    def nil_dirichlet_y0(input_, output_):
        '''5'''
        # TODO: make this conditionally on if door is open
        du = grad(output_, input_)
        u_expected_boundary = 0.0
        return du.extract(['dudy']) - u_expected_boundary

    def initial_condition(input_, output_):
        u_expected_initial = torch.sin(torch.pi*input_.extract(['x']))
        return output_.extract(['u']) - u_expected_initial

    conditions = {
        'boundx0': Condition(location=Span({'x': 0, 'y': [0, LENGTH_Y], 't': [0, DURATION]}), function=nil_dirichlet_x),
        'boundxL': Condition(location=Span({'x': LENGTH_X, 'y': [0, LENGTH_Y], 't': [0, DURATION]}), function=nil_dirichlet_x),
        'boundy0': Condition(location=Span({'x': [0, LENGTH_X], 'y': 0, 't': [0, DURATION]}), function=nil_dirichlet_y0),
        'boundyL': Condition(location=Span({'x': [0, LENGTH_X], 'y': LENGTH_Y, 't': [0, DURATION]}), function=nil_dirichlet_yL),
        'initial': Condition(location=Span({'x': [0, LENGTH_X], 'y': [0, LENGTH_Y], 't': 0}), function=initial_condition),
        'heat_eq': Condition(location=Span({'x': [0, LENGTH_X], 'y': [0, LENGTH_Y], 't': [0, DURATION]}), function=heat_equation_2D),
        'data': Condition(input_points=input_tensor , output_points=output_tensor),
        }

This is the code I am currently running and the nil_dirichlet_y0 is the side of the door. In my input_tensor I use in data I have a column with Door either 0 or 1, respectively Closed and Open, and a column with the temperature outside of the fridge called Environment.
I was thinking something like this
    def nil_dirichlet_y0(input_, output_):
        '''5'''
        if input_tensor.extract(['Door']) == 0:
            du = grad(output_, input_)
            u_expected_boundary = 0.0
            return du.extract(['dudy']) - u_expected_boundary
        else if input_tensor.extract(['Door']) == 1:
            u_expected_boundary = input_tensor.extract(['Environment'])
            return input_.extract(['u']) - u_expected_boundary

But after trying to train the model I ofcourse get an error that occurs because I am trying to compare the entire Door tensor at once instead of the values one at a time.
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[60], line 22
     17 pinn.span_pts(
     18     {'n': 200, 'mode': 'grid', 'variables': 't'},
     19     {'n': 20, 'mode': 'grid', 'variables': ['x', 'y']},
     20     locations=['heat_eq'])
     21 pinn.span_pts(150, 'random', locations=['boundx0', 'boundxL', 'initial', 'boundyL', 'boundy0'])
---> 22 pinn.train(10, 1)

File ~/opt/anaconda3/lib/python3.8/site-packages/pina/pinn.py:252, in PINN.train(self, stop, frequency_print, save_loss, trial)
    250 predicted = self.model(pts)
    251 for function in condition.function:
--> 252     residuals = function(pts, predicted)
    253     local_loss = (
    254         condition.data_weight*self._compute_norm(
    255             residuals))
    256     single_loss.append(local_loss)

Cell In[58], line 38, in Heat2D.nil_dirichlet_y0(input_, output_)
     36 def nil_dirichlet_y0(input_, output_):
     37     '''5'''
---> 38     if input_tensor.extract(['Door']) == 0:
     39         du = grad(output_, input_)
     40         u_expected_boundary = 0.0

File ~/opt/anaconda3/lib/python3.8/site-packages/torch/_tensor.py:1295, in Tensor.__torch_function__(cls, func, types, args, kwargs)
   1292     return NotImplemented
   1294 with _C.DisableTorchFunctionSubclass():
-> 1295     ret = func(*args, **kwargs)
   1296     if func in get_default_nowrap_functions():
   1297         return ret

RuntimeError: Boolean value of Tensor with more than one value is ambiguous

Is there any way to do this?

Hi👋🏻 @Bovhasselt , and sorry for the late reply. I am not very sure you can do this operation with torch in general. By assuming a value is binary, I think you will break the gradient due to discontinuity. A nice workaround could be to process data in batches with the same kind of Door value. By doing this you could call torch.all and obtain a different behaviour. Let me know if it helps😄

Hi @Bovhasselt I will close the issue since it has been opened for more than 5 months. Feel free to reopen it if you need extra help!


substring /virtualNetworks/ not found in subnet_id

azureml-core:
1.54.0.post1:
Windows:
Python 3.8:

Describe the bug
I'm trying to print workspace.compute_targets, but I'm getting the following error:

In the file computeinstance.py, class ComputeInstance, function _initialize(), line 121, the code is trying to extract the information of virtualNetworks from the variable subnet_id. But the information about virtualNetworks inside subnet_id has the letter v capitalized for me: VirtualNetworks
To Reproduce
Try to get the compute_networks of a workspace with virtual network.
Expected behavior
Print all compute_targets without errors.
Suggestion
Change vnet_name = subnet_id[subnet_id.index("/virtualNetworks/") + len("/virtualNetworks/"):subnet_id.index("/subnets")] to vnet_name = subnet_id[subnet_id.index("/VirtualNetworks/") + len("/VirtualNetworks/"):subnet_id.index("/subnets")]

Thanks for the feedback I added this t our internal backlog for prioritization.

Hi @julio-cmdr ,
Tried to reproduce this issue but unable to reproduce.
Could you please check the Network Security Group (NSG) associated with the subnet or network.
Inspect the inbound and outbound rules in the NSG to understand firewall settings.

HI @julio-cmdr,
Could you please edit and save subnet. If edit and save not working then try with delete and recreate if possible.

HI @julio-cmdr,
Could you please update about above suggestion, Did above approach working for you?

Hi @isaudagar.
Thanks for your reply. Unfortunally, I'm using a resource group that I don't have access to manage the subnets.
To carry out your suggestion to edit and save the subnet (or maybe recreate it), I believe I will need to open an internal ticket, which will probably not be answered, considering the impacts that will be generated by modifying the subnet.
Were you able to reproduce the errors?

Hi @julio-cmdr, Thanks for response. We tried multiple ways to reproduce but not able to reproduce.

No, the problem has not been resolved. I still have the same problem.

Hi @julio-cmdr, As per my investigation that substring is case sensitive in Python.
subnet resource ID has lowercase "/virtualnetworks/" and the you are searching for index of  "/virtualNetworks/".
please use lowercase and update us.

Hi @isaudagar
I'm just trying to list workspace.compute_targets in my code. The problem is within the azureml-core library. The source of the error is described in this issue.

Hi @julio-cmdr,
Could you please explain about below point letter v capitalized, how did you capitalized?
substring is case sensitive in Python and due to that reason getting error.

This is the original source code of the library azureml-core, with letter v in lower case:

After editing the library in my local environment, capitalizing letter v, I got it working. But I'm reporting it here so that it can be fixed for all users.
If you prefer, we can have a meeting on Microsoft Teams. I am working in the GMT-3 time zone.

Hi @julio-cmdr, Above code is working for all users if we are making this change it will break for them . I want to know how and why it is v letter capital for you. Could you please share your teams id to connect.
My teams ID<EMAIL_ADDRESS>

Thank you for reporting this issue, This is now fixed and will be released in the next update to azureml-core package.

Thank you, @isaudagar!

After updating to azureml-core==1.55.0.post2, it's working!


PE-4192: hide the Add button for native android and ios

Changes

Hides the button for native apps, so it's only enabled on browser

--- Releases ---
Android release: https://appdistribution.firebase.google.com/testerapps/1:305132849030:android:6cf0cd5ec064fad3ffce07/releases/0mrdhfsojf0f0
iOS release: https://appdistribution.firebase.google.com/testerapps/1:305132849030:ios:06ea33a95a2e0d42ffce07/releases/6p6t2c4lmj9c8

The ticket said to disable the button and that's what you did, but in fact we want to hide it

Change in requirements. Now hiding the button instead of disabling it 👍


Use stronger check for PropertyEffects clients. Fixes #5017

Check for private variable __isPropertiesEffectsClient to ensure a client is opting into synchronous batching and path notifications.
Reference Issue
Fixes #5017

I think you need to update your local version of the gen-typescript-declarations. It is pulling in changes that shouldnt be there post 1.0.0


C# 7 §13.10.5 (The return statement) wrongly references §10.5 (User-defined conversions)

Describe the bug
In the C# 7 draft,  §13.10.5 (Statements / Jump statements / The return statement) contains this text:

The return statement returns control to the current caller of the function member in which the return statement appears, optionally returning a value or a variable_reference (§10.5).

but §10.5 (Conversions / User-defined conversions) does not describe variable_reference.
Example
Not applicable.
Expected behavior
The reference should be to §9.5 (Variables / Variable references), instead.
Additional context
The reference to §10.5 was already incorrect when it was added in https://github.com/dotnet/csharpstandard/commit/2565ad2debc9726360b9959a677ea85d4fd2a484; the mismatch was not caused by any subsequent renumbering.  I don't think it is necessary to check the other references that came to draft-v7 via https://github.com/dotnet/csharpstandard/pull/213 and https://github.com/dotnet/csharpstandard/pull/795.

Added PR #829 to fix this typo.


Add tabbed view for example in landing page

We need a tabbed view for the example on the landing page, like on the Node.js page (see https://googlecloudplatform.github.io/gcloud-node/).  One tab should be devoted to Compute Engine because the user does not have to provide the project ID and auth code.  Other platforms require the user to provide additional information.

Resolved by #191


Fix default sort order bug

This is a feature addition and cannot be merged. Good job on the code though @tyouwei !


Manage qtree names as '/volume/qtreename' instead of just 'qtreename' #77

Changed property "name" so it can be "/volume/qtreename" as qtree are unique at volume level not at SVM level. Refet to #77

The old behaviour with still work (calling with name/volume).
New behaviour will be used passing a "/volume/qtreename" or using property "volume" and property "qtname".
Changed property "volume" to default to volume name is it's specified in the "name" property.
Added a new property "qtname" that will default to the qtree name specified in the "name" property (or the * "name" property itself for the hold behaviour).
Used "qtname" instead of "name" in the qtree management functions.
Added tests for netapp_qtree type.

What's the status of this PR?
This is a quite useful, almost required, addition for anybody wanting to multiple netapp volumes on the same SVM via Puppet.

Hi,
from the name parameter:
unless (value =~ /^[\w-]+$/) or (value =~ /^/\w+/\w+$/)
I think qtrees can have dots and hyphens in their name.
  unless (value =~ /^[\w\-]+$/) or (value =~ /^\/\w+\/[\w\-.]+$/)

perhaps?

It's correct.
Adding "-" and "." as allowed character to the qtree name.

@hunner
It seems that still don't work.
I had to change ew
I'm testing the change with
    netapp_qtree { 'testtitleonly':
      ensure => present,
      volume => 'voltst11',
    }
    netapp_qtree { 'testnameonly':
      ensure => present,
      qtname => 'testname',
      volume => 'voltst11',
    }
    netapp_qtree { '/voltst11/test1':
      ensure => present,
      qtname => 'test2',
      volume => 'voltst11',
    }
    netapp_qtree { '/voltst12/test2':
      ensure => present,
      qtname => 'test2',
      volume => 'voltst12',
    }

Trying i run i get this error in the debug log somewhere related to one of the 4 resource (probably because it see something like duplicated):
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: undefined method call' for :qtname:Symbol
`
On the puppetserver I get this errors:
2017-06-02 14:19:16,340 ERROR [qtp200232015-70063] [puppetserver] Puppet undefined method `call' for :qtname:Symbol
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource.rb:708:in `parse_title'
org/jruby/RubyArray.java:1613:in `each'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource.rb:697:in `parse_title'
org/jruby/RubyArray.java:1613:in `each'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource.rb:695:in `parse_title'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource.rb:457:in `to_hash'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource.rb:105:in `to_data_hash'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:486:in `to_data_hash'
org/jruby/RubyArray.java:2409:in `collect'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:486:in `to_data_hash'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/puppetdb.rb:53:in `munge_catalog'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/puppetdb.rb:99:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/puppetdb.rb:52:in `munge_catalog'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/puppetdb.rb:99:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/puppetdb.rb:51:in `munge_catalog'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/puppetdb.rb:13:in `save'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/puppetdb.rb:99:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/puppetdb.rb:11:in `save'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/store_configs.rb:24:in `save'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/indirection.rb:200:in `find'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/indirected_routes.rb:121:in `do_find'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/indirected_routes.rb:48:in `call'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:294:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/indirected_routes.rb:47:in `call'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:82:in `process'
org/jruby/RubyArray.java:1613:in `each'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:81:in `process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:87:in `process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:87:in `process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:60:in `process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:58:in `process'
file:/opt/puppetlabs/server/apps/puppetserver/puppet-server-release.jar!/puppetserver-lib/puppet/server/master.rb:42:in `handleRequest'
Puppet$$Server$$Master_1292261758.gen:13:in `handleRequest'
request_handler_core.clj:273:in `invoke'
jruby_request.clj:46:in `invoke'
jruby_request.clj:31:in `invoke'
request_handler_service.clj:34:in `handle_request'
request_handler.clj:3:in `invoke'
request_handler.clj:3:in `invoke'
core.clj:2515:in `invoke'
ring_middleware.clj:284:in `invoke'
core.clj:168:in `invoke'
core.clj:211:in `invoke'
core.clj:45:in `invoke'
core.clj:357:in `invoke'
core.clj:51:in `invoke'
ringutils.clj:83:in `invoke'
master_core.clj:426:in `invoke'
ring.clj:21:in `invoke'
ring.clj:12:in `invoke'
comidi.clj:249:in `invoke'
http.clj:152:in `invoke'
http.clj:152:in `invoke'
http.clj:148:in `invoke'
comidi.clj:336:in `invoke'
jetty9_core.clj:423:in `invoke'
normalized_uri_helpers.clj:81:in `invoke'
2017-06-02 14:19:16,340 ERROR [qtp200232015-70063] [puppetserver] Puppet Server Error: undefined method `call' for :qtname:Symbol
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource.rb:708:in `parse_title'
org/jruby/RubyArray.java:1613:in `each'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource.rb:697:in `parse_title'
org/jruby/RubyArray.java:1613:in `each'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource.rb:695:in `parse_title'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource.rb:457:in `to_hash'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource.rb:105:in `to_data_hash'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:486:in `to_data_hash'
org/jruby/RubyArray.java:2409:in `collect'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:486:in `to_data_hash'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/puppetdb.rb:53:in `munge_catalog'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/puppetdb.rb:99:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/puppetdb.rb:52:in `munge_catalog'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/puppetdb.rb:99:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/puppetdb.rb:51:in `munge_catalog'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/puppetdb.rb:13:in `save'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/puppetdb.rb:99:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/catalog/puppetdb.rb:11:in `save'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/store_configs.rb:24:in `save'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/indirector/indirection.rb:200:in `find'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/indirected_routes.rb:121:in `do_find'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/indirected_routes.rb:48:in `call'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:65:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:294:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/api/indirected_routes.rb:47:in `call'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:82:in `process'
org/jruby/RubyArray.java:1613:in `each'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:81:in `process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:87:in `process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/route.rb:87:in `process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:60:in `process'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler/around_profiler.rb:58:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/profiler.rb:51:in `profile'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/network/http/handler.rb:58:in `process'
file:/opt/puppetlabs/server/apps/puppetserver/puppet-server-release.jar!/puppetserver-lib/puppet/server/master.rb:42:in `handleRequest'
Puppet$$Server$$Master_1292261758.gen:13:in `handleRequest'
request_handler_core.clj:273:in `invoke'
jruby_request.clj:46:in `invoke'
jruby_request.clj:31:in `invoke'
request_handler_service.clj:34:in `handle_request'
request_handler.clj:3:in `invoke'
request_handler.clj:3:in `invoke'
core.clj:2515:in `invoke'
ring_middleware.clj:284:in `invoke'
core.clj:168:in `invoke'
core.clj:211:in `invoke'
core.clj:45:in `invoke'
core.clj:357:in `invoke'
core.clj:51:in `invoke'
ringutils.clj:83:in `invoke'
master_core.clj:426:in `invoke'
ring.clj:21:in `invoke'
ring.clj:12:in `invoke'
comidi.clj:249:in `invoke'
http.clj:152:in `invoke'
http.clj:152:in `invoke'
http.clj:148:in `invoke'
comidi.clj:336:in `invoke'
jetty9_core.clj:423:in `invoke'
normalized_uri_helpers.clj:81:in `invoke'

Changing qtname to newproperty give me more different errors for the duplicate "volume name:
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Resource Statement, Cannot alias Netapp_qtree[testnameonly] to ["voltst11"] at /etc/puppetlabs/code/environments/fduranti_devel/site/profile/manifests/storage/netapp.pp:19; resource ["Netapp_qtree", "voltst11"] already declared at /etc/puppetlabs/code/environments/fduranti_devel/site/profile/manifests/storage/netapp.pp:15 at /etc/puppetlabs/code/environments/fduranti_devel/site/profile/manifests/storage/netapp.pp:19:5 on node itnapsvmtest01.q8int.com
Info: Using cached catalog from environment 'fduranti_devel'
Debug: #<Class:0x00000004ab89d0>: false value when expecting true
Error: Failed to apply catalog: Cannot alias Netapp_qtree[testnameonly] to ["voltst11"] at /etc/puppetlabs/code/environments/fduranti_devel/site/profile/manifests/storage/netapp.pp:19; resource ["Netapp_qtree", "voltst11"] already declared at /etc/puppetlabs/code/environments/fduranti_devel/site/profile/manifests/storage/netapp.pp:15

I had to change volume to newproperty too to have a run without this problems but the existing volume are still not recognized.
On my SVM Initially there are only 3 qtree that are:
/voltst11/qt1 /voltst12/qt2 /voltst12/test2
So the first run should be able to create 3 new qtree but it don't find the 4th and try to recreate it so it seems that something on the exists? is not working correctly (the property_hash is not populated) or the prefetch is not putting the right resource... I'm expecting that in the part  where there's the "Debug: Returning qtrees array. " and "Debug: Prov.name = . " the third one that correspond to "/voltst12/test2" should report the Netapp_Qtree['/voltst12/test2'].
Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Checking if this is an actual qtree, not a volume.
Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Checking if this is an actual qtree, not a volume.
Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Checking if this is an actual qtree, not a volume.
Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Processing rule for qtree 'qt1'.
Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Volume for 'qt1' is 'voltst11'.
Debug: Creating instance for 'qt1'.

Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Checking if this is an actual qtree, not a volume.
Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Checking if this is an actual qtree, not a volume.
Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Processing rule for qtree 'qt2'.
Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Volume for 'qt2' is 'voltst12'.
Debug: Creating instance for 'qt2'.

Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Checking if this is an actual qtree, not a volume.
Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Processing rule for qtree 'test2'.
Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Volume for 'test2' is 'voltst12'.
Debug: Creating instance for 'test2'.

Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Checking if this is an actual qtree, not a volume.
Debug: Processed all qtree instances.
Debug: Returning qtrees array.
Debug: Prov.name = .
Debug: Prov.name = .
Debug: Prov.name = .
Debug: Puppet::Provider::Netapp_qtree.cmode: checking existance of Netapp qtree testtitleonly against volume voltst11
Debug: Value =
Debug: Puppet::Provider::Netapp_qtree.cmode: creating Netapp Qtree testtitleonly on volume voltst11.
Debug: apicommand is a - String
Debug: Args array length = 4
Debug: Executing api call qtree-create qtree testtitleonly volume voltst11
Debug: Facter: custom fact "url" was not found.
Debug: Puppet::Provider::Netapp_qtree.cmode: Qtree testtitleonly created successfully on volume voltst11.

Notice: /Stage[main]/Profile::Storage::Netapp/Netapp_qtree[testtitleonly]/ensure: created
Debug: Puppet::Provider::Netapp_qtree.cmode: Got to flush for resource testtitleonly.
Debug: Property_hash ensure =
Debug: /Stage[main]/Profile::Storage::Netapp/Netapp_qtree[testtitleonly]: The container Class[Profile::Storage::Netapp] will propagate my refresh event
Debug: Puppet::Provider::Netapp_qtree.cmode: checking existance of Netapp qtree testname against volume voltst11
Debug: Value =
Debug: Puppet::Provider::Netapp_qtree.cmode: creating Netapp Qtree testname on volume voltst11.
Debug: apicommand is a - String
Debug: Args array length = 4
Debug: Executing api call qtree-create qtree testname volume voltst11
Debug: Facter: custom fact "url" was not found.
Debug: Puppet::Provider::Netapp_qtree.cmode: Qtree testname created successfully on volume voltst11.

Notice: /Stage[main]/Profile::Storage::Netapp/Netapp_qtree[testnameonly]/ensure: created
Debug: Puppet::Provider::Netapp_qtree.cmode: Got to flush for resource testname.
Debug: Property_hash ensure =
Debug: /Stage[main]/Profile::Storage::Netapp/Netapp_qtree[testnameonly]: The container Class[Profile::Storage::Netapp] will propagate my refresh event
Debug: Puppet::Provider::Netapp_qtree.cmode: checking existance of Netapp qtree test2 against volume voltst11
Debug: Value =
Debug: Puppet::Provider::Netapp_qtree.cmode: creating Netapp Qtree test2 on volume voltst11.
Debug: apicommand is a - String
Debug: Args array length = 4
Debug: Executing api call qtree-create qtree test2 volume voltst11
Debug: Facter: custom fact "url" was not found.
Debug: Puppet::Provider::Netapp_qtree.cmode: Qtree test2 created successfully on volume voltst11.

Notice: /Stage[main]/Profile::Storage::Netapp/Netapp_qtree[/voltst11/test1]/ensure: created
Debug: Puppet::Provider::Netapp_qtree.cmode: Got to flush for resource test2.
Debug: Property_hash ensure =
Debug: /Stage[main]/Profile::Storage::Netapp/Netapp_qtree[/voltst11/test1]: The container Class[Profile::Storage::Netapp] will propagate my refresh event
Debug: Puppet::Provider::Netapp_qtree.cmode: checking existance of Netapp qtree test2 against volume voltst12
Debug: Value =
Debug: Puppet::Provider::Netapp_qtree.cmode: creating Netapp Qtree test2 on volume voltst12.
Debug: apicommand is a - String
Debug: Args array length = 4
Debug: Executing api call qtree-create qtree test2 volume voltst12
Debug: Facter: custom fact "url" was not found.
Error: Executing api call qtree-create qtree test2 volume voltst12 failed: "Failed to create qtree \"test2\" in volume \"voltst12\" Vserver \"ITNAPSVMTEST01\". Reason: Qtree (or directory) \"test2\" already exists."
Error: /Stage[main]/Profile::Storage::Netapp/Netapp_qtree[/voltst12/test2]/ensure: change from absent to present failed: Executing api call qtree-create qtree test2 volume voltst12 failed: "Failed to create qtree \"test2\" in volume \"voltst12\" Vserver \"ITNAPSVMTEST01\". Reason: Qtree (or directory) \"test2\" already exists."

I'll try to add some more debug print and do some additional test in the afternoon.

Thanks for the example code. I'll get that into an acceptance test too, since it looks like there isn't anything for qtrees yet.

Ok some more tests done:
I've added some debug output for getting the list of resources from the self.prefetch this way:
    def self.prefetch(resources)
      Puppet.debug("Puppet::Provider::Netapp_qtree.cmode: Got to self.prefetch.")
      # Itterate instances and match provider where relevant.
      qtrees_list=instances
      resources.each do |name, res|
        Puppet.debug("name = #{name}. ")
        Puppet.debug("Res.qtname = #{res[:qtname]}. ")
        Puppet.debug("Res.name = #{res[:qtname]}. ")
        Puppet.debug("Res.vol = #{res[:volume]}. ")
        Puppet.debug("Res = #{res}. ")
      end
      qtrees_list.each do |prov|
        Puppet.debug("Prov.name = #{resources[prov.name]}. ")
        if resource = resources[prov.name]
          resource.provider = prov
        end
      end
    end

I try to create/manage the same 4 resource as before but What I get as the debug output is just one resource (the last):
Debug: Processed all qtree instances.
Debug: Returning qtrees array.
Debug: name = .
Debug: Res.qtname = test2.
Debug: Res.name = test2.
Debug: Res.vol = voltst12.
Debug: Res = Netapp_qtree[/voltst12/test2].
Debug: Prov.name = .
Debug: Prov.name = .
Debug: Prov.name = .
Debug: Prov.name = .
Debug: Prov.name = .
Debug: Prov.name = .

Changing the property name from qtname to name I can get the 4 resource correctly displayed but the match is still not working:
Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Checking if this is an actual qtree, not a volume.
Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Checking if this is an actual qtree, not a volume.
Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Checking if this is an actual qtree, not a volume.
Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Processing rule for qtree 'qt1'.
Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Volume for 'qt1' is 'voltst11'.
Debug: Creating instance for 'qt1'.

Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Checking if this is an actual qtree, not a volume.
Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Processing rule for qtree 'testtitleonly'.
Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Volume for 'testtitleonly' is 'voltst11'.
Debug: Creating instance for 'testtitleonly'.

Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Checking if this is an actual qtree, not a volume.
Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Processing rule for qtree 'testname'.
Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Volume for 'testname' is 'voltst11'.
Debug: Creating instance for 'testname'.

Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Checking if this is an actual qtree, not a volume.
Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Processing rule for qtree 'test2'.
Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Volume for 'test2' is 'voltst11'.
Debug: Creating instance for 'test2'.

Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Checking if this is an actual qtree, not a volume.
Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Checking if this is an actual qtree, not a volume.
Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Processing rule for qtree 'qt2'.
Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Volume for 'qt2' is 'voltst12'.
Debug: Creating instance for 'qt2'.

Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Checking if this is an actual qtree, not a volume.
Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Processing rule for qtree 'test2'.
Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Volume for 'test2' is 'voltst12'.
Debug: Creating instance for 'test2'.

Debug: Puppet::Provider::Netapp_qtree.cmode.prefetch: Checking if this is an actual qtree, not a volume.
Debug: Processed all qtree instances.
Debug: Returning qtrees array.
Debug: name = testtitleonly.
Debug: Res.name = testtitleonly.
Debug: Res.vol = voltst11.
Debug: Res = Netapp_qtree[testtitleonly].
Debug: name = testname.
Debug: Res.name = testname.
Debug: Res.vol = voltst11.
Debug: Res = Netapp_qtree[testnameonly].
Debug: name = test2.
Debug: Res.name = test2.
Debug: Res.vol = voltst12.
Debug: Res = Netapp_qtree[/voltst12/test2].
Debug: Prov.name = .
Debug: Prov.name = .
Debug: Prov.name = .
Debug: Prov.name = .
Debug: Prov.name = .
Debug: Prov.name = .
Debug: Puppet::Provider::Netapp_qtree.cmode: checking existance of Netapp qtree testtitleonly against volume voltst11
Debug: Value =
Notice: /Stage[main]/Profile::Storage::Netapp/Netapp_qtree[testtitleonly]/ensure: current_value absent, should be present (noop)
Debug: /Stage[main]/Profile::Storage::Netapp/Netapp_qtree[testtitleonly]: The container Class[Profile::Storage::Netapp] will propagate my refresh event
Debug: Puppet::Provider::Netapp_qtree.cmode: checking existance of Netapp qtree testname against volume voltst11
Debug: Value =
Notice: /Stage[main]/Profile::Storage::Netapp/Netapp_qtree[testnameonly]/ensure: current_value absent, should be present (noop)
Debug: /Stage[main]/Profile::Storage::Netapp/Netapp_qtree[testnameonly]: The container Class[Profile::Storage::Netapp] will propagate my refresh event
Debug: Puppet::Provider::Netapp_qtree.cmode: checking existance of Netapp qtree test2 against volume voltst11
Debug: Value =
Notice: /Stage[main]/Profile::Storage::Netapp/Netapp_qtree[/voltst11/test1]/ensure: current_value absent, should be present (noop)
Debug: /Stage[main]/Profile::Storage::Netapp/Netapp_qtree[/voltst11/test1]: The container Class[Profile::Storage::Netapp] will propagate my refresh event
Debug: Puppet::Provider::Netapp_qtree.cmode: checking existance of Netapp qtree test2 against volume voltst12
Debug: Value =
Notice: /Stage[main]/Profile::Storage::Netapp/Netapp_qtree[/voltst12/test2]/ensure: current_value absent, should be present (noop)
Debug: /Stage[main]/Profile::Storage::Netapp/Netapp_qtree[/voltst12/test2]: The container Class[Profile::Storage::Netapp] will propagate my refresh event

@hunner Seems that having 2 namevar (volume, qtname or volume, name) don't work well with this module... or at least I'm not able to make it work. Any suggestion is welcomed.

@hunner it seems this is one of the problem I'm facing... the prefetch use name as the key even if I'm using composite namevar.

@hunner I had to rollback some changes in the params definition as they was really not working correcly (was not accepting all resource probably for some bug I've reported above).
The last commit I've created seems to work correctly in all cases. I had to change the way self.prefetch work completely as it was not working correctly.
The sevenmode provides still need the changes but If possible I prefer to finalize/test the cmode provider first.

Cool! I can get back to this today so will take a look.

Good let me know if you want to do/try some more change with the composite namevar and self.title_patterns instead of using defaultto but I was not able to get all the resources in the provider self.prefetch. Every test I've done I was only able to get 1 or 3 of the 4 resources. It seems related to some kind of bug but it's a really strange behaviour.
I'll do some more tests tomorrow (I've a class that create/remove 15 qtree trying to mix and match volume/qtname/title usage). When all works correctly I'll replicate the changes to the sevenmode version of the Provider.

I'm sorry the title_patterns didn't work out. I couldn't figure out why in a reasonable timeframe and your solution is working, so we'll go with that. I did file #87 as part of this work, and also submitted https://github.com/fduranti/puppetlabs-netapp/pull/2 for this PR to allow puppet resource to work.
prefetch matching on qtname and volume is good. Does prefetch need to handle a case when volume is empty? The provider code does at https://github.com/puppetlabs/puppetlabs-netapp/blob/c0e0b97db07b5c4d85bf5cfb3f1de23b2eeac813/lib/puppet/provider/netapp_qtree/cmode.rb#L48 but I'm not aware of a situation where the volume would be empty.
I haven't even tried the sevenmode stuff; that predates my looking at the module and I'm not sure how to test that against a simulator.

The title_pattern is really a nice and elegant solution but for some reason there's probably some kind of bug that will not let it work. The only real difference in the last implementation I've done is that I still define the "name" param but I think there's something in the title_pattern itself that create problems.
I'm not aware of a situation where there's a qtree but not a volume name but the provider was checking for it so I've changed it to just assign an empty string in case the volume string is empty (not sure if I can just ignore the unless. In our case the "volume" should at least be defined so that the match can be done without errors.
What I've done is to change this to an if-else to just assign the empty string:
From this:
      qtree_hash[:volume] = qtree_info.child_get_string("volume") unless qtree_info.child_get_string("volume").empty?

To this:
        if qtree_info.child_get_string("volume").empty?
          qtree_hash[:volume] = ""
        else
          qtree_hash[:volume] = qtree_info.child_get_string("volume")
        end

I'm not sure if assigning directly qtree_hash[:volume] = qtree_info.child_get_string("volume") will work the same (not a ruby programmer here :) )
sevenmode provider is exactly the same as cmode and it should work exactly as cmode (the difference is that the data is got from a single node instead of getting it from a svm but the key should always be /volume/qtree so I'll replicate the change when all is finished... not sure for how much time netapp will still support 7mode too.

Just a question about fduranti#2, is this needed for the automatic check (that use puppet resource)? Is it possible to use the defined resource with 'puppet resource' instead of puppet device to test it in some way?

Defined resource types cannot be used with puppet resource, only native ruby types. This is one of the primary benefits of native types, actually; being able to introspect an existing system to see what puppet thinks the state is, and often in a way that doesn't require intimate working knowledge of the system. I know barely anything about NetApp filers, but I can run puppet resource netapp_volume against a vserver and get an understandable listing, for example. And with --debug I can see the actual commands that puppet runs to get the information. This is really helpful for learning new systems when a type was originally implemented by a subject matter expert.

I'm not sure if assigning directly qtree_hash[:volume] = qtree_info.child_get_string("volume") will work the same (not a ruby programmer here :) )

I believe so; I think qtree_info.child_get_string("volume") can return an empty string and "".empty? is a boolean result (ruby functions that end in ? practically universally return true/false, and ones that end in ! almost universally mutate the object. Like var.flatten!).
If we never have to consider a case where volumes will be blank, then that may mean we can just strip out any code handling this. And then it may not even make sense to break a qtree apart from /volume/qtname into two separate properties... But I guess that's what we're stuck with.

@hunner Sorry to be a bit late on this. This week I had no much time to test or do anything. I will try to finish the work on cmode provider (removing the if on volume name as it should be always present) and do all the correct changes to the sevenmode provider.

@hunner I've finished and added the last changes we discussed to the cmode and also modified sevenmode version.

@hunner thanks for the merge :)


Generated index.html is always a "skeleton" one

I'd expect the source/index.html to be taken.

the problem is that there needs to be a slightly different index.html; source/index.html is also the skeleton one, just for the generator so it could be deleted when config.json is deleted.
I've always assumed that index.html was 100% vanilla - is there something you keep in yours?

We've only modified index.html files in our projects. Either for splash screens or sorting out bad browsers early, etc. Partly adaptions needed for the header to hint for the availbility of iOS apps. Or the definition of the home icon, favicon, etc.
The generated one sets the title to "skeleton", no matter how the application is named. And it needs to be customizable IMHO.

Maybe some kind of template with placeholders for what you need to replace/insert?

I already ran into this problem ... I had also expected that source/index.html was going to be used.

fixed in next release;

the application title can be customised via compile.json or a fully custom index.html can be placed in boot/index.html
if there is a source/index.html a warning is output that it is no longer used

@johnspackman I've a top-level boot/index.html, but it doesn't seem to be used at all. Assuming that the boot directory needs to be at the same level like the compile.json is.

ah - I forgot that the Manifest.json needs to specify it, eg:
{
	"info": { /* snip */ },
	"provides": {
		"namespace": "demoapp",
		"encoding": "utf-8",
		"class": "source/class",
		"resource": "source/resource",
		"translation": "source/translation",
		"boot": "source/boot",
		"type": "application"
	},

Yeah. That works. Thanks :+1:

Cool, I need to update the docs and then I'll close this


[PROPOSAL] Pipeline Validation Plugin

I'd like to suggest a feature for drone that would probably help plugin developers/security.
Issue
When using global secrets, you can lock down the images that the secret is exposed to, which is really helpful in preventing secrets from being "printenv" printed in a different step.
However, the user can still configure their pipeline to run like:
---
pipeline:
  docker:
    image: plugins/docker
    commands:
      - printenv
    secrets:
      - docker_password

Or:
pipeline:
  docker:
    image: plugins/docker
    entrypoint: [ /bin/sh ]
    command: [ -c, env ]
    secrets: [ docker_password ]

And potentially:
pipeline:
  hack:
    image: alpine
    commands:
      - echo "env" > /tmp/bin/terraform
      - chmod +x /tmp/bin/terraform
    volumes:
      - data-volume:/tmp/bin
  terraform:
    image: plugins/terraform
    volumes:
      - data-volume:/bin

volumes:
  data-volume:

Which will print the env, including the secret.
Idea
Allow Drone Server to specify a list of images that can only be run in "plugin" mode.
This might be done by:

Config file (like global secrets/registry)
Env vars (though this can get unwieldy like DRONE_ADMIN)

Considerations

If a build step is privileged there will always be a workaround.
Shared volumes
Mount a volume in the container that overrides the entrypoint or the plugin's binary

Alternatives
Somehow allow configuration to prevent secrets from being injected into a container being run in the "plugin syntax" vs non "plugin syntax".

Right now we add a new field every time we want to limit if and when a secret is exposed to a step. For example, we have the images field and events field that let you whitelist docker images and build events.
Instead of adding new fields each time we want to provide additional control over secrets, I think we could take some inspiration from the browser's Content-Security-Policy.
Restrict secret based on image and events:
images: drone/*; events: push tag

Restrict secret based on image and author
images: drone/* plugins/drone-docker; authors: bradrydzewski tonglil

Restrict secret to steps based yaml attributes:
images: plugins/drone-docker; command: none; entrypoint: none, commands: none

I think something like this will be more powerful and secure in the long run, and way more flexible. I would love help putting together some sort of specification for the syntax ...

I am going to take this in a slightly different direction. I want to add a new type of plugin, which I am tentatively going to refer to as a Pipeline Validation Plugin. This plugin could be used to implement global policies, including how secrets are used, which images are use, which host volumes can be mounted in trusted mode, etc. It is slightly more generic and will solve for some other security-related features that have been requested.

Stupid question: how do you protect against this?

echo $secret > file

is a (technically) valid use case, as it’s doing exactly what is intended, which is to use the secret. No matter what you do, you can’t (on your end) figure out the intent of the user, which is what is “bad” in this case, to distinguish good from bad...
Am I missing something? I do think you should be providing some to restrict what is available to the job (e.g. don’t inject prod deploy secrets to dev jobs), but I just don’t see how you tell the difference between intentional good and malicious bad (and how you restrict the latter while allowing the former).

@nnordrum lets assume someone submits a malicious pull request that makes the following modification to our configuration file:
---
pipeline:
  docker:
    image: plugins/docker
 +  commands:
 +    - echo $docker_password > file
    secrets:
      - docker_password

We could write a validation plugin that implements the following logic, which would cause the above configuration file to fail validation:

only expose docker_password to steps using plugins/docker
where commands is undefined
where entrypoint is undefined
where volumes is undefined

It is important to note, however, if you expose your secrets to steps that execute user-defined scripts there may be nothing you can do to protect against this. For this reason, we recommend against exposing secrets to pull requests if your repository is public.
So even though a validation plugin cannot guarantee security against all attack vectors, it can still prevent many common attack vectors.

Ok, so you’re basically saying, “only execute trusted whitelisted plugins”. I need to get out of my “everybody is behind the corporate firewall” mentality where logging trumps security (since “more secure” is inherently “less flexible”; ie this PR ;) )

(Btw, that “mount a volume” link doesn’t work anymore)

We have added support in master for validation extensions.  These extensions allow you to augment the default Drone linting rules and include your own custom linting rules by providing a micro-service.
We have a project template that you can use to generate a validation extension at boilr-validate.
This does not solve the original issue, however, it should enable solving this issue through extensions. I look forward to seeing how people use this plugin to solve various problems. If someone creates an extensions please let us know and post at discourse.drone.io.


Mailcap template typo correction

Tiny typo correction in the mailcap template file.
Corresponding issue: #380

Thanks :) I'm not sure why that test failed on travis, must be a timing issue.


FIND-QCA-PATH STAT3->TP53 errors

This had to do with RasGap not being handled in the edge prioritization code. Solved in 16fb62462e325d95bde4517b9d5c337fdf11c91a


chore(gateway,lavalink): update to async-tungstenite 0.12

Newest version no longer pulls in pin-project and bytes 0.5

I messed up with #667 and tungstenite types are still exposed in two places:

https://docs.rs/twilight-gateway/0.3.0/twilight_gateway/shard/enum.ShardStartError.html#variant.Establishing.field.source
https://docs.rs/twilight-gateway/0.3.0/twilight_gateway/shard/enum.CommandError.html#variant.Sending.field.source

so this needs to be targeted to 0.4.

I messed up with #667 and tungstenite types are still exposed in two places:

https://docs.rs/twilight-gateway/0.3.0/twilight_gateway/shard/enum.ShardStartError.html#variant.Establishing.field.source
https://docs.rs/twilight-gateway/0.3.0/twilight_gateway/shard/enum.CommandError.html#variant.Sending.field.source

so this needs to be targeted to 0.4.


Homology of a cube

Dear Ripser developers,
While trying to compute the persistent homology of a cube, expecting results similar to a 2-sphere, I get the following results:
point cloud
--------------
0.25 0.25 0.25
0.75 0.25 0.25
0.25 0.75 0.25
0.75 0.75 0.25
0.25 0.25 0.75
0.75 0.25 0.75
0.25 0.75 0.75
0.75 0.75 0.75

Results:

The issues are the following:
1.There are only 5 generators of dimension 1.
2.There are no generators in dimension 2.
3.There is a generator in dimension 3 that should ideally be in dimension 2.
Could you please shed some light to this issue?
Thank you in advance,
Alexandros

Dear Alex,
this result is (surprisingly) correct. It is not too difficult to see that there should only be 5 generators in dimension 1. It is a very interesting and non-obvious exercise to show that there are no generators in dimension 2, but a generator in dimension 3.
Best, Uli


fix(npm): instanbul false shrinkwrap fix

when istanbul is set to false (using nyc for example) npm shrinkwrap will fail due to
peerDependencies

Coverage remained the same at 100.0% when pulling 8d2bd74a04e1ec7a9f6c00e165526084b2bf0b23 on vrtxf:master into a0225eb9ffc094a90d6d9a6f06fa680c04a46390 on knpwrs:master.

Is this the proper usage of optionalDependencies? Might not matter as much with npm 3, mostly semantic.

Yes, with peerDependencies you need to resolve those manually in your project - they don't get installed initially but you can't npm shrinkwrap without having them in your dep tree.
I use nyc instead of istanbul so it doesn't make sense for me to have them both.

I mean in terms of ownership. The idea behind making it a peerDependency is that the user's project owns it. optionalDependencies is meant for things that may fail to build. I think it would be better to remove the dependency altogether and release a new (major) version along with new documentation.

I haven't considered that in terms of ownership at all, you'd have to walk me through the needed changes so I can commit them !

Is this still a go ? Thanks

If you wanted to go ahead and remove the dependency and write new documentation for what is required to get istanbul working then I can take a look at it.

I'm going to go ahead and close this since I'm working on updating dependencies


Rule idea: application name as module prefix

Hello
Since all application share the same module namespace, it would be real neat to add an option to the module name rule to require all module names to be prefixed by the application name.

For single application repositories one could just add it to the regex.
For multi application repositories with a single elvis configuration that would be insufficient.

Perhaps like this
{
 elvis_style,
 module_naming_convention,
 #{regex => "^([a-z][a-z0-9]*_?)*(_SUITE)?$",
   ignore => [],
   prefix => application}
}

Makes sense?

@fredrikelinder This is possible by just modifying the regex. What advantage would the prefix option bring vs. modifying the regex option?

For multiple application repositories with a single elvis.config file a possible regular expression could be: ^(app1|app2|app3)_([a-z][a-z0-9]*_?)*(_SUITE)?$.

That would allow app1 as prefix for app2 files

That would allow app1 as prefix for app2 files, which isn't desirable.
In other words: without the prefix option there is no way to warn about files in app2 application being named app1_* (using your example regex).

elvis.config supports multiple config. maps, each with its own dirs option.
Stuff like
[{elvis, [{config, [
    #{ dirs => ["apps/app1"],
       filter => "*.erl",
       ruleset => erl_files,
       rules => [{elvis_style, module_naming_convention,
                  #{ regex => ["^app1_([a-z][a-z0-9]*_?)*(_SUITE)?$"] }}] },
    #{ dirs => ["apps/app2"],
       filter => "*.erl",
       ruleset => erl_files,
       rules => [{elvis_style, module_naming_convention,
                  #{ regex => ["^app2_([a-z][a-z0-9]*_?)*(_SUITE)?$"] }}] }
]}]}].

is possible and solves your particular issue.

exactly!


Mastodon user feeds are incorrectly displayed

Describe the bug
Mastodon creates auto-generated RSS feeds for users. These only have descriptions containing the post contents and no titles which results in an empty list of items being displayed.
To Reproduce
Steps to reproduce the behavior:

Add this feed URL<EMAIL_ADDRESS>Observe all items have empty titles

eb265663d14bb0ae21dba71425e881e8af567e5d fixes this issue

We use the description as a title in case a title is not present. If both of them are not present, we just skip that post.


resolves issues #11, #13, #16 (also resolves merge conflicts)

Just in case, I created separate branches for each issue:
issue #11: pull https://github.com/raphaeljolivet/java2typescript/pull/12
issue #13: pull https://github.com/raphaeljolivet/java2typescript/pull/14
issue #16: pull https://github.com/raphaeljolivet/java2typescript/pull/17
but as there are merge conflicts, it made more sense to create one pull request to fix them all.
After this merge request all projects compile and tests pass :)

Hi,
I have added you as contributor to this repo.
Honnestly, I don't have time to check / support it.
You have made a couple of wise commits, and I trust you for upcomming changes.
Please try to merge the PR yourself to see if it's working fine.


Separate vertical friction/damp from horizontal friction/damp

Version 3.1.rc3official
For RigidBody2D and RigidBody, it would be much better to have separated properties for friction and damp on a per axis basis, instead of global values. Most typical platformer games require very different friction values for vertical movement when pressing on a wall collider (normally lower or none), compared to horizontal friction regarding running and deceleration (usually much higher).
Of the five most common views for physics games (top down, 3/4, side view, isometric, and perspective 3d) only strictly top-down games wouldn't typically use different friction or damp values for horizontal and vertical movement and collisions, so it would be a good idea to expose the axial components of friction and damp. Gravity does not suffer from that, since it is already vector based, and you can change it on the default settings.
Work arounds obviously exist, either by code or on the IDE (using different one-way collision nodes for wall friction and floor friction occupying the same space, which makes it harder to model levels and consumes more resources). Both are less practical than simply exposing the x and y components of the friction and the damp (or the x, y and z components in 3d) and unnecessarily convoluted, as the intended use of the physics engines is precisely to make the development process faster.

Broadly agree with this, however I suspect that what is exposed in Godot is probably pretty tightly coupled with what features are available in Bullet. I had a quick look but couldn't really work out whether Bullet supports this.
As an alternative to changing parameters of wall / floor models you can probably tweak this kind of thing by writing your own _integrate_forces, e.g. multiply your linear_velocity by e.g. (0.97, 0.99, 0.97) on each physics tick, depending on whether you detect your character as being in air or on ground etc.

The concept of a "rigid body" is universal and primarily used for simulating Newtonian (i.e. realistic) physics, so modifying it to benefit a specific genre of game (platformers with wall jumps) is a terrible idea.

I hope the physics engine wont forbid this. I respectfully disagree on the Newtonian physics argument. The concept of a rigid body within a game engine like Godot is to simulate physics only in respect to purpose of the engine, which is to make games :). Not only platforms would benefit form this. As mentioned, basically any game with physics which is not using a strictly from above view could benefit for this extra level of control, and implementing it (if possible) wouldn't hurt realistic simulations, so it is not really a trade-off relationship to favor a genre in detriment of others.
I would even say that enabling useful jumping behavior or gamey-ish vertical pushes is probably among the most important use-cases for even having a physics system within a game engine in the first place, and should have precedence than, say, making astronomy simulation and the sort, but indeed, separating the axial components of this forces, if possible, wouldn't change or detract anything from a standard realistic simulation if you ever wanted to make one with Godot.
If I understand it correctly, removing this limitation would make it easier to have Rigid Bodies character controllers working as expected for most games (in 2d and 3d), and wuold circumvent a lot of the  workarounds taken in the RigidBody Character controller demo, like disabling the integrator. If possible, I think it would be best to have axis based friction, damp and bounce (gravity is already vector based).

Well, consider these scenarios:

Damping: best case scenario is you'll be able to control maximum fall speed separately from maximum lateral movement speed. Even then, the maximum speed will correlate to something unrelated to the damping value given (which is a value between 0 and 1...)
Friction: if vertical friction was low and lateral friction was high, what would happen if you decided you wanted sloped surfaces? Movement speed would be increased—the amount determined by how steep the slope was—even when moving up a slope.
Bounce: ??? i don't know what this would be used for

Not only would this not give someone a consistent and reliable character controller, but it would cause issues and inconsistencies that would have to be addressed by code anyway.
Also, about the whole "gravity is a vector" thing you mentioned: I don't really know how that sets a precedent for what you're proposing. It's completely unrelated (and would complicate things if it was).
Getting custom movement logic right is deceptively difficult, and having to code stuff yourself is an unavoidable reality when dealing with it. The resulting code can wildly vary depending on what you're trying to do, so there isn't much room for "one-size-fits-all" solutions (though move_and_slide_with_snap() gets pretty close). Instead of ignoring this reality and asking for these near-sighted workarounds, it's best you see this for what it is, grit your teeth, and learn what you need to deal with it. This is directed to you and anyone else you're supposedly trying to help.

Good point on the friction on slopes, that one eluded me, I can imagine even more unforeseen quirks that were not apparent for me at first, thanks. I am well aware that any refined movement logic entails complexity that can only me nailed down by coding, and in the projects that this aspect of physics has posed a problem I have already solved it, either with code or by using jointed nodes with different physics materials. Perhaps that last one is the proper lazy way to do it on the editor.
However, I am indeed very favorable towards having quality of life stuff such as move_and_slide whenever possible, and figured this could lead to easily modeling a similar experience with Rigidbody nodes, getting the free physics as a bonus. My proposal was solely on the intuitive assumption that dividing these properties by axis could allow many creative ways to tackle varied needed behaviors by just setting thing up in the editor or just changing a few properties by code (unidirectional dampers, vertical-only or horizontal-only trampolines, multi-use wall/floor cubes, etc), without really getting in the way of anything else. Godot already has a lot of such smart tricks available, such as simply making conveyor belts with Staticbodies by setting a linear force to affect other nodes, so I thought this would be on line with that "sandbox" an quick setting ethos.

Feature and improvement proposals for the Godot Engine are now being discussed and reviewed in a dedicated Godot Improvement Proposals (GIP) (godotengine/godot-proposals) issue tracker. The GIP tracker has a detailed issue template designed so that proposals include all the relevant information to start a productive discussion and help the community assess the validity of the proposal for the engine.
The main (godotengine/godot) tracker is now solely dedicated to bug reports and Pull Requests, enabling contributors to have a better focus on bug fixing work. Therefore, we are now closing all older feature proposals on the main issue tracker.
If you are interested in this feature proposal, please open a new proposal on the GIP tracker following the given issue template (after checking that it doesn't exist already). Be sure to reference this closed issue if it includes any relevant discussion (which you are also encouraged to summarize in the new proposal). Thanks in advance!


Add good default task description to the templates

@stinenyhus I can see that the current prompttempate task_description is currently a dummy placeholder. Can we remove that and instead have the default task_descriptions for each of the templates which match the template style (I should have caught this in the review).

Yes, definitely! I will look into this now

Perfect. I actually already did a bit of the work as I needed it to create another test (see it on the PR #14 )


Add option to loop through multiple waveforms when running benchmarking

This update gives the option to run the benchmarking code ( #12 ) on multiple waveforms. It also updates the input arguments to set the help option and indicate which arguments are required when running against a single waveform versus running against multiple waveforms. An upcoming PR will print out the filename being run.
UPDATE BENCHMARK.MD

It would be cleaner just to allow the input to be an individual file OR a folder. Does it need to be more complicated than this?

@tompollard , I was planning to allow a benchmarking run which pulls files from multiple PhysioNet databases. Having said that, it would be more efficient to simply download the files and they could be placed into a single folder, so that might not be necessary.

@tompollard , I decided to stick with the design that I had because I also need to run benchmarking across multiple file formats. This will be much easier to automate with a control file.

@tompollard , I decided to stick with the design that I had because I also need to run benchmarking across multiple file formats. This will be much easier to automate with a control file.

Up to you, and I know you are working to a tight timeline. I think it makes the package unnecessarily fiddly! It isn't clear to me why having to pass a CSV as an agrument is easier than just pointing to a folder of well-organised files. If it is necessary to specify details beyond what is captured in the files, then these details could be captured in a metadata file in the folder.


Suggest type hinting

🚀 Pull Request
Description

A simple note in the documentation to suggest type hinting when contributing to the codebase. Moves us forward with #4272 / #4382 / #4383, since everyone seems in agreement.

Consult Iris pull request check list

This is looking good to me. Unless there are any objections I will merge this in by the end of the week

Oh dear! I was just about to merge this but I see there is now a conflict! @trexfeathers do you mind rebasing?


Config :log_color_app is not working

As title, i set log_color_app = blue, yet my application log text is still in white

Here' s my screenshots

It's working

@tiger-regit strange..mine not working..btw i already abandon this project since its not being maintained anymore..Currently im using realize


update

Codecov Report

Merging #8 into master will decrease coverage by 2.69%.
The diff coverage is 26.66%.

@@            Coverage Diff             @@
##           master       #8      +/-   ##
==========================================
- Coverage   76.24%   73.55%   -2.70%
==========================================
  Files           6        6
  Lines         261      276      +15
==========================================
+ Hits          199      203       +4
- Misses         62       73      +11

Impacted Files
Coverage Δ

gmail/parsePandas/parse.py
39.43% <26.66%> (-3.43%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2bd6709...0405dc8. Read the comment docs.


Optimizations in powermock-reflect

When I started to use Powermock instead of Easymock, some test which used to take seconds started to take more than one minute. The proposed changes make performance of Powermock more or less comparable with Easymock.

@leventov,
thank you for pull request. Could you fix a build? Looks like you used Java 1.8.

@thekingnothing fixed


feat(http): delete endpoint

Closes #13215
Add predicate endpoint, service, swagger, permission, test cases.

[ ] CHANGELOG.md updated with a link to the PR (not the Issue)
[x] Well-formatted commit messages
[] Rebased/mergeable
[x] Tests pass
[x] http/swagger.yml updated (if modified Go structs or API)
[x] Documentation updated or issue created (provide link to issue/pr)
[ ] Signed CLA (if not already signed)

Agreed with @edd. We can leave the code in a commit if we want to be able to bring it back at some point, but we probably don't need to parse for tokens we don't support.


Provide mongo credentials in all configuration.toml file

Mongo is going to be started with enabled the access controll mode.(see here)
This means that each service that connects to mongo will need to authenticate itself even when
EDGEX_SECURITY_SECRET_STORE = false, or the service is run by snap.
Signed-off-by: difince<EMAIL_ADDRESS>PR Checklist
Please check if your PR fulfills the following requirements:

[ ] The commit message follows our guidelines: https://wiki.edgexfoundry.org/display/FA/Contributor's+Guide
[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)

PR Type
What kind of change does this PR introduce?

[x] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[ ] Documentation content changes
[ ] Other... Please describe:

What is the current behavior?

Issue Number: https://github.com/edgexfoundry/edgex-go/issues/2380
What is the new behavior?
Does this PR introduce a breaking change?

[ ] Yes
[X] No

Are there any specific instructions or things that should be known prior to reviewing?
Other information

Codecov Report

Merging #2381 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #2381   +/-   ##
=======================================
  Coverage   37.57%   37.57%
=======================================
  Files         157      157
  Lines       12768    12768
=======================================
  Hits         4798     4798
  Misses       7709     7709
  Partials      261      261

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e5ebbe2...1e36588. Read the comment docs.

Added security-related reviewers.  Larger question -- should we be propagating "standard" persistence credentials into individual service configuration files?

Added security-related reviewers. Larger question -- should we be propagating "standard" persistence credentials into individual service configuration files?

This is a good question to discuss:  I thought the current Mongo's credentials for services are generated on the fly and uploaded to Vault via Vault worker.  Is this credentials from toml file mongo's ROOT username and password? If so, it looks insecure via this approach...

@jim-wang-intel No, these are not  mongo's ROOT username and password.
You are correct, Mongo credentials are generated on the fly and uploaded into Vault via vault-worker but only when the security is enabled (i.e when EDGEX_SECURITY_SECRET_STORE=True )
If the security is disabled, microservices should nevertheless authenticate itself in front of mongo. This authentication will become mandatory when mongo access control is enabled

should we be propagating "standard" persistence credentials into individual service configuration files

Is you question equal to:

"Is the configuration file the right place for storing these "standard" credentials ? "

accepting that "Standard Credential" = The credentials used when security is disabled


Suppress errors using pragmas / pseudo comments

For some rules like unused_variables, pragmas or pseudo comments can be used to suppress the error message.
It would be nice to have this available for more rules, esp:

unused_types (##NEEDED, #EC NEEDED)
unused_methods (##NEEDED, #EC NEEDED)
check_subrc (##SUBRC_OK, pseudo already there)

You have a generic place to maintain the pragma in the rule metadata. However, it looks like each rule has to implement it's own logic to ignore errors if a pragma is present. If you make it generic, I could easily add these to the meta (and maybe some more for other rules).

yea, right now there is no generic implementation of it, hmm

for unused_methods I dont remember where the pragma should be placed

it's actually ##CALLED (not ##NEEDED) in method implementation.
method test ##CALLED.
...
endmethod.

pragma possibility added for these rules:

unused_types
unused_methods
check_subrc

the pseudo comment is a bit more work

thanks 👍

suggest closing the issue, alternatively move the 2 pseudo comments to a new issue


Markdown lint

Coverage remained the same at 100.0% when pulling e69ac246fd63afda03cb6b1de53dd1f2749d6170 on markdown-lint into 24c106d43d9f4c5a9897c462a7d2cfb33b80b857 on master.