How to play in full screen after recording?

I only found 100% zooming, the top and bottom bars are taking too much space, is it possible to play the gif in full screen?

You can reduce the size of the ribbon (top tool bar) by clicking on that arrow to the right, but that's how far you can go within the app. You can try exporting as video or gif and watching the animation on a player too.

You can reduce the size of the ribbon (top tool bar) by clicking on that arrow to the right, but that's how far you can go within the app. You can try exporting as video or gif and watching the animation on a player too.

@NickeManarin Ah, I see what you mean, I was looking for something that does the arrow's job but missed. I guess that proves the bottom right arrow UX is kinda counter-intuitive, I think most Windows users are used to MS word's handling to this case (please ignore the red search icon):

Could you please consider changing the collapse UX to the above, and perhaps add a full screen mode in a future version?

@NickeManarin Ah, I see what you mean, I was looking for something that does the arrow's job but missed. I guess that proves the bottom right arrow UX is kinda counter-intuitive, I think most Windows users are used to MS word's handling to this case (please ignore the red search icon):

Could you please consider changing the collapse UX to the above, and perhaps add a full screen mode in a future version?

Yes, I'm going to improve the editor window. I already started working on a new ribbon element.

Yes, I'm going to improve the editor window. I already started working on a new ribbon element.


CARDS-2014 - Patient portal: the labels of the buttons that transition to the next survey are too long

See CARDS-2014 on jira

@sashaandjic Changes in the last commit:

Made the button label even shorter (next survey's alias only)
The pagination's Next button now has an icon on the last page (when the next step is another survey or a review screen)

It would be nice to update the aliases of the proms surveys as well.

Right, of course


Fix change shipping address on confirm page

1. Why is this change necessary?
Change shipping button on confirm page should change the shipping address.
2. What does this change do, exactly?
Fix the data attribute to change the shipping address instead of the billing address in the modal window.
3. Describe each step to reproduce the issue or behaviour.
Go on confirm page and try to change your inidvidual shipping address.
4. Please link to the relevant issues (if any).
https://issues.shopware.com/issues/NEXT-8586
5. Checklist

[ ] I have written tests and verified that they fail without my change
[ ] I have squashed any insignificant commits
[ ] I have written or adjusted the documentation according to my changes
[ ] This change has comments for package types, values, functions, and non-obvious lines of code
[x] I have read the contribution requirements and fulfil them.

Hello,
thank you for creating this pull request.
I have opened an issue on our Issue Tracker for you. See the issue link: https://issues.shopware.com/issues/NEXT-8891
Please use this issue to track the state of your pull request.

Hi @acris-cp,
thank you for your contribution. 🎉
Your changes have been merged to 6.2.
Best regards
Tobias


Added Visual Studio 2019 compiler name to platform.h

Quick fix for BX_COMPILER_NAME for Visual Studio 2019 version 16.0.0.

Woo! 😄


Save Contact Information API not updating phone and receive_marketing_info fields in the database

I'm writing to report an issue with the "Save Contact Information" API https://docs.boundless-commerce.com/#tag/Checkout/paths/~1orders~1checkout~1contact/post
It appears that the API is not correctly updating the "phone" and "receive_marketing_info" fields in the database when a user attempts to save their contact information.

Sorry, missed the issue! I will look into it this week.
Thank you!

Hi @tueanhtruong
The issue should be fixed.
Thank you!


service/header: change Broadcaster from interface to struct

Now that BlockService is only responsible for reconstructing block from shares and HeaderService is responsible for constructing new ExtendedHeaders and broadcasting them, broadcaster doesn't need an interface and can be a private struct instead.

@renaynay should we close this one already?

Not until #327 is merged.


Add SwiftLint to codebase

The goal of this issue is to add SwiftLint to the codebase to make code more consistent.
https://github.com/realm/SwiftLint

I can help with that. Would you be up to add Github Action to prevent merging inconsistent code? Usually, it's only way to ensure no inconsistent code is merged.

@Garfeild Project script + CI running on every PR would be ideal setup 👍🏻

I have tried to integrate SwiftLint as plugin. There are some pros and cons with such integration.
Pros:

It shows warning/errors during compilation time in Xcode directly.
It's easy to share with any contributor and harder to disable.

Cons

It lints whole project and couldn't lint only changed files. This is not visible for current state of the project (takes 0.17s).
Until we fix all warnings, build log will be polluted with them. Easy to miss important warnings.

After I cleaned up everything, it was 4 warnings.

Personally, I think it gives us good start and we can ignore first problem and mitigate second. And I would still introduce CI check.


500 errors when fetching show progress for unaired series.

When fetching progress on a show with unhaired episodes (but already logged plays) likes orphan black: echoes.
Which first aired back in November although it erroneously shows as unhaired on Trakt.
I get 500 errors:

To note, this seems only to occur when requesting specials (which this show doesn't contain).
These errors go away when fetching without specials (most likely an issue in the backend ?)

I also get 500 when I attempt to remove plays directly form the website:

@dvcol I just pushed a few for the API. The same fix might also fix the website. Please let me know if it's working any better for you.

Hey, seems to be fixed now, thanks for the speedy response :)
It now sends back an empty progress object instead of a 500, no matter if specials are queried or not 👍
Although this particular show aired back in November in Canada, but maybe this is due to my region settings?

We get the air dates from TMDB. I'm not sure why they are using the US release dates as the main ones, but that's why we don't consider it aired.


[Question] Is an instance of OSRM object is thread safety?

What is the right way to use the instance of OSRM object? Is it can be used as singleton?

I have to investigate it fully, but I assume it is not safe. The .net side code should be safe, but I'm not so sure about the unmanaged libosrm part.

I posted this question in osrm project (Issue 4966)

@ifle they said it's thread safe.


KeyboardDragListener addHotKeys is broken

I’m trying to implement “jump” hotkeys, like BASE.   I'm using this code in BalloonNode.js as my guide:
    this.keyboardDragHandler.addHotkeys( [
      {
        keys: [ KeyboardUtils.KEY_J, KeyboardUtils.KEY_W ],
        callback: () => {
          this.jumpBalloon( new Vector2( X_POSITIONS.AT_WALL, model.getCenterY() ) );
        }
      },
      {
        keys: [ KeyboardUtils.KEY_J, KeyboardUtils.KEY_S ],
        callback: () => {
          this.jumpBalloon( new Vector2( X_POSITIONS.AT_NEAR_SWEATER, model.getCenterY() ) );
        }
      },
      {
        keys: [ KeyboardUtils.KEY_J, KeyboardUtils.KEY_N ],
        callback: () => {
          this.jumpBalloon( new Vector2( X_POSITIONS.AT_NEAR_WALL, model.getCenterY() ) );
        }
      },
      {
        keys: [ KeyboardUtils.KEY_J, KeyboardUtils.KEY_C ],
        callback: () => {
          this.jumpBalloon( new Vector2( X_POSITIONS.AT_CENTER_PLAY_AREA, model.getCenterY() ) );
        }
      }
    ] );

I’m seeing some very buggy behavior in BASE on master.  Typing J alone shouldn't do anything, but it moves the balloon to the wall.  And if I’m not fast enough in pressing the “S” in “J+S”, the balloon moves to the wall.
I confirmed that this is NOT broken in the published version BASE 1.5.1.
@jessegreenberg confirmed that this is broken in master, and suspects changes that he and @zepumph made in  https://github.com/phetsims/scenery/issues/1331.
This is blocking progress on https://github.com/phetsims/geometric-optics/issues/279.

I'm see some additional weird behavior in geometric-optics (not pushed).  Here's my code for adding hotkeys to a ruler:
this.keyboardDragListener.addHotkeys( [
      {
        keys: [ KeyboardUtils.KEY_J, KeyboardUtils.KEY_L ],
        callback: () => {
          console.log( 'jump to Lens' );//XXX
        }
      },
      {
        keys: [ KeyboardUtils.KEY_J, KeyboardUtils.KEY_O ],
        callback: () => {
          console.log( 'jump to Object' );//XXX
        }
      },
      {
        keys: [ KeyboardUtils.KEY_J, KeyboardUtils.KEY_I ],
        callback: () => {
          console.log( 'jump to Image' );//XXX
        }
      }
    ] );

If I press J, the J+L callbak aways fires.
If I press L, O, or I without J, the associated callbacks will sometimes fire.  Whether it fires seems related to how long I hold the key down.

This was introduced by https://github.com/phetsims/scenery/issues/1331, I verified that BASE is fixed after the above commit. I also tried the example in geometric-optics in https://github.com/phetsims/scenery/issues/1334#issuecomment-1004288776 and it is working. @pixelzoom can you please verify that this is fixed?

This seems like it's working correctly now for the rulers in Geometric Optics.  See this.keyboardDragListener.addHotkeys in GORulerNode.js, if you're interested.
Thanks for the quick turnaround. Closing.


Feature request: View parent references for a given module

Issue description
I love this plugin; this is an absolutely awesome way to understand the contents of a bundle, but once we see the things making our bundle huge, we end up asking how it even got there.
It would be AWESOME to be able to click on a module box and somehow get a list of other modules that reference said thing. Then those things could be expanded to see things that reference them.
This would definitely help in knowing which require statements needs to be translated into "ensures" so that they can be deferred.

Great idea! It might be possible to extract this information the webpack stats reason field. It can be seen from this example stats file.
Would those details present in the reason fields contain the information you'd like to see, @dzearing?

@valscion yes that's exactly it. Imagine some ui like, hover of specific module and says:
Included from:

ModuleA (require)
ModuleB (require.ensure)

etc
Which can be expanded in the ui somehow that we can drill the whole graph path. If you expand and there is a linear list, it expands all the children. If there is a node that was pulled from more than 1 module, it stops expanding at that point.

That would be nice. It also sounds like a use case for a separate reporter UI, if we'll get #32 done at some point :).
I don't think we're able to do the reporter changes ourselves easily here, as the treemap reporter we're using is not controlled by us, if I've understood correctly.

I would already be happy if I could hover over a module and the consumers were colored one way, and the deps another way. Maybe that is easier to achieve…

I fear that we can't do much with the current treemap visualization, as it seems to be a demo version of some commercial code. We only have the minified/obfuscated treemap code available.

Having it would be awesome! Wouldn't it be possible to just reflect the reasons field from the JSON export in the popover window (with the sizes and so on)? This would not be perfect but would really help on this topic I think.
Thanks for this amazing tool!

Hopefully once I've got #97 figured out, we could have user-land implementations of this :). Right now implementing this faces same problems as https://github.com/th0r/webpack-bundle-analyzer/pull/85#issuecomment-313622625

Hi! Thanks for continuing work on this PR :) I'd like you to know that we haven't forgotten about this PR. It's just that this has lots of new changes, and we don't have any proper UI tests, that adding new features to the UI is starting to be much more scary maintainability-wise.
Would you happen to have any ideas on how we could possibly add even rudimentary UI tests for this?


why this project and what's the difference to SecretAgent

i am from SecretAgent  issue, i saw SecretAgent  was not update

Hi @RonaldinhoL, we're in the process of releasing Hero, which is the next evolution of SecretAgent. We're still pulling important fixes back to SecretAgent, but trying to get Hero release as soon as we can. We'll have several posts about what's new/different, but the quick summary is SecretAgent was focused on anti-blocking, and Hero is focused on easily creating stable scraper scripts (ie, they react to page changes and redesigns, and are built with a series of development tools built into Chrome).

This is a list of some differences with a migration path.
https://ulixee.org/docs/hero/help/migration-guide


Improve HoverTip placement logic

This changes the HoverTip placement logic to use measurements from the actual DOM node rather than basing everything on the maximum sizes.
This avoids some counter-intuitive behaviour, most importantly situations where the label would overflow off the left side of the screen for no obvious reason.

Before
After

Fixes #394
Fixes #256

Coverage decreased (-0.06%) to 41.597% when pulling 779dbc42720c4b0f05f7b0c881fd33fc160718bf on jlfwong/improved-hovertip-placement into 48d692c2a330fcb1b183684836f2d73eb1e93bae on main.


Function Runtime : Container is unhealthy. Recycling container...

Have followed the instruction to install Azure functions runtime on windows server 2016.
-Installation went sucessfull.
i can access the portel rom url to IIS.
-i can  create the function app .
-and deploy using visual studio and all.
-Also have downloaded the azfuncrun/functions:0.latest docker image manually because it kept looping to download.
-This image is downloaded and installed.
However the container app function recycles and stops and spins another container.
-It is a simple timertrigger function with one line (defsult) to log the executions.
Please refer to the logs below.
2017/08/21 4:16:12 PM [WIN-SPLKPM61TUA] Container is Unhealthy testtesttest_5
2017/08/21 4:16:12 PM [WIN-SPLKPM61TUA] Container testtesttest is unhealthy. Recycling container...
2017/08/21 4:16:12 PM [WIN-SPLKPM61TUA] Function App 'testtesttest' is stopping...
2017/08/21 4:16:12 PM [WIN-SPLKPM61TUA] Issuing stop command to container: testtesttest_5
2017/08/21 4:16:12 PM [WIN-SPLKPM61TUA] Disposed container health check timer.
2017/08/21 4:16:12 PM [WIN-SPLKPM61TUA] Stop complete for container: testtesttest_5
2017/08/21 4:16:12 PM [WIN-SPLKPM61TUA] Function App 'testtesttest' has been stopped.
2017/08/21 4:16:12 PM [WIN-SPLKPM61TUA] Function App 'testtesttest' is starting...
2017/08/21 4:16:12 PM [WIN-SPLKPM61TUA] Provisioning container testtesttest_6
2017/08/21 4:16:53 PM [WIN-SPLKPM61TUA] Container testtesttest_6 has been created. (Id: 4bb9043e35acc52936c435f1d8cccf5ee7a7c6461a60fe7dd176080c78494c64
Investigative information

Timestamp: 2017/08/21 4:16:53 pm
Function App name: testtesttest
Function name(s) (as appropriate): TimetTriggerCSharp1
Invocation ID: 4bb9043e35acc52936c435f1d8cccf5ee7a7c6461a60fe7dd176080c78494c64
Region: Runtime on premise.

Repro steps
Deploy function to azure runtime and try and start observe the logs.
Expected behavior
Log line to appear in the logs with the following code.
using System;
public static void Run(TimerInfo myTimer, TraceWriter log)
{
log.Info($"C# Timer trigger function executed at: {DateTime.Now}");
}
Actual behavior
Container starts and stops continuously without running the function.
Known workarounds
Run it in the Azure function (Cloud).
Related information
NON

Having the exact same issue

a) Hi i got around the issue by running this:
run the following in the VM power shell:
Get-VM | Set-VMProcessor -ExposeVirtualizationExtensions $true
Resource :  (https://github.com/Azure/Azure-Functions/issues/359)
b) And then change the trigger or refresh the function for a change to take place.
c) Wait a while while recycling of containers happens.
d) Eventually will work.
Hope it helps.

Yes, we use Hyper-V containers and if you run the workers on a virtual machine, you need to enable nested virtualization for Hyper-V VMs to be able to create Hyper-V containers.

I don't see any VMs running in Hyper-V, there is no documentation on Windows Container configuration. If I run "docker ps" I see the docker image get spun up but then keeps getting killed and recreated.
I see "Handler for GET /v1.29/containers: No such container" errors in the Event Log

-Also get this error this error .../v1.29/ ...  It seems misleading error. Because even if my containers work i still get this error in the event logs.
-Although virtualization for Hyper-V VMs worked for me.
-My containers occasionally keeps spanning another containers and then after a while is stable again.

Hyper-V containers uses Hyper-V Technology but you won't see them in the Hyper-V management.  For more info please see:
https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/hyperv-container
Regarding the containers getting killed and recreated, please look into the functions worker in the following windows event log for more details:

Microsoft-Windows-FunctionsRuntime/Operational

Please let us know if you find something there

I get a number of errors there, but also after the below errors I get "Functions Host Activation Service has started successfully!'
5002
WriteContainerLogsToDisk failed: System.FormatException: The string was not recognized as a valid DateTime. There is an unknown word starting at index 0.
at System.DateTimeParse.Parse(String s, DateTimeFormatInfo dtfi, DateTimeStyles styles)
at Microsoft.Functions.Runtime.Activation.Service.Containers.WindowsContainer.CaptureContainerLogsSinceTimeStamp(String[]& standardOut, String[]& standardErr)
at Microsoft.Functions.Runtime.Activation.Service.Containers.WindowsContainer.WriteConainerLogsToDisk()
5002
System.InvalidOperationException: Operation is not valid due to the current state of the object.
at Microsoft.Functions.Runtime.Data.FunctionManager.GetDefaultConnectionString(RoleType role)
at Microsoft.Functions.Runtime.Data.FunctionManager..ctor(RoleType roleType)
at Microsoft.Functions.Runtime.Activation.Service.FunctionAppOwnership.OnTimerElapsed(Object state)

I'm glad it finally worked.
We have just recently announced "Azure Functions Runtime" preview 2
https://blogs.msdn.microsoft.com/appserviceteam/2017/12/05/announcing-azure-functions-runtime-preview-2/
In this version we have fixed several issues including the String.FormatException. Additionally, we have  added a lot of new improvements.


Run with relation basis vectors

Hi,
Thank you for this wonderful work! I got the following error when I ran with "-num_bases 50" (as suggested in the paper): AttributeError: 'CompGCNConvBasis' object has no attribute 'cache'. Can I just remove Lines 40 and 41 in compgcn_conv_basis.py?

Yes, removing that should work fine, but the cache is useful for the link prediction and node classification tasks where we're operating on the same graph at each iteration. Right now, the code does the norm calculations for each iteration. I've made changes to the file in this commit. Feel free to close this after pulling the new changes. Let us know in case of further issues.

Thank you very much for your prompt reply!


Is it possible to connect any argb device with the arduino or do these work different?

I was thinking of linking all my argb devices including some fans and the lian li strimmer plus cause msi mystic light is really bad, is this theoretically possible or do these work different than normal ws2812 stripes?

yes theoretically its possible to link any device, as long as you know the type of leds it uses and how its wired.

thanks that sounds very good, what do you exactly mean by types of leds? I thought every argb devices uses ws2812 or something working the same, or could they have different ones when using an argb header?

Yes they use some kind of ledchipset, which can be configured with the FastLED library which is included in every example sketch.

In principle this would be to emulate iCue devices and studio.
If you have the original devices, there are other projects; where it identifies all the devices and synchronizes them all, without having to use MystiLight, iCue, etc.


mul got incompatible shapes for broadcasting

I'm trying to run this code in jax backend, and I've got a static inference shape result successfully. However, when I try to execute dynamically, I get this error message, which really confused me a lot.
Here is the code and what I got as results.
version:
python  3.10
keras  3.5.0
jax   0.4.31
import os
import re
import jax
import numpy as np
os.environ['KERAS_BACKEND']='jax'
import keras

layer = keras.layers.BatchNormalization(
    axis=-1,
    momentum=0.99,
    epsilon=0.001,
    center=False,
    scale=False,
    beta_initializer="zeros",
    gamma_initializer="ones",
    moving_mean_initializer="zeros",
    moving_variance_initializer="ones",
    synchronized=False,
    trainable=True,
    autocast=True,
)

result_static = layer.compute_output_shape([2, 3])

result_dynamic = layer(
    inputs=np.random.rand(*[2, 3]),
    training=True,
    mask=np.random.rand(*[4]),
)

The static result:

The dynamic result:

Hi @Isa-Fay,
I looked into this and found out that the mask argument has a shape of (4,), which is incompatible with the input shape of (2, 3). The mask should typically have the same shape as the input tensor, or be broadcastable to that shape.
You can use mask=np.random.rand(*[2, 3]) instead.

Thanks for your reply!
There is a problem with the input, but I think it's worth paying attention to the inconsistencies in the static and dynamic outputs, which may cause confusion and difficulty for users. Hopefully keras could add some input checks instead of passing illegal parameters directly to the backend.

Thank you for your feedback!
I agree that the inconsistency between the static and dynamic outputs can be confusing for users. Adding input checks at the Keras layer level would definitely help prevent these kinds of errors by validating shapes and dimensions before passing them to the backend.
I think I will create a Pull Request regarding the same by today

This is now working in Keras-nightly with the above linked PR fix, attaching the expected outcome in the Gist here


python38Packages.google-cloud-secret-manager: 2.7.0 -> 2.7.1

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for python38Packages.google-cloud-secret-manager is: "Secret Manager API API client library"
meta.homepage for python38Packages.google-cloud-secret-manager is: "https://github.com/googleapis/python-secret-manager"
meta.changelog for python38Packages.google-cloud-secret-manager is: ""
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS

Warning: a test defined in passthru.tests did not pass

/nix/store/ixcqy9d8k9s2vafjgxf1a00w3sz0skd5-python3.8-google-cloud-secret-manager-2.7.1/bin/.fixup_secretmanager_v1_keywords.py-wrapped passed the binary check.

/nix/store/ixcqy9d8k9s2vafjgxf1a00w3sz0skd5-python3.8-google-cloud-secret-manager-2.7.1/bin/.fixup_secretmanager_v1beta1_keywords.py-wrapped passed the binary check.

/nix/store/ixcqy9d8k9s2vafjgxf1a00w3sz0skd5-python3.8-google-cloud-secret-manager-2.7.1/bin/fixup_secretmanager_v1_keywords.py passed the binary check.

/nix/store/ixcqy9d8k9s2vafjgxf1a00w3sz0skd5-python3.8-google-cloud-secret-manager-2.7.1/bin/fixup_secretmanager_v1beta1_keywords.py passed the binary check.

4 of 4 passed binary check by having a zero exit code.

0 of 4 passed binary check by having the new version present in output.

found 2.7.1 with grep in /nix/store/ixcqy9d8k9s2vafjgxf1a00w3sz0skd5-python3.8-google-cloud-secret-manager-2.7.1

found 2.7.1 in filename of file in /nix/store/ixcqy9d8k9s2vafjgxf1a00w3sz0skd5-python3.8-google-cloud-secret-manager-2.7.1

directory tree listing: https://gist.github.com/1ad6d9c4f18d5f26892a5da694eaa33c

du listing: https://gist.github.com/0242ce689f10f82f1c4cfcde97458c45

Rebuild report (if merged into master) (click to expand)

8 total rebuild path(s)

2 package rebuild(s)

2 x86_64-linux rebuild(s)
2 i686-linux rebuild(s)
2 x86_64-darwin rebuild(s)
2 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
python38Packages.google-cloud-secret-manager
python39Packages.google-cloud-secret-manager

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/ixcqy9d8k9s2vafjgxf1a00w3sz0skd5-python3.8-google-cloud-secret-manager-2.7.1 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A python38Packages.google-cloud-secret-manager https://github.com/r-ryantm/nixpkgs/archive/b1511bd4c7406634ab1b04cd47514b90d82f3ebc.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/ixcqy9d8k9s2vafjgxf1a00w3sz0skd5-python3.8-google-cloud-secret-manager-2.7.1
ls -la /nix/store/ixcqy9d8k9s2vafjgxf1a00w3sz0skd5-python3.8-google-cloud-secret-manager-2.7.1/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

1 package failed to build:

python39Packages.google-cloud-secret-manager

1 package built:

python38Packages.google-cloud-secret-manager

Maintainer pings
cc @siriobalmelli @SuperSandro2000 for testing.

Result of nixpkgs-review pr 138222 at b1511bd4 run on aarch64-linux 1

2 packages built successfully:

python38Packages.google-cloud-secret-manager
python39Packages.google-cloud-secret-manager


Add hcloud images

I'm still testing around and am not 100% sure what I'm doing

BEAKER_PUPPET_COLLECTION=puppet6 BEAKER_setfile=ubuntu1804-64{hostname=ubuntu1804-64.example.com} BEAKER_HYPERVISOR=hcloud bundle exec rake beaker

brings me:
To use retry middleware with Faraday v2.0+, install `faraday-retry` gem
/usr/bin/ruby -I/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/rspec-core-3.12.0/lib:/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/rspec-support-3.12.0/lib /home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/rspec-core-3.12.0/exe/rspec --pattern spec/acceptance/\*\*/\*_spec.rb
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/beaker-rspec-7.1.0/lib/beaker-rspec/helpers/serverspec.rb:43: warning: already initialized constant Module::VALID_OPTIONS_KEYS
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/specinfra-2.83.4/lib/specinfra/configuration.rb:4: warning: previous definition of VALID_OPTIONS_KEYS was here

Hosts file 'ubuntu1804-64{hostname=ubuntu1804-64.example.com}' does not exist.
Trying as beaker-hostgenerator input.

Hypervisor for ubuntu1804-64.example.com is hcloud
Beaker::Hypervisor, found some hcloud boxes to create
Provisioning hcloud
Generating SSH keypair
Creating hcloud SSH key
provisioning ubuntu1804-64.example.com
invalid input in fields 'source_server', 'image'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/hcloud-1.1.0/lib/hcloud/typhoeus_ext.rb:54:in `check_for_error'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/hcloud-1.1.0/lib/hcloud/client.rb:150:in `block in request'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/typhoeus-1.4.0/lib/typhoeus/request/callbacks.rb:146:in `block in execute_callbacks'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/typhoeus-1.4.0/lib/typhoeus/request/callbacks.rb:145:in `each'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/typhoeus-1.4.0/lib/typhoeus/request/callbacks.rb:145:in `execute_callbacks'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/typhoeus-1.4.0/lib/typhoeus/request/operations.rb:35:in `finish'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/typhoeus-1.4.0/lib/typhoeus/easy_factory.rb:164:in `block in set_callback'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/ethon-0.16.0/lib/ethon/easy/response_callbacks.rb:74:in `block in complete'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/ethon-0.16.0/lib/ethon/easy/response_callbacks.rb:74:in `each'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/ethon-0.16.0/lib/ethon/easy/response_callbacks.rb:74:in `complete'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/ethon-0.16.0/lib/ethon/easy/operations.rb:34:in `perform'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/typhoeus-1.4.0/lib/typhoeus/request/operations.rb:16:in `run'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/typhoeus-1.4.0/lib/typhoeus/request/cacheable.rb:18:in `run'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/typhoeus-1.4.0/lib/typhoeus/request/block_connection.rb:31:in `run'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/typhoeus-1.4.0/lib/typhoeus/request/stubbable.rb:25:in `run'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/typhoeus-1.4.0/lib/typhoeus/request/before.rb:26:in `run'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/hcloud-1.1.0/lib/hcloud/client.rb:117:in `prepare_request'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/hcloud-1.1.0/lib/hcloud/abstract_resource.rb:9:in `prepare_request'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/hcloud-1.1.0/lib/hcloud/server_resource.rb:18:in `create'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/bundler/gems/beaker-hcloud-6d0512902e85/lib/beaker/hypervisor/hcloud.rb:76:in `create_server'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/bundler/gems/beaker-hcloud-6d0512902e85/lib/beaker/hypervisor/hcloud.rb:28:in `block in provision'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/bundler/gems/beaker-hcloud-6d0512902e85/lib/beaker/hypervisor/hcloud.rb:27:in `each'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/bundler/gems/beaker-hcloud-6d0512902e85/lib/beaker/hypervisor/hcloud.rb:27:in `provision'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/beaker-4.38.1/lib/beaker/hypervisor.rb:37:in `create'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/beaker-4.38.1/lib/beaker/network_manager.rb:72:in `block in provision'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/beaker-4.38.1/lib/beaker/network_manager.rb:71:in `each_key'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/beaker-4.38.1/lib/beaker/network_manager.rb:71:in `provision'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/beaker-rspec-7.1.0/lib/beaker-rspec/beaker_shim.rb:35:in `provision'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/beaker-rspec-7.1.0/lib/beaker-rspec/spec_helper.rb:57:in `block in <top (required)>'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/rspec-core-3.12.0/lib/rspec/core.rb:98:in `configure'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/beaker-rspec-7.1.0/lib/beaker-rspec/spec_helper.rb:5:in `<top (required)>'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/beaker-rspec-7.1.0/lib/beaker-rspec.rb:5:in `require'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/beaker-rspec-7.1.0/lib/beaker-rspec.rb:5:in `<module:BeakerRSpec>'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/beaker-rspec-7.1.0/lib/beaker-rspec.rb:1:in `<top (required)>'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/voxpupuli-acceptance-1.2.0/lib/voxpupuli/acceptance/spec_helper_acceptance.rb:12:in `require'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/voxpupuli-acceptance-1.2.0/lib/voxpupuli/acceptance/spec_helper_acceptance.rb:12:in `configure_beaker'
/home/bastelfreak/code/puppet-hdm/spec/spec_helper_acceptance.rb:8:in `<top (required)>'
/home/bastelfreak/code/puppet-hdm/spec/acceptance/docker_spec.rb:3:in `require'
/home/bastelfreak/code/puppet-hdm/spec/acceptance/docker_spec.rb:3:in `<top (required)>'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/rspec-core-3.12.0/lib/rspec/core/configuration.rb:2117:in `load'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/rspec-core-3.12.0/lib/rspec/core/configuration.rb:2117:in `load_file_handling_errors'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/rspec-core-3.12.0/lib/rspec/core/configuration.rb:1617:in `block in load_spec_files'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/rspec-core-3.12.0/lib/rspec/core/configuration.rb:1615:in `each'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/rspec-core-3.12.0/lib/rspec/core/configuration.rb:1615:in `load_spec_files'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:102:in `setup'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:86:in `run'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:71:in `run'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:45:in `invoke'
/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/rspec-core-3.12.0/exe/rspec:4:in `<main>'
/usr/bin/ruby -I/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/rspec-core-3.12.0/lib:/home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/rspec-support-3.12.0/lib /home/bastelfreak/code/puppet-hdm/.vendor/ruby/3.0.0/gems/rspec-core-3.12.0/exe/rspec --pattern spec/acceptance/\*\*/\*_spec.rb failed

aha!
$ bundle exec beaker-hostgenerator --hypervisor hcloud ubuntu2004-64
---
HOSTS:
  ubuntu2004-64-1:
    platform: ubuntu-20.04-amd64
    image: ubuntu-20.04
    hypervisor: hcloud
    roles:
    - agent
CONFIG:
  nfs_server: none
  consoleport: 443

a problem I noticed: We generate an FQDN like this: ubuntu1804-64.example.com. We will create two instances with that name, one for Puppet 6 and one for Puppet 7. At Hetzner, a project cannot contain multiple virtual machines with the same FQDN. If we assume we've one project for the whole Vox Pupuli organisation, the name needs to contain puppet+PR info, or a random string? I don't think that the name is actually important for the running tests.

FQDN generation is something we now do here: https://github.com/voxpupuli/puppet_metadata/blob/1d92f203598362d9719971cb5b6cab6c085f9689/lib/puppet_metadata/beaker.rb#L48
I'd actually prefer some option to set this and an env var like BEAKER_HOSTGENERATOR_DOMAIN where beaker-hostgenerator generates the FQDN.
Then for the hcloud hypervisor it could actually add some randomness to it. For reference, libvirt has this: https://github.com/voxpupuli/beaker-vagrant/blob/50d671f73862e7a2271b1b361036794489a64672/lib/beaker/hypervisor/vagrant_libvirt.rb#L7-L11

@ekohl I considered modifying the image name in the hcloud hypervisor file. but hcloud only supports a tiny subset of all the operating systems beaker-hostgenerator has. Filtering that might add a bit too much complexity. Or do you think we shouldn't filter that and generate all operating systems for hcloud?

Or do you think we shouldn't filter that and generate all operating systems for hcloud?

I think this. Just like we generate a docker image name on the assumption that it exists. It may exist in the future and otherwise you get an error from the hypervisor. For example, it's reasonable to assume that at some point there will be an image for Ubuntu 24.04 and it's nice not to need any code change for it when it comes out.

@ekohl I refactored the image generation. I just need to think about the FQDN now.


Extend Eloquent Model instead of just ArrayAccess

Description
Change Stancl\Tenancy\Tenant to extend Illuminate\Database\Eloquent\Model  instead of ArrayAccess.
Why this should be added
In order to use the Tenant model for various patterns in the Laravel framework like: Observer, guards, eloquent builder, then it must extend the Illuminate\Database\Eloquent\Model.
Laravel ecosystem tools like Scout and Nova relies on Illuminate\Database\Eloquent\Model classes.

I might refactor the DB storage driver to use models (in 2.2.0 I refactored away from that solution because it was very messy). But Stancl\Tenancy\Tenant definitely can't be a model because it's independent of the underlying storage. Also it would introduce a lot of bloat and messiness.


added Eph English syntax analysis Example

🚀 Deployed on https://deploy-preview-57--cbcgb-com-sgbs-training.netlify.app


fix: Update swarm example for Jakarta

Fixes https://github.com/edgexfoundry/edgex-docs/issues/516
Signed-off-by: Bryon Nevis<EMAIL_ADDRESS>PR Checklist
Please check if your PR fulfills the following requirements:

[ ] I have added unit tests for the new feature or bug fix (if not, why?)
[X] I have fully tested (add details below) this the new feature or bug fix (if not, why?)

Related docs PR:
https://github.com/edgexfoundry/edgex-docs/pull/672
Testing Instructions
https://docs.edgexfoundry.org/2.2/security/Ch-Docker-Swarm-SecuringDeviceServices/

Converting back to draft until I have time to make edits.

Will not fix.  See ssh+spiffe example instead.


Add OSS projects, rename VMware correctly, and remove Heptio

Signed-off-by: jonasrosland<EMAIL_ADDRESS>Pre-submission checklist:
Please check each of these after submitting your pull request:

[x] Are you only including a repo_url if your project is 100% open source? If so, you need to pick the single best GitHub repository for your project, not a GitHub organization.
[x] Is your project closed source or, if it is open source, does your project have at least 250 GitHub stars?
[x] Have you picked the single best (existing) category for your project?
[x] Does it follow the other guidelines from the new entries section?
[x] Have you included a URL for your SVG or added it to hosted_logos and referenced it there?
[x] Does your logo clearly state the name of the project/product and follow the other logo guidelines?
[x] Does your project/product name match the text on the logo?
[x] Have you verified that the Crunchbase data for your organization is correct (including headquarters and LinkedIn)?
[ ] ~5 minutes after opening the pull request, the CNCF-Bot will post the URL for your staging server. Have you confirmed that it looks good to you and then added a comment to the PR saying "LGTM"?

Deploy preview for landscape failed.
Built with commit 68e7c227be63571027e98d6567be673e95466d1e
https://app.netlify.com/sites/landscape/deploys/5c78424c653dc70008b40887

Hi, Jonas. Thanks for the updates. The build failed with an error that it couldn't find heptio.svg so if you delete the file you need to remove all references to it.
Separately, we need all of the new logos to include the project name. So, please locate the SVG version of https://github.com/heptio/velero/blob/master/docs/img/velero.png or find the original AI and covert at cloudconvert.com.
Please Slack me if you have questions.

Thank you @dankohn, I'll get that fixed ASAP.

LGTM


RFC: Firebase and other potential tdo enhancements

I figured that Firebase could be used to remove some of the limitations which are now present in tdo:

Concurrent use is not allowed
Boards cannot be shared between users
No proper login, github tokens feel hacky

Firebase would enable:

Proper user login (Google, Facebook, GitHub)
Concurrent use on multiple devices with real-time sync
Shared boards between users
Offline use

I'm sure all these would be great, however, I'm not sure how much time this would take and how many people would be interested in these features?
What are the other things missing in tdo?

@nnako
Yes, I meant Google Firebase. It has a free plan which seems sufficient. Also, please note that tdo is based on Cx which is free for projects of non-commercial nature.
The issue regarding mobile devices should be easy to fix. I wasn't aware of it.
I'm not sure regarding other features. I built tdo as a side project to manage my tasks. I wanted simplicity and keyboard support. What you're proposing is a full blown project management solution. What is the gain for people who are currently using tdo from all that?  This seems to me like a completely new product.

@mstijak
thanks for your replies.
My goal would be to push the "frontend user interface side" of tdo just a bit further in order to display not just the basic TODO (GTD) features but some "control oriented" items as well. In my daily work as a private person, I try to combine calendar items from different sources and different persons. All these items have the character of TODOs, so I want to get away from using different programs (Thunderbird, LotusNotes, EXCEL, remind, ...) but have a universal application. The solution I see is:

creating a universal user interface (tdo seems perfect for this) and
creating a common text-based file interface to store exported items (example: todo.txt)
developing import and export filters to convert from the formats to the universal file format

So, file-based import and export would be essential for me in order to split and merge these different lists.
I have about 20 different private projects at the same time with different groups of team members. And because of this number of projects, I need further functionalities (like "deadlines", "scores", "responsibles", ...) directly displayed on the frontend user interface. For every project member to see and for the backend to access them. So that I don't have to track every single item myself, but the system would be able to see all the measures. What I call "backend" is the layer of functionalities which take these "scores", "deadlines", ... and combine them in a reasonable way and maybe generate further actions (automatically). For this backend to grow, I would wish for a simple Python interface, as I already have developed most of the functionalities.
This programming interface could be my own contribution. For all "normal" users it would be optional to use. Not mandatory. They don't have to take note of it. So, in my view, tdo just needs these following additional features which I would like to support developing:

"attributes" like score, deadline, responsible person,... for each task
file-based import and export of tasks in a format which is to be defined
a good defined modular function interface e.g. for python modules (or JavaScript if you like)

The non-commercial nature is ok for now, but I would like to be able to switch to commercial mode maybe in a few years, depending on the "success" of these projects and the experiences we make while working with the tdo system. In general, I prefer completely free open source solutions for these things. And I would always try to strip off those bindings in the long run.
For tdo we could try to put together these things without damaging the existing features.
What do you think?

@nnako: I'm sorry to disapoint you, but I have no interest in implementing these features. Feel free to fork the project and implement these by yourself.

@mstijak: no problem. A pitty, but that's life sometimes. I would be happy if you could occasionally help me find relevant places within the tdo code. I'm not sure if I will be able to fully understand the application in order to get started with my implementations.
Thank you for tdo and thank you for any help.

@mstijak I find tdo to have all i desire but one feature, a due date. I would be more than happy to implement this myself, however I would need to know where I would implement this feature in the codebase. I was thinking possibly when a user edits a task they would have at the front HH:MM/DATE where DATE is either tdfor today,tm for tomorrow or M/D/Y where the last one is chosen from either that or D/M/Y in settings. Could you direct me to the proper part of the codebase?

@AlexDvorak I think that's a good idea. I have started working on the new Firebase based tdo and had a similar enhancement in mind: https://github.com/codaxy/tdo/issues/7
In order to implement this feature, we should add a parsing function to extract data from the entered text. Also, the task render function should be updated to present additional information.
This file is a good starting point.
https://github.com/codaxy/tdo/blob/master/app/routes/board/Task.js
Since starting v2, I have no plans to maintain this repository anymore. If you wish to submit a PR, please do it in the new repository - https://github.com/codaxy/tdo


Parsing Content-Disposition header fails because of simple quotes

I have the following Content-Disposition header:
Content-Disposition: attachment; filename="Cours%20de%20l'oeuf%20jusqu'au%2024%20mars%202017.xlsx"

It fails to be parsed correctly using http.parse_options_header because the simple quotes are confusing the regexp. It worked correctly before the merge of pull request #933 fixing issue #932 which didn't include simple quotes in the regexp
Not sure how to fix this, though

@pszalanski do you have any idea ?

@untitaker knows more about this, but I don't think that's a valid use of quotes.

From what I understand, the only forbidden value is " (double quote) which must be escaped.
In any case this header was generated by latest chrome version with a file containing simple quote in its filename on the disk. Not something I can change easily
Also if there's only one simple quote in the file name (apostrophe for example), then it works as expected.

Why is Chrome generating an attachment header that's sent to your Werkzeug server? Not saying you're wrong, just curious.

It's a file upload using javascript, payload is in the body itself (not form encoded or multi part upload)

The problem seem to be two-fold (if I read the RFC and regexp correctly): it tries to find encoding and language even though no '*' is present before the '=', and both encoding and language is captured as anything but whitespace ([^\s]) while RFC 5987 states otherwise.
>>> _option_header_piece_re.match('; name=utf-8\'en\'test; filename="some_file.txt"').groups()
('name', 'utf-8', 'en', 'test')

Rearranging the regexp to look like below seem to work. At least for these two cases:
_quoted_string_re = r'"[^"\\]*(?:\\.[^"\\]*)*"'
_option_header_piece_re = re.compile(
    r';\s*(%s|[^\s;,=\*]+)\s*'
    r'(?:\*=\s*?(?:([^\s]+?)\'([^\s]*?)\'))?(?:=?)(%s|[^;,]+)?\s*' %
    (_quoted_string_re, _quoted_string_re))

>>> _option_header_piece_re.match('; name=utf-8\'en\'\'test\'; filename="some_file.txt"').groups()
('name', None, None, "utf-8'en''test'")

>>> _option_header_piece_re.match('; name*=utf-8\'en\'test; filename="some_file.txt"').groups()
('name', 'utf-8', 'en', 'test')

>>> _option_header_piece_re.match('; name="\'test\'"; filename="some_file.txt"').groups()
('name', None, None, '"\'test\'"')

Right, so the part about allowing double quotes in encoding and language doesn't actually make a difference if you correct the part about the asterisk. But if you just correct the quotes part then it works for quoted strings.


feat(purl): first purl iteration

Guessing purl thanks to cpe2purl database, upstream purl (e.g. github, gitlab, sourceforge) are preferred over distribution specific purl (e.g. debian, ubuntu, fedora, etc.)
In this first iteration, only json and cyclonedx is handled. Moreover, purl2cpe can't be installed through pypi so purl2cpe.db has to be manually built. I couldn't push it as database is too big (around 350 MB).

@ffontaine I have now had a look at the purl2cpe database. As you have noticed, the purl's don't have any version information included which is very disappointing. Is there any reason why the upstream sources have been limited? Why not include the language ecosystems such as pypi, npm or distro sources such as deb?
This change would require the database to be included as part of the lib4sbom. I suggest it is stored in a separate directory in the same way the license data is maintained and then accessed in a similar way (see license.py). Maybe create a new class PurlGenerator?

Hey @anthonyharrison!
as @terriko mentioned in #3771:

Some things won't have CPE entries and thus won't be in purl2CPE. But we may know (from bug reports) that there's a product with the same name that is absolutely not the same thing. So we'll need to provide a "is not" database to reduce false positives.
I suggest using a similar setup to what purl2cpe does -- allow humans to submit pull requests, make all the data readable, provide a way to load it into a queryable database.

we are gonna need to make a database similar to purl2cpe to incorporate purls/products that doesn't have CPEs, so would it be reasonable to maintain the whole database in-house instead of making a new one? by doing so we could also include version info from cpe into purl which is given in the database:
pkg:github/silverstripe/silverstripe-framework|cpe:2.3:a:silverstripe:framework:4.13.25:*:*:*:*:*:*:*

As you said in this comment, some decisions needs to be made.
I was thinking, since we will be generating purl ourselves with more info like version, subpath i.e: #3833
So why not maintain our own database with more informative purls, which might help in more precise mapping (I may be wrong here).
And in cases where something doesn't have CPE, we can add an appropriate entry for that.
All of this is based on the assumption that we will be utilizing purl2cpe via installing their database.
P.S: I'm in the process of making GSOC proposal, and am a little confused how would purl2cpe integration would help without version info.

@Dev-Voldemort  I can see various discussions about GSOC and purls. Can I suggest you keep the discussions on the GSOC thread and not in lib4sbom.
Lib4sbom is a SBOM generator/parser library. The data (e.g. PURL) should be coming from the script which is using lib4sbom the calling script should be responsible for ensuring the data is correct. Validating that the purl is correct (other than it is of the correct format) is not the responsibility of the library.


Add Metadata (or set an script) to Tileset Tiles to determine behaviour on contact with specific tiles (grates, oneway platforms, vertical stairs, grass, etc)

Example: when in tileset setup scene and creating a tile StaticBody2D or KinematicBody2D, be able to set an script and set something like:
# grate.gd
# this is the Tile
extends KinematicBody2D
func _ready():
     set_meta("tile_type", "grate")

This way in the character collision event when can so something like this:
# player.gd
extends KinematicBody2D
func _fixed_process(delta):
     if(is_colliding()):
          if(get_collider().has_meta("tile_type")):
               if(get_collider().get_meta("tile_type") == "grate"):
                    print("you can go through if your character has acquired a mist form")
               if(get_collider().get_meta("tile_type") == "dropdown"):
                    print("you can drop down if you press down+jump")
               if(get_collider().get_meta("tile_type") == "verticalstair"):
                    print("you can climb up if you press up")
               if(get_collider().get_meta("tile_type") == "grass"):
                    print("you can play the sound of walking on grass")

Right now this can be done with a separated KinematicBody2D on the scene, but it would be simpler to define this just one time on the Tileset scene and then be reusable as a tile instead of creating many separated objects on the scene and setting each one of them as something different, like a grate, platforms that you can dropdown from, vertical stairs, foolr types to play a different walking sound, etc...
Thanks in advance
Regards

I think, this will help to set up stages of block puzzle game also.
:+1:

Just updated the sample a little to give more examples of possible functionality (this is thinking in a Platformer/Metroidvania case, but I think the possibilities are endless)

Updated again to demonstrate the idea of different walking sound according to the Tile Meta Data (grass, wood, metal, etc. there are so mny possibilities)

This is a brilliant idea, the possibilities for this feature is just beyond endless.  : )

Duplicate of #12634 (more recent but it has more visibility).


HBASE-25163 Increase the timeout value for nightly jobs

Increase the timeout value to 16 hours.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+0 :ok:
shelldocs
0m  0s
Shelldocs was not available.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

_ master Compile Tests _

+0 :ok:
mvndep
0m 22s
Maven dependency ordering for branch

_ Patch Compile Tests _

+0 :ok:
mvndep
0m  7s
Maven dependency ordering for patch

+1 :green_heart:
shellcheck
0m  0s
There were no new shellcheck issues.

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.


Use themable SVGs in react native for common icons

Problem solved
Short description of the bug fixed or feature added

PR-Codex overview
This PR updates wallet icons and functions across multiple components for improved authentication provider visuals.
Detailed summary

Updated wallet icons for email, phone, and passkey authentication providers.
Refactored getAuthProviderImage function to handle different authentication providers.
Replaced getWalletIcon calls with getAuthProviderImage in multiple components.

The following files were skipped due to too many changes: packages/thirdweb/src/react/core/utils/walletIcon.ts

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

#4462  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @joaquim-verges and the rest of your teammates on  Graphite


Issue with Text File reporter and Json formatting

Hi,
I am using the Text file reporter, but I see two issues:

The output file only includes the last reading value. It overrides older values an only keeps the  last value in the file
I use Json formatting but the output is not in Json.

This is the code I am using to build the reporter:
 private void ConfigureMetrics(IServiceCollection services)
        {

            services.AddMetrics()
             .AddJsonSerialization()
             .AddJsonMetricsSerialization()
             .AddJsonHealthSerialization()
             .AddHealthChecks()
            .AddReporting(factory =>
            {

                factory.AddTextFile(
                new TextFileReporterSettings
                {
                    ReportInterval = TimeSpan.FromSeconds(reportingInterval),
                    FileName = metricFile
                });
                factory.AddConsole(
                  new ConsoleReporterSettings
                  {
                      ReportInterval = TimeSpan.FromSeconds(reportingInterval),
                  });

            });
        }

       private void ConfigureServices(IServiceCollection services)
        {

            var loggerFactory = new LoggerFactory();
            loggerFactory.AddConsole((l, s) => s == LogLevel.Trace);
            services.AddSingleton<ILoggerFactory, LoggerFactory>();
            services.AddLogging();
        }

//where I build the reporter
          public MetricCollector(CancellationToken token)
        {
            IServiceCollection serviceCollection = new ServiceCollection();

            ConfigureServices(serviceCollection);
            ConfigureMetrics(serviceCollection);

            var provider = serviceCollection.BuildServiceProvider();
            provider.GetRequiredService<IMetrics>();

            metrics = provider.GetRequiredService<IMetrics>();

            var reporterFactory = provider.GetRequiredService<IReportFactory>();
            var reporter = reporterFactory.CreateReporter();
            // start in a separate thread
            Thread thread = new Thread(new ThreadStart(() => reporter.RunReports(metrics, token)));
            thread.Start();

        }

And this the the output I get:
# MEASUREMENT: [Application.Health] Score
# TAGS:
               app = Microsoft.Azure.Devices.Edge.Hub.Service
            server = INTERN-SB-08
               env = release
             mtype = gauge
              unit = Health Score
# FIELDS:
             value = 1
--------------------------------------------------------------
# MEASUREMENT: [Application] Input Rate Meter
# TAGS:
               app = Microsoft.Azure.Devices.Edge.Hub.Service
            server = INTERN-SB-08
               env = release
             mtype = meter
              unit = calls
         unit_rate = min
# FIELDS:
       count.meter = 31
            rate1m = 60
            rate5m = 60
           rate15m = 60
         rate.mean = 61.4064259432144
--------------------------------------------------------------
# MEASUREMENT: [appmetrics.internal] report_success  items
# TAGS:
               app = Microsoft.Azure.Devices.Edge.Hub.Service
            server = INTERN-SB-08
               env = release
              item = TextFileReporterProvider`1
             mtype = counter
              unit = items
# FIELDS:
             total = 1
           percent = 100
--------------------------------------------------------------
# MEASUREMENT: [appmetrics.internal] report_success
# TAGS:
               app = Microsoft.Azure.Devices.Edge.Hub.Service
            server = INTERN-SB-08
               env = release
             mtype = counter
              unit = items
# FIELDS:
             value = 1
--------------------------------------------------------------

This is the total output file, despite that I see multiple reading in the console reporter.
I am using the latest versions available in nuget.

opened issue in main AppMetrics repository

Writing only last values is intended, I guess it depends on why you're using a text file to persist metrics. We can make this a configurable option.
This is confusing, I have a task to move those serialization extension methods, these only apply to middleware components. What you need to do instead is to add the desire payload builder to your reporter, see http://app-metrics.io/reporting/text-file.html

Thanks for the quick reply.

Yes I want to persist to a file. I would appreciate if you can make this configurable.
Is there any example on how I should build my own payload builder?

At the moment that's very low priority for my use and don't see it valuable enough for others to jump on it right now, happy to accept a PR though.
http://app-metrics.io/reporting/formatting/index.html

Closing this one, I've created a task in the reporting report for the text file reporting adjustments see above ^


optimize interpolation_layer.py

if scale_factor is 1 (1mm resolution), interpolation is the identity operation and my be skipped
Adressing https://github.com/Deep-MI/FastSurfer/issues/187
Speed improvement is about 1% (205 to 203 seconds on cpu)
Tested on bert -- identical for both cpu and gpu

It is checking for all resolutions to be equal to 1 but what if the fixed inner resolution is not 1 mm ? Shouldn't it check if voxel size is equal to inner resolution?

the inner resolution is unknown to the interpolation layer, it checks, if all samples in the batch have scale_factor 1 (which is the case if len(scales) == 1 and scales[0] == ndarray/Tensor(1,1). (the scale factor is already the fraction of inner and outer resolution)
So this is a more bare solution that changes the interpolation layer to skip interpolation if none is required.

Also since speed improvement is negligible I still wonder where the significantly increased run times in VINN come from, if not the interpolation layer.

I guess we could profile the run times....


[CMake] Configure standard library dependency on 'libSwiftScan' in the latter target definition

Swift configures the standard library cmake build before the 'tools' build, which means the dependency cannot be established from the former, because the tools dependency targets have not yet been created.

@swift-ci smoke test

@swift-ci smoke test

@swift-ci smoke test


Make "legacy" database routing opt-in instead of opt-out

Also, adds support for a read-only replica DB, and puts the list of apps in django settings, instead of the router module.
Changes

Consolidated two routers into 1
"legacy" routing made opt-in, instead of opt-out
added support for a read-only "replica" database.
the new "opt-in" list is now in settings.base

Checklist

[ ] PR has an informative and human-readable title
[ ] Changes are limited to a single goal (no scope creep)
[ ] Code can be automatically merged (no conflicts)
[ ] Code follows the standards laid out in the development playbook
[ ] Passes all existing automated tests
[ ] New functions include new tests
[ ] New functions are documented (with a description, list of inputs, and expected output)
[ ] Placeholder code is flagged
[ ] Visually tested in supported browsers and devices
[ ] Project documentation has been updated
[ ] Reviewers requested with the Assignee tool :arrow_right:

I can't give this the review it deserves, but I give it all my love and hope for the future.

@chosak DATABASE_ROUTING now only controls the insertion of the 'legacy' database.

I spent some time re-reading GHE issue 453, and it seems like there was agreement that:

eregs
retirement
paying for college 2

Could go on just fine without the split databases. This leaves:

Picard (itself, on the chopping block)
comparisontool (paying for college 1)
credit card agreements

Pinging @higs4281 for a gut check on retirement and PFC2

gut check on retirement and PFC2

I think it would be great to include both in the unified database. The data flow for PFC2, in particular, would get simpler.


[Create news] New Tag is not shown among other tags

Environment: Windows 10 Pro, Mozilla Firefox 82.0.3 (64-біт)
Chrome 86.0.4240.198 (Розробка) (64-біт)
Reproducible: always
Build found: 05.12.2020
Preconditions
New tag is added in Database and at least 1 news is created in Database with this tag (e.g. 'Testing').
Steps to reproduce

Go to https://ita-social-projects.github.io/GreenCityClient/#/welcome
Sign in (e.g<EMAIL_ADDRESS>'Password'="Green.city#500")
Click on 'Eco news' button
Click on 'Create news' button
Pay attention to block with tags.

Actual result
Tag 'Testing' is not present among other tags.

Expected result
Tag 'Testing' is present among other tags.

I tried creating news on Swagger in "Eco News Controller" and the news were created with tag "Testing". When I clicked 'Edit news' button on the web page tag was also present.

New tag was successfully added through Management page.


Mention dependencies (ie. php 5.4)

Please do mention what are the system requirements ... for example now I found out that running the command
php composer.phar require "voryx/thruway":"0.3.*"

returns error
 Problem 1
    - voryx/thruway 0.3.1 requires php >=5.4 -> no matching package found.
    - voryx/thruway 0.3.0 requires php >=5.4 -> no matching package found.
    - Installation request for voryx/thruway 0.3.* -> satisfiable by voryx/thruway[0.3.0, 0.3.1].

Thanks

Thruway does require php 5.4 for many reasons. We use language features that are only available in 5.4 (traits, array syntax) and there are dependencies that require 5.4 (reactphp/react, ratchet/pawl, etc.).

I do not doubt it in any way, just until I opened the composer file, I had no idea such dependancy exists, would be fine to mentioned in the docs as requirement. My whole installation of crossbar.io failed on that, as I am not able to install the PHP component because of php 5.3 ("non negotiable must be" as my colleagues call it :) ). Would be nice to know it in advance, thats all :)

Updated README with requirements https://github.com/voryx/Thruway/commit/db8870a2162b6263b3fdd80a7a502f3338b922d2


Sessions are not yet supported?

session_id parameter for the connection can be used to setup a session with Clickhouse
One application is using TEMPORARY TABLE

I don't think HTTP connection has session support.


updating exposed scope property within directive's link function cause exception in v1.4.9

Updating an exposed scope property within directive's link function cause exception in v1.4.9, with not in v1.4.8.
I couldn't find the relevant change in the changelog.
Here is a demo to show the problem:
https://jsfiddle.net/kkop339m/
Change the script tag to import v1.4.9 and run the code with console open.
You will see the following exception:

Error: [$compile:nonassign] Expression 'undefined' used with directive 'sampleDirective' is non-assignable!
http://errors.angularjs.org/1.4.9/$compile/nonassign?p0=undefined&p1=sampleDirective
at angular.js:68
at parentSet (angular.js:9092)
at parentValueWatch (angular.js:9105)
at Object.regularInterceptedExpression (angular.js:14646)
at Scope.$digest (angular.js:16087)
at Scope.$apply (angular.js:16359)
at bootstrapApply (angular.js:1680)
at Object.invoke (angular.js:4535)
at doBootstrap (angular.js:1678)
at bootstrap (angular.js:1698)

Same code runs in v1.4.8 with no errors.

We've fixed a regression with that in 1.4.9. It's expected that this throws, because you haven't marked your scope definition with ? to make it optional. It's not in the changelog, because the fix is in parse and not in compile: https://github.com/angular/angular.js/commit/4473b81cdaf16c5509ac53d80b9bdfb0a7ac5f30
Here's the original issue: https://github.com/angular/angular.js/issues/13367
So it's not a regression, but it restores the behavior we actually want. The problem is that the original behavior is "lost" since 1.4, so for quite a while.

Thanks @Narretz!

Thanks @Narretz !
Maybe a note about it in the "breaking changes" section would assist, because it cause one of my apps to break and it took a while to figure out the reason.

That's the thing, it's not technically a breaking change, because we fixed a regression.

Regression or not.  It was there for 9 versions over 8 months... This actually breaks applications that might have missed a parameter but continued to work.  And its not even listed in the changelog...

SOOOOOOoooooooo, what this all spells out to me, is that I should just put a question mark after EVERY scope attribute, just to be safe and work like old angular 1.4.7 worked??????? This is  whacky and hardly covered in any changelog.

Don't get me wrong.  I am a fan of fixing this behavior.  However for us it is a breaking change as in some cases people missed marking properties as optional but treated it as such.  However as a very large application it is impossible to know where we might have done this.  For now I have upgraded to 1.4.9 / 1.5.0 but modified the angular source to treat it as a warning so we can fix without loss of functionality.
Relevant source:
https://github.com/angular/angular.js/blob/4883e957971f1c551a0f560427bba0224f291449/src/ng/compile.js#L3032

Thank you for the reply. I've never needed nor ever used the optional attribute specification even though it seems to have existed since the beginning of Angular. And so for me, I can't see why we need an optional specifier at all, especially when my 1.4.6 code works so great. Specifying something as optional, just doesn't feel like JavaScript at all. But mass adding a ? to all me two-way binds =? fixed my code across the board without checking if any scope attribute actually needs it. Thanks again

The scope definition is a bit like an interface for your directive. So you can define explicitly to the consumer if a property must be set or can be set. To me that's a strength, not a weakness.

Narretz, a declaration of an optional attribute, is documentation and should not cause code to break. Yo, nice spin and healthy light on the subject but it was correct for how code should flow in older Angular versions. I should be able to default a scope attribute that wasn't supplied, without specifying it

Hey, I get it though, directives being interface like and being an implementation contract of sorts.  O well


peerinvalid Peer<EMAIL_ADDRESS>wants gulp@>=4

I met this error when npm install gulp-debug
peerinvalid Peer<EMAIL_ADDRESS>wants gulp@>=4

The latest gulp version is 3.9.1. How can I install gulp@>=4?
I have to npm install gulp-debug@3

https://github.com/gulpjs/gulp/releases/tag/v4.0.0

@sindresorhus Weird. I can only get 3.9.1 with npm install gulp@latest.


misp-modules keeps restarting

I have just done a fresh install of MISP-dockerized 1.1.1, on a Digital Ocean Ubuntu 18.04 Docker droplet, and misp-modules restarts over and over again with the following log.
No module named 'requests'
2019-08-01 15:52:30,989 - misp-modules - INFO - Launch MISP modules server from current directory.
2019-08-01 15:52:30,991 - misp-modules - INFO - Helpers failed cache.py due to Redis not running or not installed. Helper will be disabled.
2019-08-01 15:52:30,993 - misp-modules - INFO - MISP modules osqueryexport imported
2019-08-01 15:52:30,993 - misp-modules - INFO - MISP modules testexport imported
2019-08-01 15:52:30,994 - misp-modules - INFO - MISP modules threat_connect_export imported
2019-08-01 15:52:30,995 - misp-modules - WARNING - MISP modules pdfexport failed due to No module named 'pymisp'
2019-08-01 15:52:30,996 - misp-modules - INFO - MISP modules threatStream_misp_export imported
2019-08-01 15:52:30,997 - misp-modules - INFO - MISP modules nexthinkexport imported
2019-08-01 15:52:30,997 - misp-modules - INFO - MISP modules cef_export imported
2019-08-01 15:52:30,998 - misp-modules - INFO - MISP modules cisco_firesight_manager_ACL_rule_export imported
2019-08-01 15:52:30,999 - misp-modules - INFO - MISP modules liteexport imported
2019-08-01 15:52:31,000 - misp-modules - WARNING - MISP modules goamlexport failed due to No module named 'pymisp'
2019-08-01 15:52:31,002 - misp-modules - INFO - MISP modules mispjson imported
2019-08-01 15:52:31,011 - misp-modules - INFO - MISP modules email_import imported
2019-08-01 15:52:31,012 - misp-modules - WARNING - MISP modules csvimport failed due to No module named 'pymisp'
2019-08-01 15:52:31,013 - misp-modules - INFO - MISP modules ocr imported
2019-08-01 15:52:31,017 - misp-modules - WARNING - MISP modules goamlimport failed due to No module named 'pymisp'
2019-08-01 15:52:31,018 - misp-modules - INFO - MISP modules threatanalyzer_import imported
2019-08-01 15:52:31,019 - misp-modules - INFO - MISP modules testimport imported
2019-08-01 15:52:31,021 - misp-modules - WARNING - MISP modules joe_import failed due to No module named 'pymisp'
2019-08-01 15:52:31,023 - misp-modules - WARNING - MISP modules vmray_import failed due to No module named 'requests'
2019-08-01 15:52:31,023 - misp-modules - INFO - MISP modules cuckooimport imported
2019-08-01 15:52:31,024 - misp-modules - WARNING - MISP modules openiocimport failed due to No module named 'pymisp'
2019-08-01 15:52:31,025 - misp-modules - WARNING - MISP modules stiximport failed due to No module named 'pymisp'
yara is missing, use 'pip3 install -I -r REQUIREMENTS' from the root of this repository to install it.
2019-08-01 15:52:31,028 - misp-modules - INFO - MISP modules yara_query imported
2019-08-01 15:52:31,029 - misp-modules - WARNING - MISP modules joesandbox_query failed due to No module named 'jbxapi'
2019-08-01 15:52:31,030 - misp-modules - WARNING - MISP modules securitytrails failed due to No module named 'dnstrails'
dnspython3 is missing, use 'pip install dnspython3' to install it.

I tried to build an image from dcso/misp-dockerized-misp-modules:2, running pip3 install -I -r REQUIREMENTS and pip3 install dnspython3, but the docker-build fails with:
Collecting opencv-python==<IP_ADDRESS> (from -r REQUIREMENTS (line 41))
  Could not find a version that satisfies the requirement opencv-python==<IP_ADDRESS> (from -r REQUIREMENTS (line 41)) (from versions: )
No matching distribution found for opencv-python==<IP_ADDRESS> (from -r REQUIREMENTS (line 41))
You are using pip version 18.1, however version 19.2.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
The command '/bin/sh -c cd /usr/local/src/misp-modules && pip3 install -I -r REQUIREMENTS' returned a non-zero code: 1

Is there anything that I can do about that?
Thanks.

Hi @carey-at-xq,
Sorry for my late answer.
You have two options at the current time. use our dev image: dcso/misp-dockerized-misp-modules:3-dev or wait until I released v.1.2.0. (I hope in this week if all is working).
The best is go to your config/config.env file and add MISP_MODULES_CONTAINER_TAG=3-dev

Thanks, Max. Will do.
On Mon, 5 Aug 2019 at 11:53, Max H<EMAIL_ADDRESS>wrote:

Hi @carey-at-xq https://github.com/carey-at-xq,
Sorry for my late answer.
You have two options at the current time. use our dev image:
dcso/misp-dockerized-misp-modules:3-dev or wait until I released v.1.2.0.
(I hope in this week if all is working).
The best is go to your config/config.env file and add
MISP_MODULES_CONTAINER_TAG=3-dev
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/DCSO/MISP-dockerized/issues/80?email_source=notifications&email_token=AL7AAPCFTMPM4G4TKJLZB6DQDABEHA5CNFSM4IISKYSKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3ROQDQ#issuecomment-518187022,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AL7AAPH3Z2VYNS6ZVF4276TQDABEHANCNFSM4IISKYSA
.


markdownify summary and abstract in list.html

Purpose
Describe the purpose of this change. If there is an existing issue that is resolved by this pull request, please reference it in the form Fixes #1234 where 1234 is the relevant issue number.
Screenshots
before:

after:

@gcushen  ping


Update readme.content.md

synched changes between readmes

no need to create Pull Request per each file; added in https://github.com/MarshalX/atproto/pull/492

Made additional language requests per request.


Weebpack configuration

I am using typescript in my expressjs project and I am adminjs too, so can I build my expressjs project with webpack, is it possible or not?

If you mean the frontend, then no. AdminJS uses it's own Rollup configuration for bundling custom components which you'd have to replicate in Webpack to produce the same type of files.

Okay, Thank you


Write access to the team compass for council members

I think it would make sense to give write access to all council members to this team compass repo.
So they could for example edit the top comment of the weekly notes (https://github.com/jupyter/notebook-team-compass/issues/5), close issues and make changes to the docs site.
@jupyter/notebook-council

This would require one of the owners of the Jupyter organization on GitHub to make that change.


Does not clear up the html part nor picks up CSS

Thanks for this gem!
With Rails 3.2.1 and mailers that use layouts it seems to keep the html_part with  tags, and it doesn't seem to find the css file.
This is how I include the stylesheet
!!! HTML
%html
  %head
    %meta(content="text/html; charset=UTF-8" http-equiv="Content-Type")
    = stylesheet_link_tag 'mailers/all'
    = yield(:head)
  %body
    #content
      = yield

Thanks for this report. I am planning to update one of our Rails apps to
3.2 very soon, and will look into this issue.
On Tue, Jan 31, 2012 at 3:14 AM, Gleb Mazovetskiy <
<EMAIL_ADDRESS>
wrote:

With Rails 3.2 and mailers that use layouts it seems to keep the html_part
with  tags, and it doesn't seem to find the css file.

Reply to this email directly or view it on GitHub:
https://github.com/ndbroadbent/actionmailer_inline_css/issues/11

Could it be related to the fact that I'm using SASS with Rails asset pipeline?
Is the asset pipeline supported?

OK, tried changing the parts' bodies instead of clearing them.
Now it renders, but no CSS (does not pick up the file from local) .
Also, the host does not come from headers when rendering it in a preview context. I think host should be an option

Oh yes, I think is definitely due to the asset pipeline. This line in particular - It looks for the CSS file in the 'public' directory.
The asset pipeline shouldn't be too hard to support. For development, it should be pretty easy to find the CSS via Sprockets, and compile the SASS files. In production, the CSS would be precompiled, so a bit easier. It's going to take a bit of work, though...

No worries, I already implemented it (thanks to premailer-rails3 source), now struggling with host url substitution.
 becomes  :(
Once I fix that I'll submit a pull request

Right, so I've sent a pull requests with assets support and improved parts handling.
However, I'm still struggling to make URL absolutization work


Print Version in DISCON Header

Minor update: print version in DISCON header.  Uses ROSCO_toolbox version in setup.py. Not necessarily same as ROSCO controller, but we can sync the versions going forward and this is a step towards helping us debug.
Note: this is based on #29 and includes those changes

This is being accounted for in #31. Closing.


[TEST] basic tests for json input

@mdamour1976 please review

Build Commands:
Build:

ant -f engine/build.xml clean-all resolve jacoco jacoco-integration checkstyle

Finally:
rm -r ~/.ivy2/local || echo "no publish local to remove"
Changed files:
engine/test-src/org/pentaho/di/trans/steps/jsoninput/JsonInputTest.java
Checkstyle violations:
No errors found
License header violations:
No violations found
Unit test coverage change:
org.pentaho.di.trans.steps.jsoninput.JsonReader:

branchPercentCoverage :25.00%
linePercentCoverage :50.56%
methodPercentCoverage :85.71%
complexityPercentCoverage :46.43%
instructionPercentCoverage :44.31%

org.pentaho.di.trans.steps.uniquerows.UniqueRowsMeta:

instructionPercentCoverageChange :-0.17%

org.pentaho.di.trans.steps.jsoninput.JsonInputData:

linePercentCoverage :100.00%
methodPercentCoverage :100.00%
complexityPercentCoverage :100.00%
instructionPercentCoverage :100.00%

org.pentaho.di.trans.steps.sortedmerge.SortedMergeMeta:

instructionPercentCoverageChange : +0.18%

org.pentaho.di.trans.steps.jsoninput.JsonInputMeta:

branchPercentCoverage :14.42%
linePercentCoverage :19.43%
methodPercentCoverage :33.78%
complexityPercentCoverage :20.63%
instructionPercentCoverage :12.63%

org.pentaho.di.trans.steps.jsoninput.JsonInputField:

linePercentCoverage :31.82%
methodPercentCoverage :41.67%
complexityPercentCoverage :31.25%
instructionPercentCoverage :28.82%

org.pentaho.di.trans.steps.jsoninput.JsonInput:

branchPercentCoverage :30.69%
linePercentCoverage :47.35%
methodPercentCoverage :64.29%
complexityPercentCoverage :18.10%
instructionPercentCoverage :39.32%

org.pentaho.di.core.auth.core.impl.DefaultAuthenticationConsumerFactory:

branchPercentCoverageChange : +7.14%
complexityPercentCoverageChange : +7.69%

org.pentaho.di.trans.steps.jsoninput.NJSONArray:

linePercentCoverage :60.00%
methodPercentCoverage :71.43%
complexityPercentCoverage :71.43%
instructionPercentCoverage :57.50%


Add update method example to the README

Not being able to directly add/modify ticket attributes then saving has caused some frustration. This is also a bit counter-intuitive when combined with Rails syntax where .update is a direct method call to save modified values without needing to .save. See https://support.zendesk.com/hc/en-us/articles/226316187-Adding-a-comment-to-a-ticket-via-the-API for more examples of confusion.

Thanks for the contribution @Wingedillidan !


WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5a/Branch_1/Conv2d_0a_1x1/BatchNorm/beta/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5a/Branch_1/Conv2d_0a_1x1/BatchNorm/gamma/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5a/Branch_1/Conv2d_0a_1x1/weights/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5a/Branch_1/Conv2d_0b_3x3/BatchNorm/beta/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5a/Branch_1/Conv2d_0b_3x3/BatchNorm/gamma/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5a/Branch_1/Conv2d_0b_3x3/weights/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5a/Branch_1/Conv2d_1a_3x3/BatchNorm/beta/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5a/Branch_1/Conv2d_1a_3x3/BatchNorm/gamma/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5a/Branch_1/Conv2d_1a_3x3/weights/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5b/Branch_0/Conv2d_0a_1x1/BatchNorm/beta/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5b/Branch_0/Conv2d_0a_1x1/BatchNorm/gamma/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5b/Branch_0/Conv2d_0a_1x1/weights/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5b/Branch_1/Conv2d_0a_1x1/BatchNorm/beta/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5b/Branch_1/Conv2d_0a_1x1/BatchNorm/gamma/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5b/Branch_1/Conv2d_0a_1x1/weights/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5b/Branch_1/Conv2d_0b_3x3/BatchNorm/beta/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5b/Branch_1/Conv2d_0b_3x3/BatchNorm/gamma/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5b/Branch_1/Conv2d_0b_3x3/weights/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5b/Branch_2/Conv2d_0a_1x1/BatchNorm/beta/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5b/Branch_2/Conv2d_0a_1x1/BatchNorm/gamma/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5b/Branch_2/Conv2d_0a_1x1/weights/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5b/Branch_2/Conv2d_0b_3x3/BatchNorm/beta/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5b/Branch_2/Conv2d_0b_3x3/BatchNorm/gamma/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5b/Branch_2/Conv2d_0b_3x3/weights/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5b/Branch_2/Conv2d_0c_3x3/BatchNorm/beta/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5b/Branch_2/Conv2d_0c_3x3/BatchNorm/gamma/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5b/Branch_2/Conv2d_0c_3x3/weights/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5b/Branch_3/Conv2d_0b_1x1/BatchNorm/beta/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5b/Branch_3/Conv2d_0b_1x1/BatchNorm/gamma/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5b/Branch_3/Conv2d_0b_1x1/weights/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5c/Branch_0/Conv2d_0a_1x1/BatchNorm/beta/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5c/Branch_0/Conv2d_0a_1x1/BatchNorm/gamma/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5c/Branch_0/Conv2d_0a_1x1/weights/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5c/Branch_1/Conv2d_0a_1x1/BatchNorm/beta/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5c/Branch_1/Conv2d_0a_1x1/BatchNorm/gamma/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5c/Branch_1/Conv2d_0a_1x1/weights/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5c/Branch_1/Conv2d_0b_3x3/BatchNorm/beta/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5c/Branch_1/Conv2d_0b_3x3/BatchNorm/gamma/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5c/Branch_1/Conv2d_0b_3x3/weights/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5c/Branch_2/Conv2d_0a_1x1/BatchNorm/beta/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5c/Branch_2/Conv2d_0a_1x1/BatchNorm/gamma/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5c/Branch_2/Conv2d_0a_1x1/weights/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5c/Branch_2/Conv2d_0b_3x3/BatchNorm/beta/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5c/Branch_2/Conv2d_0b_3x3/BatchNorm/gamma/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5c/Branch_2/Conv2d_0b_3x3/weights/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5c/Branch_2/Conv2d_0c_3x3/BatchNorm/beta/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5c/Branch_2/Conv2d_0c_3x3/BatchNorm/gamma/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5c/Branch_2/Conv2d_0c_3x3/weights/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5c/Branch_3/Conv2d_0b_1x1/BatchNorm/beta/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5c/Branch_3/Conv2d_0b_1x1/BatchNorm/gamma/Momentum] is not available in checkpoint
WARNING:root:Variable [SecondStageFeatureExtractor/InceptionV2/Mixed_5c/Branch_3/Conv2d_0b_1x1/weights/Momentum] is not available in checkpoint
WARNING:root:Variable [global_step] is not available in checkpoint
WARNING:tensorflow:From C:\Users\HP\AppData\Roaming\Python\Python36\site-packages\tensorflow\contrib\slim\python\slim\learning.py:737: Supervisor.init (from tensorflow.python.training.supervisor) is deprecated and will be removed in a future version.
Instructions for updating:
Please switch to tf.train.MonitoredTrainingSession
WARNING:tensorflow:From C:\Users\HP\AppData\Roaming\Python\Python36\site-packages\tensorflow\contrib\slim\python\slim\learning.py:737: Supervisor.init (from tensorflow.python.training.supervisor) is deprecated and will be removed in a future version.
Instructions for updating:
Please switch to tf.train.MonitoredTrainingSession
2019-06-09 13:28:28.866255: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
WARNING:tensorflow:From C:\Users\HP\AppData\Roaming\Python\Python36\site-packages\tensorflow\python\training\saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
WARNING:tensorflow:From C:\Users\HP\AppData\Roaming\Python\Python36\site-packages\tensorflow\python\training\saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
INFO:tensorflow:Restoring parameters from D:/Lesson8/models/research/object_detection/faster_rcnn_inception_v2_coco_2018_01_28/model.ckpt
INFO:tensorflow:Restoring parameters from D:/Lesson8/models/research/object_detection/faster_rcnn_inception_v2_coco_2018_01_28/model.ckpt
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Starting Session.
INFO:tensorflow:Starting Session.
INFO:tensorflow:Saving checkpoint to path training/model.ckpt
INFO:tensorflow:Saving checkpoint to path training/model.ckpt
INFO:tensorflow:Starting Queues.
INFO:tensorflow:Starting Queues.
INFO:tensorflow:global_step/sec: 0
INFO:tensorflow:global_step/sec: 0
2019-06-09 13:28:48.008457: I tensorflow/core/kernels/data/shuffle_dataset_op.cc:101] Filling up shuffle buffer (this may take a while): 543 of 2048
2019-06-09 13:28:58.047575: I tensorflow/core/kernels/data/shuffle_dataset_op.cc:101] Filling up shuffle buffer (this may take a while): 750 of 2048
2019-06-09 13:29:08.472681: I tensorflow/core/kernels/data/shuffle_dataset_op.cc:101] Filling up shuffle buffer (this may take a while): 1055 of 2048
2019-06-09 13:29:18.090619: I tensorflow/core/kernels/data/shuffle_dataset_op.cc:101] Filling up shuffle buffer (this may take a while): 1229 of 2048
2019-06-09 13:29:28.404542: I tensorflow/core/kernels/data/shuffle_dataset_op.cc:101] Filling up shuffle buffer (this may take a while): 1427 of 2048
2019-06-09 13:29:40.714413: I tensorflow/core/kernels/data/shuffle_dataset_op.cc:101] Filling up shuffle buffer (this may take a while): 1575 of 2048
2019-06-09 13:29:47.415514: I tensorflow/core/kernels/data/shuffle_dataset_op.cc:101] Filling up shuffle buffer (this may take a while): 1623 of 2048
2019-06-09 13:29:53.573397: I tensorflow/core/kernels/data/shuffle_dataset_op.cc:140] Shuffle buffer filled.
2INFO:tensorflow:Recording summary at step 0.`


Train model using domain data

Hi, Thank you for amazing framework. I'm planning to use this framework for question answer system.
Can you please help me how can I train model with my own domain data? I know that nboost doesn't support training, but can you help me how to train outside nboost and configure to nboost later.
Thank you.

Do you mean train a reranking model? Or a model to extract answer spans from passages?
In the former case I use the train triples from say MS Marco which are of format:
<query> <relevant passage example> <not relevant passage example> turn that into data like:
<query> <relevant passage example> label = 1
<query> <irrelevant passage example> label = 0

and train a binary classification model. Here is sample code in the format of the original BERT release (which is super old dont use this)
class MSMarcoProcessor(DataProcessor):

  def _create_example(self, query, doc, label, set_type, i):
    guid = "%s-%s" % (set_type, i)
    text_a = tokenization.convert_to_unicode(query).lower()
    text_b = tokenization.convert_to_unicode(doc).lower()
    if set_type == "test":
      label = "0"
    else:
      label = tokenization.convert_to_unicode(label).lower()
    return InputExample(guid=guid, text_a=text_a, text_b=text_b, label=label)

  def get_train_examples(self, data_dir):
    """See base class."""
    print('Converting to Train to tfrecord...')

    train_dataset_path = os.path.join(data_dir, 'triples.train.small.tsv')

    print('Counting number of examples...')
    num_lines = 100000 # sum(1 for line in open(train_dataset_path, 'r'))
    print('{} examples found.'.format(num_lines))
    examples = []

    with open(train_dataset_path, 'r') as f:
      for i, line in enumerate(f):
        if i > 2000000:
          break
        query, positive_doc, negative_doc = line.rstrip().split('\t')
        examples.append(self._create_example(query, positive_doc, str(1), 'train', i))
        examples.append(self._create_example(query, negative_doc, str(0), 'train', i + 0.5))
    return examples

  def get_dev_examples(self, data_dir):
    """See base class."""
    return []

  def get_test_examples(self, data_dir):
    """See base class."""
    return self.get_dev_examples(data_dir)

  def get_labels(self):
    """See base class."""
    return ["0", "1"]

  def _create_examples(self, lines, set_type):
    """Creates examples for the training and dev sets."""
    examples = []
    for (i, line) in enumerate(lines):
      if i == 0:
        continue
      guid = "%s-%s" % (set_type, i)
      text_a = tokenization.convert_to_unicode(line[3]).lower()
      text_b = tokenization.convert_to_unicode(line[4]).lower()
      if set_type == "test":
        guid = line[0]
        label = "0"
      else:
        label = tokenization.convert_to_unicode(line[0]).lower()
      examples.append(
        InputExample(guid=guid, text_a=text_a, text_b=text_b, label=label))
    return examples

@pertschuk retraining of passage re-ranking model.
Ok i'll try to train binary classification model.
If i face any problem i'll post here.

@Sharathmk99 I would like to know how does the results look for the question answering system after training the model? please post your findings.

@pidugusundeep we have trained binary classification model using hugging face transformers using sample data. It works fine. But our goal is to use transfer learning of bert-tiny model and add our own domain data. I'll keep you posted. Thank you

@pertschuk We have trained a binary classification model using sample data as below.
We have used BERT tokenizer to convert the input examples to features to feed the data for training.
Created one more layer for tuning. Saved the checkpoints after training.This is implemented in Tensorflow.
Could you please guide us on how to add those checkpoints to implement in nboost framework?
Thank you!

@jishajoseph what version of tensor flow?

@pertschuk The tensorflow version is 1.15.0

@jishajoseph try running nboost --model_dir <path_to_your_model> --class TfBertRerankModelPlugin
class Tfv1BertRerankModelPlugin for tensorflow 1.x versions.. the model path should be a directory containing a .ckpt file, bert_config.json, and vocab.txt

Sure..I will try doing the above and I will let you know.
Thank you.

@pertschuk As nboost does not have class argument to be passed, it throws error when I try the above.
nboost: error: unrecognized arguments: --class TfBertRerankModelPlugin class Tfv1BertRerankModelPlugin

@jishajoseph sorry flag above should be --model . Running nboost --help shows all available flags

@pertschuk there is no such plugin present - TfBertRerankModelPlugin

@pommedeterresautee @Sharathmk99
I just pushed a clean and up-to-date example how to train BERT and other transformer model on MS Marco:
https://github.com/UKPLab/sentence-transformers/blob/master/examples/training/ms_marco/train_cross-encoder.py
The result is a HuggingFace Transformer model that could be used with nboost. The models I trained so far outperform the nboost models with comparable model size & run-time (they will soon be added to Hugginface models repository).

https://github.com/UKPLab/sentence-transformers/blob/master/examples/training/ms_marco/train_cross-encoder.py
the code here uses num_label=1, which is not compatible with nboost

You can just set it to num_labels=2 and use it without further changes.


Gradle PactVerify: Provider bug of converting null to string

Hi,
I have the following scenario:
The Actual JSON looks like:
"complete": {
"certificateUri": "http://storage.stg.livemarket.com.au/carfacts_certs/150513/abcrdu05.pdf",
"issues": {
"idNotFound": { }
},
"nevdis": {
"body": null,
"colour": null,
"engine": null,
My PACT file looks like this:
"complete" : {
"certificateUri" : "http://storage.stg.livemarket.com.au/carfacts_certs/150513/abcrdu05.pdf",
"issues" : {
"idNotFound" : { }
},
"nevdis" : {
"body" : null,
"colour" : null,
"engine" : null,
But when I run gradle pactVerify, my test fails due to:
$.body.complete.nevdis.body -> Expected 'null' but received null
Diff:
@9
"nevdis": {

       "body": "null",

       "body": null,
      "colour": null,

The PACT file has "body" : null, but not sure why the Provider test output suggest that it is generated as "body" : "null". I am wondering is it because above "body" : null is an empty JSON structure like below?
"issues" : {
"idNotFound" : { }
},
"nevdis" : {
"body" : null,
"colour" : null,
"engine" : null,
Please advice.
Thank you very much.
Cheers,
Chuan

This issue is now fixed in version 2.2.3