Circuit/cable trace broken when using single-termination circuits

Environment

Python version: 3.7.3
Nautobot version: 1.1.5

Steps to Reproduce

Create Device with at least one Interface
Create Circuit with a single Circuit Termination -- e.g. an Internet transit or an IX
Connect Interface to Circuit Termination with a Cable
Run the trace API function on the Interface

Expected Behavior
I would get a one-item tuple back in the API response, containing the Device's Interface, followed by the Cable and finally the Circuit Termination of the Circuit.

Observed Behavior
I received an error, see below:
{"error": "UUID('1739cb71-2173-5bff-91a6-adb03ce073c0')", "exception": "KeyError", "nautobot_version": "1.1.5", "python_version": "3.7.3"}

This breaks all Internet, Peering and IX circuits migrated from our NetBox. Since there's no equivalent of a (network) Provider in Nautobot yet, we're unable to model these circuits in a way that can be processed by our automation.
Additionally, in case of single-termination circuits, the connected_endpoint parameter just shows as none in the API response -- in NetBox, this was (correctly) shown as circuits.circuittermination.

I can take a look at this, I did a little work with the changes for CircuitTermination.

@ggidofalvy-tc I was unable to replicate the first part of your issue.  When trying the same steps you outlined, I got the following:
[
    [
        {
            "id": "e0173340-2fd3-45e5-820f-aa7f60c925fe",
            "url": "http://<IP_ADDRESS>/api/dcim/interfaces/e0173340-2fd3-45e5-820f-aa7f60c925fe/",
            "device": {
                "id": "382f62fc-eb55-4a3b-967a-f1c3302182af",
                "url": "http://<IP_ADDRESS>/api/dcim/devices/382f62fc-eb55-4a3b-967a-f1c3302182af/",
                "name": "sw01",
                "display": "sw01"
            },
            "name": "FastEthernet1/1",
            "cable": "b7f79b9a-fdf2-44c8-af3f-24fa23577ff7",
            "display": "FastEthernet1/1"
        },
        {
            "id": "b7f79b9a-fdf2-44c8-af3f-24fa23577ff7",
            "url": "http://<IP_ADDRESS>/api/dcim/cables/b7f79b9a-fdf2-44c8-af3f-24fa23577ff7/",
            "type": "",
            "status": {
                "value": "connected",
                "label": "Connected"
            },
            "label": "",
            "color": "",
            "length": null,
            "length_unit": ""
        },
        {
            "id": "c49ad841-b9a2-4d09-9d23-d11032f461c0",
            "url": "http://<IP_ADDRESS>/api/circuits/circuit-terminations/c49ad841-b9a2-4d09-9d23-d11032f461c0/",
            "circuit": {
                "id": "0e43a2fb-bf3c-4c44-aaba-a0828c53367f",
                "url": "http://<IP_ADDRESS>/api/circuits/circuits/0e43a2fb-bf3c-4c44-aaba-a0828c53367f/",
                "cid": "10001",
                "display": "10001"
            },
            "term_side": "A",
            "cable": "b7f79b9a-fdf2-44c8-af3f-24fa23577ff7",
            "display": "Side A"
        }
    ]
]

I also do not see the connected_endpoint parameter.

After upgrading to 1.2.2, I no longer see this issue happening and the connected_endpoint parameter is properly populated yet again.


The remote procedure call failed and did not execute

Have you tried trouble shooting?
Trouble shooting doc
Agent Version and Platform
Version of your agent? 6.3.9600
OS of the machine running the agent? Windows
VSTS Type and Version
VisualStudio.com or On-Prem TFS? Visual Studio
If On-Prem TFS, which release? 2015 RTM/QU1/QU2/...
If VisualStudio.com, what is your account name? http://dotnet.visualstudio.com
What's not working?
Please include error messages and screenshots.
2018-07-11T22:24:55.7748096Z The remote procedure call failed and did not execute.
2018-07-11T22:24:55.7748585Z
2018-07-11T22:24:55.7806295Z System.SystemException: The remote procedure call failed and did not execute.
2018-07-11T22:24:55.7806488Z
2018-07-11T22:24:55.7806772Z    at System.Security.Principal.Win32.LsaOpenPolicy(String systemName, PolicyRights rights)
2018-07-11T22:24:55.7807062Z    at System.Security.Principal.SecurityIdentifier.TranslateToNTAccounts(IdentityReferenceCollection sourceSids, Boolean& someFailed)
2018-07-11T22:24:55.7807390Z    at System.Security.Principal.SecurityIdentifier.Translate(IdentityReferenceCollection sourceSids, Type targetType, Boolean forceSuccess)
2018-07-11T22:24:55.7807663Z    at System.Security.Principal.SecurityIdentifier.Translate(Type targetType)
2018-07-11T22:24:55.7807939Z    at System.Security.Principal.WindowsIdentity.GetName()
2018-07-11T22:24:55.7808167Z    at System.Security.Principal.WindowsIdentity.get_Name()
2018-07-11T22:24:55.7808418Z    at System.Management.Automation.InformationRecord..ctor(Object messageData, String source)
2018-07-11T22:24:55.7808674Z    at System.Management.Automation.Cmdlet.WriteInformation(Object messageData, String[] tags)
2018-07-11T22:24:55.7808937Z    at Microsoft.PowerShell.Commands.WriteHostCommand.ProcessRecord()
2018-07-11T22:24:55.7809160Z    at System.Management.Automation.CommandProcessor.ProcessRecord()

Agent and Worker's Diagnostic Logs
Available upon request.  I didn't want to accidentally expose something, but I'm happy to send the log if needed
Related Repositories
Please ensure you are logging issues to the correct repository in order to get the best support.

Tasks Repository - contains all of the inbox tasks we ship with VSTS/TFS. If you are having issues with tasks in Build/Release jobs (e.g. unreasonable task failure) please log an issue here.
Hosted Agent Image Repository - contains the VM image used in the VSTS Hosted Agent Pool. If you are having Build/Release failures that seems like they are related to software installed on the Hosted Agent (e.g. the DotnetSDK is missing or the AzureSDK is not on the latest version) please log an issue here.

If you are hitting a generic issue about VSTS/TFS, please report it to the Developer Community

@chcosta where is the error coming from? from a build/release job?

Build job - https://dotnet.visualstudio.com/public/_build/results?buildId=5387&view=logs

2018-07-11T22:24:55.7748096Z The remote procedure call failed and did not execute.
2018-07-11T22:24:55.7748585Z
2018-07-11T22:24:55.7806295Z System.SystemException: The remote procedure call failed and did not execute.
2018-07-11T22:24:55.7806488Z
2018-07-11T22:24:55.7806772Z    at System.Security.Principal.Win32.LsaOpenPolicy(String systemName, PolicyRights rights)
2018-07-11T22:24:55.7807062Z    at System.Security.Principal.SecurityIdentifier.TranslateToNTAccounts(IdentityReferenceCollection sourceSids, Boolean& someFailed)
2018-07-11T22:24:55.7807390Z    at System.Security.Principal.SecurityIdentifier.Translate(IdentityReferenceCollection sourceSids, Type targetType, Boolean forceSuccess)
2018-07-11T22:24:55.7807663Z    at System.Security.Principal.SecurityIdentifier.Translate(Type targetType)
2018-07-11T22:24:55.7807939Z    at System.Security.Principal.WindowsIdentity.GetName()
2018-07-11T22:24:55.7808167Z    at System.Security.Principal.WindowsIdentity.get_Name()
2018-07-11T22:24:55.7808418Z    at System.Management.Automation.InformationRecord..ctor(Object messageData, String source)
2018-07-11T22:24:55.7808674Z    at System.Management.Automation.Cmdlet.WriteInformation(Object messageData, String[] tags)
2018-07-11T22:24:55.7808937Z    at Microsoft.PowerShell.Commands.WriteHostCommand.ProcessRecord()
2018-07-11T22:24:55.7809160Z    at System.Management.Automation.CommandProcessor.ProcessRecord()
2018-07-11T22:24:55.7826729Z at Create-PerfTests, E:\A\_work\669\s\build\scripts\build.ps1: line 321
2018-07-11T22:24:55.7827067Z at Build-InsertionItems, E:\A\_work\669\s\build\scripts\build.ps1: line 339
2018-07-11T22:24:55.7827330Z at Build-Artifacts, E:\A\_work\669\s\build\scripts\build.ps1: line 278
2018-07-11T22:24:55.7827842Z at <ScriptBlock>, E:\A\_work\669\s\build\scripts\build.ps1: line 807

E:\A\_work\669\s\build\scripts\build.ps1: line 807
The error is coming from the powershell script in the source code, you need to contact the author of that script.

I think that the powershell error
2018-07-11T22:24:55.7826729Z at Create-PerfTests, E:\A\_work\669\s\build\scripts\build.ps1: line 321
2018-07-11T22:24:55.7827067Z at Build-InsertionItems, E:\A\_work\669\s\build\scripts\build.ps1: line 339
2018-07-11T22:24:55.7827330Z at Build-Artifacts, E:\A\_work\669\s\build\scripts\build.ps1: line 278
2018-07-11T22:24:55.7827842Z at <ScriptBlock>, E:\A\_work\669\s\build\scripts\build.ps1: line 807

and the remote procedure call failure are unrelated.  I took a look at the script when the issue was reported to me, and all that it's doing at that point is calling Write-Host
        $target = Join-Path $configDir "PerfTests"
        Write-Host "PerfTests: $target"

I'm not actually certain where this issue should go.  I don't think PowerShell folks will do anything with it because it seems like a system issue.
https://technet.microsoft.com/en-us/library/2007.07.howitworks.aspx
I was kind of hoping the agent could somehow protect from this occurring but that doesn't really seem feasible either.  It's not clear to me if the "remote procedure call" error would have caused the job to fail or not (given that there was an authoring issue in the build which caused a build failure regardless).  My preference would be for it to not cause a build failure.

@chcosta then you can handle the error in your script, the agent just run your script and fail the task based on return code from PowerShell.exe

It would be a poor experience if everyone had to wrap every instance of powershell output to prevent this error.
As I said, I'm not convinced that it actually causes a powershell task failure.
This isn't a script I own. I'm just reporting the issue for Jared parsons (I don't seem to be able to tag him here.

As long as the RPC error isn't blocking, this isn't a big deal, I just wanted to report it in case it was something you wanted the agent to handle.  I'm Ok with closing this issue and looking at hardware / system changes on our team if these errors become prevalent.


CVE-2020-5245: Remote Code Execution (RCE) vulnerability in dropwizard-template-config

https://nvd.nist.gov/vuln/detail/CVE-2020-5245
https://github.com/dropwizard/dropwizard/security/advisories/GHSA-3mcp-9wr4-cjqf

I am getting this too. @tkrille can you please respond?


failure to build RealmSwift

using
xcode 7.1 beta 3
CocoaPods 0.39
RealmSwift 0.96.1
it fails to compile Realm with
object_store_exceptions.hpp file not found // in object_store_exceptions.cpp

How can i work around this?

@danielpovlsen It will occur on the case-sensitive file system. Fixed at https://github.com/realm/realm-cocoa/pull/2675
The suggested workarounds are:

To use CocoaPods 0.38
Copy your project to external disk formatted by case-insensitive file system. (Using disk image is the easiest way), then execute pods install there.
That fix has been merged to master. So specify branch to install from master.
Setup without CocoaPods. It will occur only CocoaPods install

cheers dude, you're awesome.

Sorry, I confirmed that it cannot resolve the issue to specify master branch.
The most reliable way is using case-insensitive file system temporarily for the setup.

Create case-insensitive formatted disk image with Disk Utility.
Then copy your project to the disk.
Execute pod install there.

If the project would be setup correctly, you can move anywhere the project.

#2675 was included in 0.96.2, released earlier today. Given @kishikawakatsumi's comments above I'd expect this to be fixed with 0.96.2.

Still having issue here, even with 0.96.2. Anyone else?

@nekonari This issue was fixed in 0.96.2. Could you try first removing cache then re-install like following:
rm -rf ~/Library/Caches/CocoaPods
rm -rf Pods
pod install

If you don't use a case-sensitive file system, there is a probability that the another problem will happen.

@kishikawakatsumi I'm linking them manually by downloading and adding the pre-built libraries into the project. If there is some cache that stores previous framework files, please let me know :(

I got it to work. In the build error, it was specifying a cache folder. Removing it solved this.
However, for some reason, I had Embed Frameworks build phase placed below Run Realm Script build phase, which was again creating a build error. I reordered these, and now it's compiling.

I had Embed Frameworks build phase placed below Run Realm Script build phase, which was again creating a build error.

@nekonari Because strip-framework.sh script assumes that the framework is in the app bundle. So the framework should be copied before running strip-framework.sh.

@kishikawakatsumi What puzzles me is why Xcode didn't simply replace the cached framework files with the new version..

@nekonari: That might happen especially if you change the way how you integrate stuff, Xcode reasons on base of what it would expect to see with the current setup, if there are built products left from a previous setup and those files wouldn't be created anymore or in a different way, it might happen, that they get stuck. For that reason, I'd generally recommend cleaning the build folder after making changes to the integration.


Eliminate token object mutation

In some cases, the token object is still mutated (properties being added or modified), see e.g. scanPunctuator and parseBinaryExpression. For improved debugging experience and better readability, this practice should be discouraged (as far as it does not affect the performance).

It is confirmed that the performance did not suffer.


Philips power on color temperature and brightness

Add attributes for power on color temperature and power on brightness configuration.
Attribute numbers are taken from this comment: https://github.com/dresden-elektronik/deconz-rest-plugin/issues/1055#issuecomment-559993258
Tested on Philips LCT003 (ZLLExtendedColorLight). ZHAExtendedColorLight and ZHAExtendedColorLight2 are not tested but they should be the same for what I know.
Apparently, to set the power on color itself one could use the Color (0x0300) cluster, current_x (0x0003) and current_y (0x0004) attributes with manufacturer code 0x100b (4107).

There were some changes in zigpy and we are no longer copying attr dict, but using manufacturer_attributes dict instead. See some other quirk for an example. On the phone and can't provide direct links right now

Ok, will do that. I just copied the approach from the existing code for PhilipsOnOffCluster.

ah, effing philips, could be because https://github.com/zigpy/zha-device-handlers/issues/399#issuecomment-678631540
can you confirm?

Looks like that...
I've tested it now, it doesn't work with manufacturer_attributes.

Thanks for testing.

@Shulyaka So is power on to previous state not possible? It's writable in deCONZ for my hue lights, but doesn't function.

@Shulyaka So is power on to previous state not possible? It's writable in deCONZ for my hue lights, but doesn't function.

is it writtable in ZHA?
Device signature as reported by ZHA
Debug log of you writing this attribute

is it writtable in ZHA?
Device signature as reported by ZHA
Debug log of you writing this attribute

It's writable in deCONZ, can get more info later today.

It's writable in deCONZ, can get more info later today.

As per https://github.com/dresden-elektronik/deconz-rest-plugin/issues/1055#issuecomment-559993258, it should be cluster OnOff 0x0006, attribute PowerOn OnOff 0x4003, value Previous (0xff)

As per https://github.com/dresden-elektronik/deconz-rest-plugin/issues/1055#issuecomment-559993258, it should be cluster OnOff 0x0006, attribute PowerOn OnOff 0x4003, value Previous (0xff)

Correct, but when you write "Previous" it writes the data but does not change the behavior of the light. The light will reset temp/brightness on power loss.

Correct, but when you write "Previous" it writes the data but does not change the behavior of the light. The light will reset temp/brightness on power loss.

you write 255, not Previous

you write 255, not Previous

255 would turn the light on 100% brightness on reset. I'm looking for previous temp/brightness on reset.

255 would turn the light on 100% brightness on reset. I'm looking for previous temp/brightness on reset.

you have to do it for all clusters. And we have to assume you do it correctly. For the light that is on the network and is not powered off.

you have to do it for all clusters. And we have to assume you do it correctly. For the light that is on the network and is not powered off.


How to pass settings to Ferrum::Browser?

Hi! Thanks for this gem!
I am running this inside a Docker container, and by reading the related topic on Ferrum documentation I saw that it needs to be called like this in order for Chrome to start:
Ferrum::Browser.new(browser_options: { 'no-sandbox': nil })

However I see no way of passing these settings to Ferrum::Browser here. Right now I am monkey-patching the code in my project, but I think it deserves mentioning this since other people in a similar situation might be blocked by this as well.

@route Yes, I am having the same problem as @giovannibonetti in this regard. I need to pass settings to Ferrum to run my traffic through a proxy, as specified here. But I see no way to do this without monkey-patching as he said. I also need to login to certain pages, although that might be considered a separate problem.

Totally agree will add support soon

You can pass it like this guys https://github.com/rubycdp/vessel/blob/master/examples/quotes.toscrape.com.rb#L7

@route Thanks! But, I get an error when I try to run that script:
./scraper.rb:8:in `<class:QuotesToScrapeCom>': undefined method `ferrum' for QuotesToScrapeCom:Class (NoMethodError)

My Gemfile looks like this:
source 'https://rubygems.org'
git_source(:github) { |repo| "https://github.com/#{repo}.git" }

gem 'vessel', github: 'rubycdp/vessel', branch: 'master'

Any idea why it's failing?

Did you run bundle update vessel?

@giovannibonetti No, I just ran bundle update. But I just ran bundle update vessel like you said, and it says:
Using vessel 0.1.1 from https://github.com/rubycdp/vessel.git (at master@13f6c5e)
Bundler attempted to update vessel but its version stayed the same
Bundle updated!

And I still get the same error.

Did you restart the server? Tried spring stop?

@giovannibonetti I'm not using Rails or spring so that doesn't do anything. I have no server running, just trying to run a standalone scraping script.

@ylluminarious you have to run you script with bundle exec bla-bla because I believe you are using installed gem instead of gem from master.

@route Oh, yes, you're right. That did the trick. Thanks!

I am not sure I follow this.
How do I pass an option like for example ":process_timeout" (one of the configuration options given on the ferrum page https://github.com/rubycdp/ferrum#customization ?

@tkhobbes you should use driver :ferrum, process_timeout: 200 as it's now possible to use different drivers, driver :mechanize for instance.

OK I am not sure how exactly the syntax would look like. I am trying like this: BookAttrs.run(driver: :ferrum, process_timeout: 300, xvfb: true) { |b| book = b } where BookAttrs is an object of class Vessel::Caro.... the process timeout doesn't seem to be adjusted (I get errors saying "Browser did not produce websocket url within 10 seconds, try to increase :process_timeout")

class QuotesToScrapeCom < Vessel::Cargo
  domain "quotes.toscrape.com"
  start_urls "https://quotes.toscrape.com/tag/humor/"
  driver :ferrum, process_timeout: 200
...

that doesn't seem to be working. I get a WARN: NoMethodError: undefined method driver' for Scraper::BookScraper::BookAttrs:Class`when doing the above... this is in a rails app btw.

@tkhobbes you definitely use gem not from master, but from rubygems


Put version in footer, rearrange items, format.

Built with setting main.CommitHash:

Built without setting main.CommitHash (default):

@dcrdata/explorers How about we try on this look for size?  The primary purpose of this PR was to put the version in the footer, but I ended up tweaking the text and layout too.

Up on https://beta.dcrdata.org/ for testing.

Seems to work just fine

Just found a template exec error on tx and address pages

Yeah, just saw that seems the template line is still in the txinfo or addressinfo scope, changing to {{ template "footer" $.Version }} should fix it for both

I moved stuff around instead, but you're right that we could have used $.


Micro-optimize method.data(gs) in calls.cc

Motivation

Maybe a slight perf win.
Test plan

Existing tests

Looks good to me! I'm still re-familiarizing myself with the codebase, but it doesn't look like any of the references are taken in a context where the GlobalState could be modified 👍

Yep, this is in calls.cc, which only runs during inference, where GlobalState is frozen.


MAINT:stats:Cythonize and remove Fortran statlib code

Reference issue
Part of #18566
What does this implement/fix?
I converted the private FORTRAN code that was living under statslib. Though not so long it was quite tricky to follow all the integer tricks. Then I cythonized them for a reduced performance hit.
The code is very old (from 1980s and added to SciPy 20+years ago) and only works with that particular style. I can sense that this can be so much more performant by regular numpy usage however I have no idea what the code does hence followed the old code closely and tested as much as I can.
Additional information

swilk was returning a for no apparent reason. Now a is modified in place and not returned. Also everything now works in double precision and not the weird real/float mix.
I can guarantee output of gscale within a meshgrid of (100x100). It starts overflowing around (50x50) anyways but tested regardless Also tested m+n = 150 cases with m=1, 2, 10, 20.
For swilk tested many conditions including nan inputs and other cases also passed the tests on my machine with no modification. The decimal digits are preserved  to 12-14 digits.
prho was never used and no tests failed with its absence so directly removed it.

This looks like swilk: https://real-statistics.com/tests-normality-and-symmetry/statistical-tests-normality-symmetry/shapiro-wilk-expanded-test/

I appreciate your trust in my stats knowledge but it is really not up to par to code something in SciPy hence I'll leave it to the stats residents.

I've implemented Shapiro-Wilk in Python, but I don't have complaints with the Fortran that I wanted to address at this time.

@ilayn is Cython vs Pythran a personal preference here, or does Cython have an important advantage here?

I didn't think about Pythran at all. After conversion Python version was slow so I bumped the speed up.
The impact is not much since now both are around microseconds, for gscale 760ns vs 1.75 microseconds and so on. Unless someone runs this on a 10000 for loop negligible difference.

In that case, may I suggest using Pythran? Cython is more approachable than Fortran, but Pythran is even better from that perspective. gh-18328 comes to mind - if swilk had been Python-like instead of Fortran, we probably would have applied the fix in swilk itself rather than adding that special case in shapiro.

I don't think there is significant difference. Cython is also Python code just verbose type definitions at the top of the functions. You can literally copy paste your fix to this function here without any impact. Cython compiles as much as it can with explicit defs and leaves it like regular python code hence I don't think Pythran or Cython matters.
https://github.com/scipy/scipy/pull/18679/files#diff-e395b1c082d3122404921ba93c6b06aa20b6b4e841d8b426f86dbd34bc7c7e4cR328-R330

I don't think there is significant difference.

It depends on the example in question of course, but there can be significant differences, like factor 3-5 faster. Of course that's not guaranteed to always happen, but the difference to Cythons is kinda Pythran's main raison d'être. Pythran also has more higher-level constructs (also allows using more of numpy's API in the pythran code than cython), and seems to get optimize better under compilation.

Oh, I assumed Cython was faster. If Pythran is faster, that's icing on the cake. I was after the maintainability aspect, and I was comfortable even with a speed hit. I've authored PRs with both, and since the early build issues were resolved (I'm on Windows, too), Pythran has been much easier to work with.

I mean if it was that nice i would have switched too. But there's no such free lunch, feel free to copy the body and convert to Pythran, there is no cython command it's all python code all copy pasteable. So I don't know what you mean by maintainability aspect.
For the rest it's all C and Cython code already in stats hence I'm not sure exactly where the resistance coming from.

It's not resistance, really. It's just the only opinion I have here. Feel free to ignore.

No I mean I'm interested too if there is a significant gain of course but I want to know if there is any issues cython is causing. I just don't want to write double code because currently Pyhran is opt-in and not always enabled.

It depends on the example in question of course, but there can be significant differences,

Yes but there is no way to find out when as far as I know (which is very little). Cython has the annotation to tell you where the Python interaction is hence that's what I mean by consistency.

No resistance from my side either, just wanted to clarify.

I just don't want to write double code because currently Pyhran is opt-in and not always enabled.

100% agreed about not writing twice. OTOH, pythran's been opt-in for a long time (since 1.7), and https://github.com/scipy/scipy/issues/13308 was closed recently. I think we could retake a look at making it mandatory, or at a timeline for that (e.g. start warning if pythran isn't present and see if someone complains). CC @rgommers

Ah finally CI checks out, this one is good to go from my side.

I tried comparing the two version of gscale. When I run the following code with the new version:
from scipy.stats._AB_SW import gscale as gscale2
astarts_pr = []
a1s_pr = []
for i in range(10):
    for j in range(10):
        print(i, j)
        astart_pr, a1_pr, _ = gscale2(i, j)
        astarts_pr.append(astart_pr)
        a1s_pr.append(a1_pr)  # comment out this line

The program just stops after about 10 iterations (both on command line and in IDE. In IDE, it says Process finished with exit code -1073740940 (0xC0000374)). If I comment out the last line, it runs to completion.
Note that I'm still doing an in-place build with setup.py, in case that could have something to do with it.

That's probably, I forgot to guard against i, j being 0. If you start with range(1,10) it would probably go through

Weird... yes, it does - but why? gscale2(0, 0) works, and there is not a problem unless I append the a1 result to the list.
In any case, that case would never be called by stats.ansari, and otherwise the output of the new gscale does seems to be identical to that of the original.

import numpy as np
from scipy.stats._statlib import gscale
from scipy.stats._AB_SW import gscale as gscale2
from time import perf_counter

time_main = 0
time_pr = 0
for i in range(1, 200):
    for j in range(1, 200):
        print(i, j)

        tic = perf_counter()
        astart_main, a1_main, _ = gscale(i, j)
        toc = perf_counter()
        time_main += toc - tic

        tic = perf_counter()
        astart_pr, a1_pr, _ = gscale2(i, j)
        toc = perf_counter()
        time_pr += toc - tic

        np.testing.assert_equal(a1_pr, a1_main)
        assert astart_pr==astart_main

print(time_pr/time_main)

It's very slightly slower on my machine, which is fine IMO.
I can confirm "I can guarantee output of gscale within a meshgrid of (100x100). It starts overflowing around (50x50) anyways but tested regardless". I actually went up to 200.
I could suggest that if the dtype were changed to 64-bit float, we could go up higher without overflow, but there is probably no reason to, and whether there is or not is a separate issue. So I have nothing against the gscale part of this PR. On to swilk.

For swilk, the agreement is not as close. It's may just be roundoff error being amplified, but we try to be careful about these things in stats. Consider an extreme case:
import numpy as np
from scipy.stats._AB_SW import swilk
y = np.sort([-180089.49350026142, -180089.49185713136, -180089.48736901287, -180089.48231540626,  -180089.46989864993])
n = len(y)
a = np.zeros(n//2, dtype=np.float64)
init = 0
_, w, pw, ifault = swilk(y.copy(), a.copy(), init)
w  # now 0.8985956472033496, was 0.2524259686470032
pw  # now 0.40217241177567253, was 3.542755065005787e-12

It has something to do with the variance being small relative to the location of the data, but any ideas about why the new code would produce a different result?
Here's the code I was using to stress-test this.

import numpy as np
from scipy import stats
from scipy.stats._statlib import swilk
from scipy.stats._AB_SW import swilk as swilk2
from time import perf_counter
rng = np.random.default_rng(9245923596356935)
time_main = 0
time_pr = 0
for n in range(4, 5000):
    for j in range(1, 3):
        print(n, j)

        a = np.zeros(n//2, dtype=np.float64)
        init = 0
        x = rng.normal(loc=stats.norm(scale=1e5).rvs(random_state=rng),
                       scale=stats.loguniform(1e-3, 1e6).rvs(random_state=rng),
                       size=n)
        y = np.sort(x)

        tic = perf_counter()
        _, w, pw, ifault = swilk(y.copy(), a.copy(), init)
        toc = perf_counter()
        time_main += toc - tic

        tic = perf_counter()
        w2, pw2, ifault2 = swilk2(y, a, init)
        toc = perf_counter()
        time_pr += toc - tic

        np.testing.assert_allclose(w2, w, rtol=1e-3)
        np.testing.assert_allclose(pw2, pw, rtol=1e-3)
        assert np.all(ifault2 == ifault)

print(time_pr/time_main)

For gscale;
I think I know why, it's OK with 0 inputs. I would say (0, 1) is the offending input. At the end of gscale there is a condition that says
        # reverse array
        if other > test:
            for ind in range(LL//2):
                a1v[LL-1-ind], a1v[ind] = a1v[ind], a1v[LL-1-ind

when encountered with (0, 1) probably it is trying to revert it but because boundscheck is False secretly segfaulting for reaching out to an element outside memoryview. Then probably returning garbage that can't be converted to a python object. Then things I don't understand follow and resulting with a low-level problem. Seems like a trivial fix
For swilk,
that's way more than I ever encountered and clearly unacceptable so thanks for that example. Probably it goes into some other branch. I'll check what is going on with the Fortran comparison.

For swilk, the differences are coming from working with float64. This algorithm doesn't give matching results with high precision and I don't know which one is the correct. In fact I can see how they slowly diverge with the additional precision. Is there any way we can test this behavior with some other tool?

Indeed converting to float32 makes the results match ends with
4998 1
4998 2
4999 1
4999 2
2.542812896175186

Looking at the values again, did you take out the median from the test cases like shapiro does or it doesn't have any significance?

It shouldn't affect the calculation, and it doesn't seem to (at least when floating point doesn't get in the way).
import numpy as np
from scipy import stats
stats.shapiro([1, 2, 3, 4])
stats.shapiro([10, 20, 30, 40])
stats.shapiro([15, 25, 35, 45])
# should all be the same

We added the line to subtract the median in https://github.com/scipy/scipy/pull/15777 because the Fortran version had issues when the variance is relatively small compared to the data.
It turns out that this new version was more accurate than the Fortran in this case. (I suppose we should have known that the Fortran version was wrong the probability of getting a p-value of 3.542755065005787e-12 in one of the iterations above is .000000000354%.) After subtracting the median, we get approximately the same value with the new code and Fortran code, and this is approximately the same as the new code gets without subtracting the median.
Still, if you run the stress test above (corrected with subtraction of the median) you will see some discrepancies.

I found the offending part in the gscale tests which was not immediately returning after small sample size. Now it is passing without any strange segfaults.
For the swilk I added the modifications you suggested. Also added some DOIs for quick access to things as many as I could find.
Currently, on my machine gscale_pr/gscale_sp is 1.33 and swilk_pr/swilk_sp is 2.13 (some slowdown is because of double precision anyways) which is I think acceptable.

Very nice. So apparently p value can return almost 0 and with some noise it can be marked as -eps. Do you have any suggestions how to handle that ?
The remaining failures (except lint is not related).

So apparently p value can return almost 0 and with some noise it can be marked as -eps

If that's for $n=3$, we can generate an approximation that is valid where the output is nearly zero. Otherwise, the car we can do is np.clip to keep it non-negative.

Yes it is indeed. https://github.com/scipy/scipy/actions/runs/5369718336/jobs/9741361808?pr=18679#step:3:3190
The violation is very little -1.9526481126463935e-16 and only on Linux 32 bit runs which are lately turning out to be more problematic than we'd wish for it.

Actually the problem is that the statistic is falling below 3/4. Maybe clip the statistic to keep it above 3/4 (theoretical minimum, apparently) before passing it into the formula for the p-value.

OK clamped the lower bound to 3/4. Let's see how it goes.

After a flaky TestDGamma failure this looks like passing now

Thanks @mdhaber for the review anyways, since this is for 1.12, there is no rush for it, let's wait until someone else has bandwidth to check.

Ah dammit, the rename broke the review comments. But thanks Ralf for the review. I think I addressed them all. Have a look when you have the time please.

OK comment added. It's ready to merge at your convenience.

In it went. Thanks @ilayn and @mdhaber & other reviewers.


Adding notebook for supply trend on fixture date

@LamprosArgy Hello Lampro! Overall the notebook seems very good. The only thing that I believe we should have is more comments regarding the code scope for the different blocks. For example, do you think that in line where the supply trend column is calculated by calling the htl_query we might comment before 'Now we will call the historical tonnage api by giving the parameters from the voyages in order to create the supply trend column'. Or below when we are splitting the 3 days to have a comment before like 'The result from the api is a list so in order to take the different days before we should split the list accordingly'. If you think that these comments are not really of great value please say so! Thank you!


(libreoffice-help) package "LibreOffice Help Pack 5.2.0" outdated by far, seems unmaintained for long time

Checklist

[X] I have checked the moderation queue that no new version has been submitted (only visible when logged in), or the submitted package version has failed the automated checks.

Please navigate to the following link to view the moderation queue.

[X] I have verified that the new version is a stable release.
[X] I have looked to see if there are any Bug reports that prevents a new version being submitted and that no existing Outdated Reports have been created.
[X] I have verified the new version has a Windows binary (EXE or MSI installer, Zip or other archive, etc).

New Software Version
7.5.6
Download location
https://download.documentfoundation.org/libreoffice/stable/7.5.6/win/x86_64/
Package Page
https://github.com/chocolatey-community/chocolatey-packages/tree/master/manual/libreoffice-help

please, have a look here…
https://github.com/orgs/chocolatey-community/discussions/2311#discussioncomment-7010236

This is a manually updated package. See the README and this issue that it points to.

I followed both links.
README explains the language specific settings, and it says that this package is unmaintained (yes, it is, indeed!) and refers to this issue that it points to. There, I (personally don't get why this package can't turned into AU.
But, what is obvious: the package, being a manual update package, is unmaintained for a long time. It is outdated by far. I seriously wonder what is the problem with this package? I just don't get it. I don't want to troll. I am really interested why this package cannot be turned into AU, is hard to maintained, is currently unmaintained. (I personally would like to use it.)

My suggestion would therefore be to submit a PR to have the package automatically updated.

I'm not familiar with that, don't know how to provide all necessary details for an appropriate PR. Can you @pauby  do this perhaps?

@zWhdmB5T you're frustrated that the package is not being maintained and that it should be automatic.
This repository is a community maintained repository. If you need a package to be something that it isn't, it's expected you get involved and do the work.
The best way for this package to be updated is for somebody who has a need for it to be updated and submit a pull request.

@pauby well yes, you're right, I am frustrated by this situation. But I want want to make clear that I do not demand anything but just kindly ask for. I appreciate and I value OpenSourceSoftware and related projects. I have donated quite some money to several authors and projects. But I cannot directly help here, because I am not into programming and coding at all. I am just a humble user. Sometimes writing a bug report. And sometime posting a request like in this case.  That's all I have GH for. Sorry, if my post was somewhat offensive. I didn't want to be rude.

Your post wasn't offensive or rude. I was just trying to make it clear how this repository runs and what the best and quickest way for the package to be updated, is.
There are ~400 packages in here and a small team of active maintainers.

OK, understood now. Then, thank you.
By the way, I feel always sad that I am not into programming or coding. There have been many occasions where it would have been helpful. I am very sorry.


Missing Fortified Functions vulnerability report

We have received a report of potential vulnerabilities in the BreachDetector solution as listed below:
Risk Value - M3
Issue Description - Missing Fortified Functions
Details - This finding is for Android. The shared object does not have any fortified functions. Fortified functions provide buffer overflow checks against glibc's commons insecure functions like strcpy, gets etc. The affected libraries are:
armeabi-v7a/libtool-checker.so
armeabi-v7a/libanti.so
arm64-v8a/libtool-checker.so
arm64-v8a/libanti.so
Recommendation - Use the compiler option -D_FORTIFY_SOURCE=2 to fortify functions

@nmilcoff is this project being maintained? If so, are fortified functions already being used or is that work planned?

Hi @jkavanaghdeluxe , unfortunately I have moved on to a different stack (outside of dotnet) so this project is unfortunately unmaintained at this point

@nmilcoff Thanks for the response.


Consistently getting $actualApply is not a function when running Cypress tests

My project recently upgraded from React 18.2.0 to 18.3.1 and your package is now a part of my project as a dependency. If I use npm ls call-bind-apply-helpers I get
@{redacted monorepo root}
└─┬ @{redacted project root}
  └─┬<EMAIL_ADDRESS>    └─┬<EMAIL_ADDRESS>      └─┬<EMAIL_ADDRESS>        └──<EMAIL_ADDRESS>
When I try to run my Cypress tests, I am getting the error $actualApply is not a function, but I cannot figure out why this is the case.

hmm, which browser is this occurring in? can you provide the full stack trace?

Also, can you perhaps try to figure out what $actualApply is at that time, if not a function?

It's actually not running in a browser. It is from the terminal running Cypress tests for Storybook.
I am not even sure why this is being called at all. I had never seen this package before yesterday and now it is causing my tests to fail with the error $actualApply is not a function.
I can do some more digging and try to determine why it is failing and what $actualApply is when the test fails.

I recently refactored call-bind to extract some pieces of it into this newly created package, which is why this package won't be something you've seen before :-)
I tested it pretty extensively but I may indeed have missed something.

I modified index.js to be
module.exports = function callBindBasic(args) {
	if (args.length < 1 || typeof args[0] !== 'function') {
		throw new $TypeError('a function is required');
	}

        if (typeof $actualApply !== 'function') throw new $TypeError(JSON.stringify($actualApply));

	return $actualApply(bind, $call, args);
};

and the output in the terminal is {"default":false"}

hmm - the default part is something i can fix, but it's still showing up as false which is strange. can you also console.log(typeof Reflect?.apply)?

actually, i wonder if this might be a misconfigured build tool, that's incorrectly treating .js files inside a different project as ESM.

actually, i wonder if this might be a misconfigured build tool, that's incorrectly treating .js files inside a different project as ESM. what node version are you using? if 22.12, can you try 22.11?

I'm actually on Node 18.18.0

Perhaps you've set up cypress or babel or similar to transpile things inside node_modules?

Perhaps you've set up cypress or babel or similar to transpile things inside node_modules?

I'm looking into this. But, this is something that just popped up yesterday. I understand that you have recently split this off from call-bind and that is why this package is new in the repo. I just don't know what would have changed between then and now

I also don't know, but previously it wasn't using require to get at a bound function for "apply", it was just computing it inline, so the issue might have been masked.

So, it doesn't look like I am transpiling node_modules. So, that is good.
I will do a trace backwards through the packages and find out what is calling the method and why it is failing.

I have traced my way through the code and the best that I can determine is that there is an issue here: https://github.com/ljharb/call-bind-apply-helpers/blob/main/reflectApply.js#L4
This line:
module.exports = typeof Reflect === 'function' && Reflect.apply;
Seems to be the issue. Reflect is an Object - https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Reflect
If I replace the line with module.exports = typeof Reflect === 'object' && Reflect.apply; or even jest module.exports = Reflect.apply; (which of course doesn't have the check), then everything works as expected.
I looked at your commit from the call-bind repo here: https://github.com/ljharb/call-bind/commit/81018fb78902ff5acbc6c09300780e97f0db6a34#diff-16e4f2d9a56d404fdb183325a6934eb66492d6ca41e6db70b68cd7c24dc7c315L3
and I see that you copied the code over, but I don't see how this ever worked.
Could you please provide any information that you may have on what could be the discrepancy here? Thanks!

aha, you’re right. that’s a bug in the code, but it should just result in it being false, which should still allow actualApply to work.

aha, you’re right. that’s a bug in the code, but it should just result in it being false, which should still allow actualApply to work.

It could be something on my end, but if reflectApply returns false, then this is what $reflectApply is set to in actualApply.js :

so, module.exports = $reflectApply || bind.call($call, $apply); will return {default: true}

so, I certainly will fix reflectApply, but you'll still see $reflectApply as { default: Reflect.apply } which means it will still be broken, because it should be seeing it as literally Reflect.apply.

so, I certainly will fix reflectApply, but you'll still see $reflectApply as { default: Reflect.apply } which means it will still be broken, because it should be seeing it as literally Reflect.apply.

Weirdly, changing the check from function to object works on my end. Here is the logged outputs to the console of $reflectApply in actualApply.js
As function:

As object:

Thank you for your help with this!

Can you try changing it to module.exports = x where x is any primitive value? (perhaps, try all of the types of primitives?) i wonder if perhaps you've got something interfering when it's a primitive, that doesn't interfere when it's an object/function.

Can you try changing it to module.exports = x where x is any primitive value? (perhaps, try all of the types of primitives?) i wonder if perhaps you've got something interfering when it's a primitive, that doesn't interfere when it's an object/function.

Very odd! It seems to only do {default: false} when the returned value is false.
I tried module.exports = 1; and module.exports = true;. Both returned the hardcoded value.
Then, I tried module.exports = false; and it returned {default: false}.
I have no idea why this is, but, at the very least, your change appears to have fixed this issue.
Again, I appreciate your assistance on resolving this so quickly!

Very strange. The only possibilities I can think of are either 1) your setup is doing something weird, or 2) there's a bug in node, but one I can't reproduce.
I'll close this, but my strong suspicion is that it's something about your cypress setup.


Code gen issues when using multiple handlers for the same message type

Getting the following error when attempting to have 2 handlers for the same message type that have parameter names with the same name. There is also an issue when both handler methods return the same type. I tried figuring out how to create a test for this, but was unsuccessful. Pointers?
System.InvalidOperationException: 'Compilation failures!
CS0100: The parameter name 'logger' is a duplicate
CS0229: Ambiguity between 'ILogger' and 'ILogger'
CS0229: Ambiguity between 'ILogger' and 'ILogger'
Code:
// <auto-generated/>
#pragma warning disable
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging;

namespace Internal.Generated.WolverineHandlers
{
    // START: UpdateCustomerAddressHandler822317193
    public class UpdateCustomerAddressHandler822317193 : Wolverine.Runtime.Handlers.MessageHandler
    {
        private readonly Microsoft.EntityFrameworkCore.DbContextOptions<ConcordServicing.Data.ConcordDbContext> _dbContextOptions;
        private readonly Microsoft.Extensions.Logging.ILogger<ConcordServicing.Data.Handlers.CustomerHandler> _logger2;
        private readonly Microsoft.Extensions.Logging.ILogger<ConcordServicing.CSS.Handlers.CustomerHandler> _logger1;

        public UpdateCustomerAddressHandler822317193(Microsoft.EntityFrameworkCore.DbContextOptions<ConcordServicing.Data.ConcordDbContext> dbContextOptions, Microsoft.Extensions.Logging.ILogger<ConcordServicing.Data.Handlers.CustomerHandler> logger, Microsoft.Extensions.Logging.ILogger<ConcordServicing.CSS.Handlers.CustomerHandler> logger)
        {
            _dbContextOptions = dbContextOptions;
            _logger2 = logger;
            _logger1 = logger;
        }

        public override async System.Threading.Tasks.Task HandleAsync(Wolverine.Runtime.MessageContext context, System.Threading.CancellationToken cancellation)
        {
            await using var concordDbContext = new ConcordServicing.Data.ConcordDbContext(_dbContextOptions);
            var updateCustomerAddress = (ConcordServicing.Data.Messages.UpdateCustomerAddress)context.Envelope.Message;
            ConcordServicing.CSS.Handlers.CustomerHandler.Handle(updateCustomerAddress, _logger1);
            var customerAddress = await ConcordServicing.Data.Handlers.CustomerHandler.Handle(updateCustomerAddress, concordDbContext, context, _logger2).ConfigureAwait(false);
            // Outgoing, cascaded message
            await context.EnqueueCascadingAsync(customerAddress).ConfigureAwait(false);

        }

    }

    // END: UpdateCustomerAddressHandler822317193

}

Here are the handlers:

    // transactional means that it will only send the customer updated message if the SQL Server update succeeds
    [Transactional]
    public static async Task<CustomerAddress> Handle(UpdateCustomerAddress command, ConcordDbContext db, IMessageContext context, ILogger<CustomerHandler> logger)
    {
        logger.LogInformation("Updating customer address in SQL {CustomerId}", command.Id);

        // this happens after the CSS action has succeeded
        // CSS is the source of truth
        // but we can still read our writes and act like a normal application

        var customer = await db.Customers.FindAsync(command.Id);
        if (customer == null)
            return CustomerAddress.NotFound;

        customer.Address = command.Address;

        await context.PublishAsync(new CustomerAddressUpdated
        {
            Id = customer.Id,
            Address = customer.Address
        });

        return new CustomerAddress
        {
            Id = customer.Id,
            Address = customer.Address
        };
    }

    public static void Handle(UpdateCustomerAddress command, ILogger<CustomerHandler> logger)
    {
        logger.LogInformation("Calling CSS to update customer {CustomerId}", command.Id);

        // call the CSS action first, if it fails, it will not update SQL Server
        // this can use retry policies to handle transient errors
        // this entire project will eventually just be deleted when we get everything migrated to SQL Server
        // during local dev, we won't register any of the CSS handlers and the app will be just a normal SQL Server app

        Thread.Sleep(1000);

        // TODO: do CSS action to update the address, if it fails, don't continue to the next handler

        if (Random.Shared.Next(100) < 80)
        {
            logger.LogError("Error calling CSS to update customer {CustomerId}", command.Id);
            throw new ApplicationException("Unable to process CSS action.");
        }
    }

Here is a repo for the sample: https://github.com/ejsmith/ConcordServicing

The method parameter 'logger' is being added twice to the constructor in the generated code, with the same name. Renaming logger parameters will probably solve your issue.

I tried that. It did not fix it.


Make gradle plugin defaults consider multiplatform sourcesets

By default, the gradle plugin only considers src/main/java and src/main/kotlin when looking for source directories. This means multiplatform projects must add custom configuration to work.
This PR adds multiplatform directories by using a FileTree with patterns to find all applicable directories

Hej @gustavkarlsson
What's the status of this PR? Are you actively working on it?
If not, I'd like to pick-up multiplatform support for detekt Gradle plugin,
but I don't want to step on your shoes 👌

Hej @gustavkarlsson
What's the status of this PR? Are you actively working on it?
If not, I'd like to pick-up multiplatform support for detekt Gradle plugin,
but I don't want to step on your shoes 👌

@cortinico I'm not working on it. I honestly don't know enough gradle. More than happy to see someone else do it 😄

@cortinico I'm not working on it. I honestly don't know enough gradle. More than happy to see someone else do it 😄

@cortinico I'm not working on it. I honestly don't know enough gradle. More than happy to see someone else do it 😄

#3453 should add KMM support to detekt 👍


Deterministic DOT output for --dag/--rulegraph

When I run snakemake --dag | dot or snakemake --rulegraph | dot multiple times on the same workflow, I get diagrams with identical layout but different colors for the various nodes. I'm investigating how feasible it would be to make the output from snakemake --dag and snakemake --rulegraph deterministic.
Unless I've missed or misunderstood something, the code for coloring nodes in the DAG._dot method is deterministic.
https://github.com/snakemake/snakemake/blob/cc6c9986271738dfa8394fc45720a4d756af9d0a/snakemake/dag.py#L1976-L1981
Being unfamiliar with Snakemake internals, if I had to guess, I'd assume that the non-deterministic behavior comes when iterating over self.jobs, which happens both for --dag and --rulegraph modes.
https://github.com/snakemake/snakemake/blob/cc6c9986271738dfa8394fc45720a4d756af9d0a/snakemake/dag.py#L1962
https://github.com/snakemake/snakemake/blob/cc6c9986271738dfa8394fc45720a4d756af9d0a/snakemake/dag.py#L1937-L1938
Am I on the right track here? Would it be possible with a reasonable amount of effort to generate DOT output deterministically for these modes?

PR accepted. Closing issue.

Great, thank you!


Camera does not work anymore after latest upgrades

Use https://github.com/jacksonliam/mjpg-streamer instead of the provided one, it should work now.

Sorted by manually installing of https://github.com/jacksonliam/mjpg-streamer/tree/master/mjpg-streamer-experimental


react-navigation

[ ] replace React Router with react-navigation
[ ] remove React Router from package.json, common/components/Match.js, etc.
[ ] add redirect in auth, probably universal

RN 0.42, after

Temp fixed, so let's add react-navigation

Hi! just a heads up - you may run into this bug when trying to implement react-navigation for RN 0.42 https://github.com/react-community/react-navigation/issues/550

@bjrn Thank you.

Closing because of Next Este. I will add React Native support asap.


Collect the Azure subscription ID as part of the publishing profile

The subscription id is needed to take follow-up actions on the bot registration and other resources.  For example, managing channels on ABS will require this value, and if not present, we will have to prompt the user.
This field should be added to the publishing profile at the root level of the object.

[ ] In Composer
[ ] In azureProvision.js cli tool

@zidaneymar, let's get a design on the new schema of publish profile. (with ABS scenario, we may need more info than subscription). And let's paste the design here.

@zidaneymar, let's get a design on the new schema of publish profile. (with ABS scenario, we may need more info than subscription). And let's paste the design here.

{
  "accessToken": "",
  "name": "",
  "environment": "",
  "runtimeIdentifier": "win-x64",
  "settings": {
    "applicationInsights": {
      "InstrumentationKey": ""
    },
    "cosmosDb": {
      "cosmosDBEndpoint": "",
      "authKey": "",
      "databaseId": "",
      "collectionId": "",
      "containerId": ""
    },
    "blobStorage": {
      "connectionString": "",
      "container": ""
    },
    "luis": {
      "authoringKey": "",
      "authoringEndpoint": "",
      "endpointKey": "",
      "endpoint": "",
      "region": ""
    },
    "MicrosoftAppId": "",
    "MicrosoftAppPassword": "",
    "qna": {
      "endpoint": "",
      "subscriptionKey": ""
    }
  },
  "hostname": "",
  "luisResource": "",
  "abs": {
    "botId": "",
    "botName": "",
    "appPasswordHint": "",
    "subscriptionId": "",
    "resourceGroup": "",
    "resourceId": "",
    "armEndpoint": ""
  }
}

@luhan2017  This is the concrete publishing profile for abs scenarios, do we need to move 'abs' section to root level?

@zidaneymar  let's remove the abs group and we have two options to support abs scenario:

appPasswordHint and resounrceId, the resourceId is the abs resourceId like this, we can parse the subscriptionId, resourceGroup and botServiceName from this resourceId.

/subscriptions/3e71927d-2914-4c36-bc5d-6369d1f42457/resourceGroups/composerTest0518-dev/providers/Microsoft.BotService/botServices/composerTest0518

appPasswordHint
subscriptionId  // we are assuming all the services in one publish profile are under the same subscription id? If yes, we can update provision v2's design to remove the subscriptionid.
resourceGroup // we can only guarantee the resource group is the resource group of abs.
botName // We will use subscriptionId + resourceGroup + botName to generate the resourceId, need to check with Hai if this is reliable, will user update the botName to a different value?

@benbrown, We want to make the publish profile as simple as we can, but we do want to make it extensible. Which option do you prefer or do you have other ideas on this?

Note: We need to make sure that any changes we make to this to support the new ABS fields do not break existing profiles.
We will need to capture the subscriptionId regardless of whether the bot was imported from ABS or not.  I think it would make sense to be at the root.


⚠ ClamAV has degraded performance

In c021c55, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 49 ms

Resolved: ClamAV performance has improved in 8235853.


[Docs] Update the README.md of metal3-io to show new X logo (formerly Twitter)

The use of old twitter bird logo can be replaced at most of the places with the new X one

I would like to work on this as my first issue in metal3-io.

Our twitter account does not seem active, maybe we should remove it (especially given the questionable ownership of X/Twitter). In any case, the request seems valid.
/triage accepted

Our twitter account does not seem active, maybe we should remove it (especially given the questionable ownership of X/Twitter). In any case, the request seems valid.
/triage accepted

Sure, so can you assign me so i can remove it from the footer of pages?

/assign


Update csi sidecars

Updates the csi side cars we use.
A quick smoke test shows that these work. I'll run them against kubecover once I have an operator image

I think we need to change these here too https://github.com/storageos/operator/blob/f861e7469cf8d3615288bebd7aaec478b6bf2728/.github/workflows/develop-image.yaml#L10 https://github.com/storageos/operator/blob/f861e7469cf8d3615288bebd7aaec478b6bf2728/.github/workflows/build-and-publish.yaml#L13

I think we need to change these here too
https://github.com/storageos/operator/blob/f861e7469cf8d3615288bebd7aaec478b6bf2728/.github/workflows/develop-image.yaml#L10
https://github.com/storageos/operator/blob/f861e7469cf8d3615288bebd7aaec478b6bf2728/.github/workflows/build-and-publish.yaml#L13

Thanks, updated them now


iterative_fitting not really iterative?

Reading through the code of utils._fit, it looks like the assignment to params_init in the end of the loop (https://github.com/CamDavidsonPilon/lifetimes/blob/master/lifetimes/utils.py#L247) doesn't have any effect because params_init gets overwritten in the beginning of the loop (https://github.com/CamDavidsonPilon/lifetimes/blob/master/lifetimes/utils.py#L240). Looks like line 240 should be outside the loop body.
I'd also give params_init a different name (current_params?) to make the difference between that variable and initial_params clearer.

👍 good catch, I'll PR this now

Fixed in https://github.com/CamDavidsonPilon/lifetimes/commit/97a5815bf4c5e015fc4e98389419106be180362f


Plant Array used in Scheduled email reminder

[x] Nodemailer set up
[ ] Node-scheduler set up
[x] Reminder template created
[ ] User's plant array populates list portion of template

Scrapping for now due to time.


wrong check to verify is calicoctl cluster is running

Is this a BUG REPORT (choose one):

Environment:

Cloud provider or hardware configuration:
Baremetal, VMWare

OS (printf "$(uname -srm)\n$(cat /etc/os-release)\n"):

Linux 3.10.0-693.5.2.el7.x86_64 x86_64
NAME="Red Hat Enterprise Linux Server"
VERSION="7.4 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="7.4"
PRETTY_NAME="Red Hat Enterprise Linux Server 7.4 (Maipo)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.4:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.4
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.4"

Version of Ansible (ansible --version):

ansible 2.5.5
  config file = /Users/luisyo/.ansible.cfg
  configured module search path = [u'/Users/luisyo/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.15 (default, Jun 17 2018, 12:46:58) [GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-9<IP_ADDRESS>)]

On the master this was the output of calicoctl version:
# calicoctl version  | grep 'Cluster Version' | awk '{ print $3}'
retrieve

Kubespray version (commit) (git rev-parse --short HEAD):
6204b85a37fb3eae76d7fd6b92998363acde23e0
Network plugin used:
Calico
Output of ansible run:

TASK [network_plugin/calico : Check that calico version is enought for upgrade] ***
Tuesday 04 September 2018  15:30:24 +0200 (0:00:00.612)       0:29:55.140 *****
fatal: [padm270]: FAILED! => {
    "assertion": "calico_version_on_server.stdout|version_compare('v2.6.5', '>=')",
    "changed": false,
    "evaluated_to": false,
    "msg": "Your version of calico is not fresh enough for upgrade"
}

The task

Determine if calico upgrade is needed

is wrong because it checks again an empty value, but actually calicoctl returns "Unable to retrieve Cluster Version or Type" when there is no cluster version.
Steps to reproduce: our deployment failed after installing calicoctl binary (the task didn't picked up http proxy environment variables for some reason) and later on it continued to fail on the calico upgrade check due to reasons explained before.

hmm there is no word retrieve in output if there is no cluster version available, i just checked, it can be that error is "unable to retrieve image" or something like that, which is not check fault but can be fault with connection to docker hub please post output of: calicoctl version

ok, readed badly issue, this is strange if calicoctl version returns such string, cause i had empty string, can you make some PR with changes? i've lost for some time my playground i have no place to check


Page OnAppearing doesn't get raised

I often put code in the OnAppearing handling behind a page to do various initialization activities that can't occur in the constructor. Much like the old Loading or OnLoading events from all previous UI technologies.
The OnAppearing event doesn't seem to ever be raised by Ooui in wasm. From what I can find there is no event raised when an XF page appears in the browser.

Ahh, yes this is important. I think I just forgot to implement it. I'll try to get it into 0.9.
https://github.com/praeclarum/Ooui/blob/master/Documentation/OouiFormsStatus.md#contentpage


GH-89: Add support for seek from the beginning

Fixes GH-89 (https://github.com/spring-projects/spring-kafka/issues/89)

Introduce TopicPartitionInitialOffset, where it utilize TopicPartition and Long initialOffset
The initialOffset can be:

null - do nothing; rely on the auto.offset.reset of Consumer config;
positive (including 0) - absolute offset from the begging of partition;
negative - the offset from the end of partition: consumer.seekToEnd() - initialOffset

Rework everything around to rely on a new TopicPartitionInitialOffset abstraction
The logic in the KafkaMessageListenerContainer.ListenerConsumer.initPartitionsIfNeeded() reworked to in favor of a new abstraction
remove redundant recentOffset
Reflect new TopicPartitionInitialOffset in the docs

If you are fine with the solution, I'd introduce for the @TopicPartition alongside with the partitions something like:
PartitionOffset[] partitionOffsets() default {};

annotation.

Cool - works well; tested with a "real" application (SI Sample) (positive and negative)...
		ContainerProperties properties = new ContainerProperties(
                            new TopicPartitionInitialOffset(this.topic, 0, 490L),
			new TopicPartitionInitialOffset(this.topic, 1, -3L));

Yes, I was going to suggest we add it to the annotation.
It would be very cool too if the evaluation context for the offset included #topic and #partition variables 😄

Well, we have there this one:
return this.resolver.evaluate(resolvedValue, this.expressionContext);

where BeanExpressionContext expressionContext, so any beans in the ctx are available via @ operator. Therefore any complex initialOffset logic can be resolved somewhere there.
Or do you have something different in your mind ?

In my idea that looks like:
public @interface TopicPartition {

    String topic();

	String[] partitions() default {};

	PartitionOffset[] partitionOffsets() default {};

}

where you can specify partitions() and partitionOffsets(). The last one has precedence though, if the same partition defined.
And that one is like:
@Target({})
@Retention(RetentionPolicy.RUNTIME)
public @interface PartitionOffset {

	String partition();

	String initialOffset() default "";

}

So, not sure how your "@bean.offsetFor(#topic, #partition)" may be useful.
Seems for me some much complexity...
Can't we just provide ContainerProperties into the ConcurrentKafkaListenerContainerFactory and populate the same TopicPartitionInitialOffset[] topicPartitions instead of that SpEL hell for annotations?

Can't we just...

No, because the factory is for multiple listeners; the topics/partitions are defined at the listener level.

So, not sure how your "@bean.offsetFor(#topic, #partition)" may be useful.

Well, a SpEL expression for an offset would need some context @bean.getOffset() is useless without knowing the topic/partition.
I suppose
topic = "foo", partitionOffset =
    @PartitionOffset(partition = "2", initialOffset = "@bean.getOffset('foo', 2)")

would be ok, too, if you think it's not worth the effort to add them as variables; I just thought it would be nicer to provide them as variables during the evaluation rather than having to repeat.
I guess we could consider that for a future release.

I see your point. Yes. But we are missing that in this annotation context we are only in the BeanFactory EvaluationContext. So we can do only something like #{bean.getOffset(...)}.
And we can't customize it (BeanExpressionContext) with variable for the current invocation...
I think we can do some StandardEvaluationContext manipulation to meat your requirements, but shouldn't than that sound like initialOffsetExpression as an alternative?

I see what you mean; we'd have to add more SpEL infrastructure - let's defer that for now; after all they can do something like
topic =<EMAIL_ADDRESS>partition =<EMAIL_ADDRESS>offset =<EMAIL_ADDRESS>
instead.

Pushed @PartitionOffset support.

Polishing here https://github.com/garyrussell/spring-kafka/commit/1d54ec97363ad2377d5bf1a459819df4a541baf1 for your review.

Merged as 06616f73eda4ea7f341c3d43cf211d6f4eb54211


[3.0] Consistency in the naming of service managers

Current we have the service_manager but if we look at all the other services in the file below we will notice it does not follow the same principle as the rest of them. It's logical name would be services
https://github.com/zendframework/zf2/blob/master/library/Zend/Mvc/Service/ModuleManagerFactory.php#L48
And the same goes for the ControllerLoader which should be named ControllerManager
https://github.com/zendframework/zf2/blob/master/library/Zend/Mvc/Service/ModuleManagerFactory.php#L53

I created PR https://github.com/zendframework/zend-mvc/pull/6 for changing ControllerLoader to ControllerManager.


Fix tasks states

I guess that, as long as ARM-software/devlib#90 gets merged, this PR can enter as well.


remove interactive CLI argument

CLI args to remove:

--interactive

I have tried to find this argument, but it does not exist in the code. Am I missing something?

Maybe it got removed already.


Conflicting advice

I was given conflicting advice on how to fix the following line:
if link_array.length > 0
It seems like a bug to have two different ways to fix the same line.
app/helpers/application_helper.rb:35:8: C: Use link_array.length.positive? instead of link_array.length > 0.
    if link_array.length > 0
       ^^^^^^^^^^^^^^^^^^^^^
app/helpers/application_helper.rb:35:8: C: Use !empty? instead of length > 0.
    if link_array.length > 0
       ^^^^^^^^^^^^^^^^^^^^^

RuboCop version
$ rubocop -V
0.47.1 (using Parser <IP_ADDRESS>, running on ruby 2.4.0 x86_64-darwin14)

Well, the same code can have many different offenses... If you fix it in the second way, both offenses should go away. If you fix it in the first way, the second offense should still hold. Not sure if that happens right now, but that cop should probably also detect length.positive? as an offense.

I'm closing this ticket due to no recent activity. Feel free to re-open it if you ever come back to it.


strip whitespace when it uses tabs

This pr attemps to fix #61 .
I don't know if there is a better way to do this, feel free to give suggestions.

I didn't add a test like
\t\tFoo
\t\tBar

Because "\t" will be lexed as "\\t" now, which is not a whitespace.

bors r+


Add monitor

Add a new tool to monitor all traffic passing through an AGW TNC. I would run it in a separate command prompt window from converse.

The chatter program comes close, but it only monitors traffic on one TNC port. And by default it hides repeated packets and packet types other than I and UI.


Snaps' list doesn't show up

I downloaded swiftsnapper's version that i couldn't see my snaps but just a list. When i downloaded the latest version that everybody talks about because you can finally see your snaps, my snaps' list doesn't show up anymore

Version they were talking about was https://ci.appveyor.com/project/filfat/swiftsnapper/build/0.1.141 which doesn't work anymore (i think) because of API problems.

After log in cannot view the snaps received.

Same problem,any fix?

Same as #102


hover effects added

closes #425
Description
added hover colour change

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[X] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[X] My code follows the code style of this project.
[X] My change requires a change to the documentation.
[X] I have updated the documentation accordingly.

@MAVRICK-1  please merge PR under GSSOC label


Pinning problems when a service name includes dot

Problem in GUI
Found in v0.12.1 while testing https://nft.storage  – having "." in name means bad time in both GUI and CLI.

Quick fix for GUI is to replace "." with "-" or "_" when Service is added  from GUI, but  that won't solve the problem if  one added it via CLI:
Problem in CLI
$ ipfs pin remote service add nft.storage https://nft.storage/api [secret]
$ ipfs config Pinning.RemoteServices.nft.storage.Policies.MFS.Enable
Error: failed to get config value: "Pinning.RemoteServices key has no attributes"

How to fix this?
@alanshaw @aschmahmann given that we use "nft.storage" in pin remote  examples on https://nft.storage, we should address this before the problem escalates when people start using it :sweat_smile:
I tried explicit percent encoding %2E but it does not work  (we would have to modify both webui and go-ipfs to allow that).
Is there a preexisting way to pass . correctly?
If not, we need to either

(a)  tweak go-ipfs to percent-decode service name to make %2E work,
(b) ban . from service names and replace with - in docs
(c) ?? (is there other way to resolve this?)

Does this work?
$ ipfs config Pinning.RemoteServices["nft.storage"].Policies.MFS.Enable

Sadly no:
$ ipfs config Pinning.RemoteServices["nft.storage"].Policies.MFS.Enable
Error: failed to get config value: "Pinning key has no attributes"

@lidel is this specific to pinning services or can ipfs config not properly deal with things like modifying public gateway properties (e.g. https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#gatewaypublicgateways)?

@aschmahmann  yes, it is a global problem, we seem to lack ipfs config --json notation that removes the ambiguity of .


'os'  does not belong in use cases

The project repo interface should contain the  code that looks for files in the os vs looking up a project from sql (for example if an sql adapter was needed.) the finders should all live on the adapter not in the use cases.

https://github.com/dbellotti/timekeeper/pull/16