Step 2: Identify valuable quotes to share

Description:
While analyzing and reviewing the 600 Aug 2023 emails, identify a couple valuable quotes from the emails. Find 3-8 quotes from all emails in August that can be added to the report - does not need to be specific to a theme or category.
Acceptance Criteria:

[ ] Identify and write down 3-8 quotes from the Aug 2023 emails that are of value to product owners to gather more insight into how users are feeling

Development Checklist:

Read through emails Max 2 times
Maximum of 16 categories/themes identified

Sub categories are not necessary

Maximum 3-8 quotes from all 600 emails

Dependencies

access to emails

Definition of Ready

[x] Acceptance criteria are included
[x] Dependencies are identified (technical, business, regulatory/policy)
[x] Story has been reviewed and questions answered

Definition of Done

In progress:

[ ] Acceptance criteria are complete

PO Review:

[ ] Acceptance criteria are confirmed complete
[ ] Reviewed and approved by Product Owner

Notes:

quotes look good - moving to done


Implement haproxy for connected clusters to facilitate easier GUI access

Using haproxy to pass traffic from laptop to bastion to cluster is easier than using sshuttle it seems, we should just have this enabled for ipv4 clusters too.

cc @venkataanil


proposal: module/theme.txt to .json

I would like to propose to move to a .json file for the orchard module/theme properties. (or support both txt/json).
Currently you have to lookup the documentation to find out which properties there are, which ones are required/optional and make sure you don't mistype / wrong format.
All of this (in my opinion) can be made easier and more robust by using a .json file which has nice intellisense and validation by providing a json schema
Also the new project.json file already provides many properties which we can use (which are also required for creating nuget packages) in order to prevent having 2 places for the same information (name, author etc).
I have come up with two possible solutions:
1 merge with project.json
Advantages:

schema validation
intellisense
one place for all information

Issue:

visual studio nuget dependency intellisense not working

{
  "$schema": "orchard-project-schema.json",
  "version": "2.0.0",
  "title": "Orchard Demo",
  "description": "This is an Orchard Demo extension",
  "authors": [
    "The Orchard Team"
  ],
  "projectUrl": "http://orchardproject.net",
  "licenseUrl": "",
  "tags": [
    "orchard",
    "demo"
  ],

  "orchard": {
    "minVersion": "2.0.0",
    "module": {
      "antiForgery": false,
      "path": "OrchardDemo",
      "features": [
        {
          "id": "Orchard.Demo",
          "name": "Demo",
          "description": "Demo description",
          "category": "Demo",
          "priority": 0,
          "dependencies": [
            "Orchard.Environment.Commands"
          ]
        }
      ]
    }
  },

  "dependencies": {
    "Microsoft.AspNet.Mvc": "6.0.0-*",
    "Orchard.ContentManagement": "2.0.0-*",
    "Orchard.Hosting.Web": "2.0.0-*"
  },
  "frameworks": {
    "net451": { },
    "dotnet5.4": { }
  }
}

2 separate orchard.json
Advantages:

schema validation
intellisense
separate place for all orchard specific information

{
  "$schema": "Schemas\\orchard-schema.json",

  "minOrchardVersion": "2.0.0",
  "module": {
    "antiForgery": false,
    "path": "OrchardDemo",
    "features": [
      {
        "id": "Orchard.Demo",
        "name": "Demo",
        "description": "Demo description",
        "category": "Demo",
        "priority": 0,
        "dependencies": [
          "Orchard.Demo.Commands"
        ]
      }
    ]
  }
}

{
  "$schema": "Schemas\\orchard-schema.json",

  "minOrchardVersion": "2.0.0",
  "theme": {
    "baseTheme": "base",
    "zones": [
      "zone1",
      "zone2"
    ]
  }
}

@bleroy @sebastienros

We have talked about this before - and there were reasons why we wanted to leave it as yaml

If the question is whether to unify extension manifests into a single file then I see no issue apart from the case when you want to load modules and themes from the same folder (and since you can specify custom extension folders, this is possible) and leave the option open to distinguish between them without a flag in the manifest.
To have a single project.json seems a step in the right direction to me (with the above kept in mind) since it removes something that'd be unique to Orchard in contrast to the frameworks's built-in similar format (project.json being a package manifest). The question I think here is whether we want to keep the extension manifests as human-readable as possible (YAML is better than JSON on this matter). I don't think this counts too much: anybody who ever reads extension manifests directly are solely developers and they should be able to cope with this. However I don't think there's any more advantage than "having one less Orchard-specific thing" which in this case is probably worth it.

if you don't have a manifest, we need to include the project.json even when distributing the module, not sure it's good. Also, we need a custom json schema which will copy the one from project.json ... not mentioning the risk of conflicts as project.json might change in the future. Project.json is the .csproj, target for the compiler/ide. I'd vote to keep the yaml file for now.

Good point about binary deployment: we can't have project.json there, haven't thought about this. Then nothing to discuss as this renders it a complete no-option.

I beg to differ that only developers read manifests. This being said, I don't feel super-strongly about YAML any more. JSON has won the Internet since then (too bad it doesn't support comments).

I think the pros outweigh the cons. The fact it allows intellisense is kinda cool.
if the project.json file changes in the future, doesn't matter to us as we will have a schema.
Not sure, I kinda think this is a good thing.... it sounds like we are 50/50 here!?

Can we not care, by having a central notion of a parser interface?

I want a separate file, I don't care what format it is in. And yes, json would help with intellisense.

okay, I can agree to that. @jorisdebock what do you think? keep it in JSON, but in a different file.

module.json, theme.json, same fields as before for now. It means less changes, easier migration. Until we really want to change something.

+1

ye i think a seperate file would be best.
@sebastienros  why not one file for both like the 2nd option? The schema allows module or theme but not both.

Do we want to also reconsider the difference between themes and modules? Both can provide views and code. They can still live in different directories, while being conceptually identical. Maybe only a flag in module.json differentiates them, if anything. What are the cons?

@Jetski5822 Will there be a decision to this proposal? Good idea, keep supporting both maybe (like currently in the pull request), etc.
There might be some properties incorrect but I based it off the orchard v1 documentation and the description in the json schema needs to be checked. So I would like to request a code review from someone on this.
If we go with this I would suggest uploading the schema to http://schemastore.org/json/ so visual studio can automatically use this, suggested name: "orchard.json"

I'd keep the current folders, Themes and Modules, but would not make an assumption on what they contain, and also have these folder only because they are defined in the configuration so that people can put everything in Extensions if they want. The manifest name would make the distinction: theme.json and module.json as they might have different schemas.

Continuation on this issue: https://github.com/OrchardCMS/Orchard2/issues/91


Fix lint

This PR fixes all of the lint causing the build failures identified by #148.

Once #148 is merged, I will immediately rebase this and then this can be merged to get a green CI.


[iqiyi] None: Can't find any video

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm reporting that yt-dlp is broken on a supported site
[X] I've verified that I'm running yt-dlp version 2023.06.22 (update instructions) or later (specify commit)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[X] I've read about sharing account credentials and I'm willing to share it if required

Region
No response
Provide a description that is worded well enough to be understood
[iqiyi] None: Can't find any video
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[X] If using API, add 'verbose': True to YoutubeDL params instead
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
C:\yt-dlp>yt-dlp -vU yt-dlp --cookies-from-browser Firefox https://www.iqiyi.com/v_19rthmf0b0.html
[debug] Command-line config: ['-vU', 'yt-dlp', '--cookies-from-browser', 'Firefox', 'https://www.iqiyi.com/v_19rthmf0b0.html']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>[812cdfa06] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.22621-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg N-110509-g722ff74055-20230506 (setts), ffprobe N-110509-g722ff74055-20230506
[debug] Optional libraries: Cryptodome-3.18.0, brotli-1.0.9, certifi-2023.05.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-11.0.3
[Cookies] Extracting cookies from firefox
[debug] Extracting cookies from: "C:\Users\84562\AppData\Roaming\Mozilla\Firefox\Profiles\wabojgkx.default-release-1-1685454224602\cookies.sqlite"
[Cookies] Extracted 194 cookies from firefox
[debug] Proxy map: {}
[debug] Loaded 1851 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp-nightly-builds/releases/latest
Available version<EMAIL_ADDRESS>Current version<EMAIL_ADDRESS>Current Build Hash: 64bc9dc2239dd2373d7efa206d4853301a0c32828d99a960bde6b4e244bf940d
yt-dlp is up to date<EMAIL_ADDRESS>[CommonMistakes] Extracting URL: yt-dlp
ERROR: [CommonMistakes] None: You've asked yt-dlp to download the URL "yt-dlp". That doesn't make any sense. Simply remove the parameter in your command or configuration.
  File "yt_dlp\extractor\common.py", line 708, in extract
  File "yt_dlp\extractor\commonmistakes.py", line 25, in _real_extract

[iqiyi] Extracting URL: https://www.iqiyi.com/v_19rthmf0b0.html
[iqiyi] temp_id: download video page
ERROR: [iqiyi] None: Can't find any video; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "yt_dlp\extractor\common.py", line 708, in extract
  File "yt_dlp\extractor\iqiyi.py", line 355, in _real_extract

Duplicate of #2740


[SPARK-16965][MLLIB][PYSPARK] Fix bound checking for SparseVector.

What changes were proposed in this pull request?

In scala, add negative low bound checking and put all the low/upper bound checking in one place
In python, add low/upper bound checking of indices.

How was this patch tested?
unit test added

Test build #63416 has started for PR 14555 at commit 57b9c84.

Test build #63416 has finished for PR 14555 at commit 57b9c84.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63416/
Test FAILed.

Merged build finished. Test FAILed.

Test build #63511 has started for PR 14555 at commit 111cb28.

Test build #63511 has finished for PR 14555 at commit 111cb28.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63511/
Test FAILed.

Test build #63518 has started for PR 14555 at commit 7c01da8.

Test build #63518 has finished for PR 14555 at commit 7c01da8.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63518/
Test FAILed.

Merged build finished. Test FAILed.

Test build #63530 has started for PR 14555 at commit 6ebd4da.

Test build #63530 has finished for PR 14555 at commit 6ebd4da.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63530/
Test PASSed.

Merged build finished. Test PASSed.

Test build #63603 has started for PR 14555 at commit b291423.

Test build #63603 has finished for PR 14555 at commit b291423.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63603/
Test PASSed.

OK, one last over-arching comment. I know the arg validation logic was split over two methods, and I think it was on purpose, so that calls to the SparseVector constructor didn't pay the full overhead of checking all the values. (CC @jkbradley did we talk about this a long time ago?) I think that may be something desirable to leave in place, even if it won't matter to callers of Vectors.sparse.

I think this is about ready, though to be conservative, I think we need to put the validation back where it was, so that the more expensive check only happens in Vectors.sparse and not the constructor. And then add tests accordingly.

Ping @zjffdu

@srowen Do you mean putting the validation back to method Vectors.sparse() ?  The constructor of SparseVector is public, so I think we should keep the validation in SparseVector rather than its factory Vectors.

That's a good point, it's public now. I think I agree with you then, it's fairly important to retain the validation there. It would also be inconsistent to move it back, yes, because I see the other factory methods don't do the same validation.
I suppose that if it were a big issue we could find a way to expose a private alternative constructor that skips validation where internal code knows the inputs are valid.

Last call for @mengxr or @jkbradley -- just want to make sure one's OK. Certainly good for correctness.

Hey, all.  One of my colleagues run into a bug related to the discussion above.
He called the Vectors.sparse(size: Int, indices: Array[Int], values: Array[Double]) without noticing that the indices are assumed to be ordered. And the vector he created has all value of 0, if we try to get value via apply method. However, SparseVector.toArray simply generate a array using a for loop that is order insensitive. Hence, you will get a all-0 vector when you call apply method, while you can get correct result using toArray or toDense method.
The lack of indices validation in a public method could be quite problematic.

Jenkins retest this please

Test build #64059 has started for PR 14555 at commit b291423.

Test build #64059 has finished for PR 14555 at commit b291423.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/64059/
Test PASSed.

Merged to master


Parameterisable queries

Provide the ability to parameterise a reusable query, so only variables can be passed to it each time.

Completed.


WIP: link webhook config from master branch

Link webhook code from master branch.

Q from sig-docs: master is the dev branch, right? So cannot be guaranteed to be stable? In this case, I think we don't want to be linking to it, right? We might have a version variable/shortcode that we can use instead, if appropriate.
cc @tengqm

@Bradamant3 yes, you are right. Links should be to latest stable version. I'll update this PR accordingly.


Delegate blocking ops to executor in JobRunner

@mpolden please review.

Unused file deletion also staged, thanks to overly helpful IDEA :)

plong


 For now, console.log and similar commands can do, but eventually, I want to switch to something more advanced, like https://github.com/winstonjs/winston.

https://developer.mozilla.org/en-US/docs/Learn/Server-side/Express_Nodejs/deployment

Duplicate of #96

Duplicate of #96


Flow types breaks in latest version v8.0.1 but doesn't in v7.1.1

Hi,
All my flow variables give "no-undef" error when I upgrade babel-eslint to v8.0.1 but i don't get the same error in v7.1.1. Can you please let me know what has happened here and also advise me on whether I need to do any workarounds when using v8.0.1.

Thanks for the report, I'l need you to provide more exact reproduction steps. I just know that it doesn't work with flow variables?

It breaks in 8.0.2 as well. As of now 7.1.1 works like a charm.

i am also experiencing this, works fine on 7.2.0, but i start to get the no-undef and no-unused-vars from 7.2.2.
i used to have 7.2.3 but after i updated webpack to 4.3.0 (don't know how is that related) it started failing, i use eslint-plugin-flowtype but upgrading that to 2.46.1 did not help (i also updated some other dependencies but those are the main ones)

Which version of eslint are you using? I found that when there is eslint@3 and eslint-scope, the same problem occurs, probably because of the conflicts of scope handlling: eslint@3 uses escope to handle JavaScript scopes while babel-eslint uses eslint-scope.
In my environment, eslint-scope was installed as one of the webpack@4's dependencies.
The problem was resolved when I updated eslint to its latest as eslint@4 uses eslint-scope instead of escope.
The minimum code to reproduce: https://github.com/nodaguti/babel-eslint-scoping-bug/

Thank you for the issue. Now that @babel/eslint-parser has been released, we are making this repository read-only. If this is a change you would still like to advocate for, please reopen this in the babel/babel monorepo.


Add a way to restart a scanning session

Right now, I couldn't find a way to restart a new scanning session, other than using the continuous option or making a new instance of the class.

ive made some local changes to the package to kind of enable that, if it helps, there should definitely be better options.
ScannerViewController.swift
        func updateViewController(isTorchOn: Bool, isGalleryPresented: Bool, toReset: Bool) { // new toReset binding
            if let backCamera = AVCaptureDevice.default(for: AVMediaType.video),
               backCamera.hasTorch
            {
                try? backCamera.lockForConfiguration()
                backCamera.torchMode = isTorchOn ? .on : .off
                backCamera.unlockForConfiguration()
            }

            if isGalleryPresented && !isGalleryShowing {
                openGallery()
            }
            if toReset { // if toReset, use reset function in ScannerCoordinator.swift
                delegate?.reset()
                delegate?.parent.toReset.wrappedValue = false
            }
        }

CodeScanner.swift
    public var isGalleryPresented: Binding<Bool>
    public var toReset: Binding<Bool> // new binding variable
    public var videoCaptureDevice: AVCaptureDevice?

...

        isGalleryPresented: Binding<Bool> = .constant(false),
        toReset: Binding<Bool> = .constant(false), // in init function
        videoCaptureDevice: AVCaptureDevice? = AVCaptureDevice.default(for: .video),

...

        self.isGalleryPresented = isGalleryPresented
        self.toReset = toReset
        self.videoCaptureDevice = videoCaptureDevice

...

  public func updateUIViewController(_ uiViewController: ScannerViewController, context: Context) {
        uiViewController.updateViewController(
            isTorchOn: isTorchOn,
            isGalleryPresented: isGalleryPresented.wrappedValue,
            toReset: toReset.wrappedValue // added this
        )
    }

Followed this to temporarily overwrite with local changes
https://developer.apple.com/documentation/swift_packages/editing_a_package_dependency_as_a_local_package

@charpple @j-cheung
Tip: there are many cool functions in this package, unfortunately Paul is shy to write about them :(
If you look CodeScanner.swift, there are many params like shouldVibrateOnSuccess ,isTorchOn, and even completion though I haven't figured it out how the last one works.
But the one you need is scanMode, and when using you can choose from .continuous,.once and .oncePerCode. Seems like for your case .continuous will be the right one.
This how my code with specified scan mode looks
CodeScannerView(codeTypes: [.qr], scanMode: .continuous, simulatedData: "Paul Hudson") { response in
            switch response {
            case .success(let result):
                print("Found code: \(result.string)")
            case .failure(let error):
                print(error.localizedDescription)
            }

And using scanInterval you can specify the time in seconds between the scans (it takes double so you can specify like 0.5 seconds and so on)
In this example I have 10 seconds delay between the scans
CodeScannerView(codeTypes: [.code39], scanMode: .continuous, scanInterval: 10 ,simulatedData: "Paul Hudson") { response in
    switch response {
    case .success(let result):
        print("Found code: \(result.string)")
    case .failure(let error):
        print(error.localizedDescription)
    }
}

Hope this helps!

@Sat1l I'm not too shy, I'm too busy – have you considered perhaps contributing some documentation in the README, given that this is an open source project?

@twostraws seems like fun, just give me some time :)

Closing as it seems to be solved. If not, please reopen and give details about what you're still struggling with.


Better english and format.

Hey!
I found the proof of concept really interesting but I noted some grammar errors and general formatting issues, so I decided to do some fixes on a fork so you can add them with a pull request.

Thank you such much <3


getOrSet in yii\di\Container

Not an issue, more like an improvement.
It would be cool to have something like yii\caching\Cache's getOrSet in yii\di\Container.
It would basically be yii\di\Container::get() with a normalized definition that gets saved before the object is built.
I know:

I can bootstrap the part in which I set the definitions or init them in the parts of my app that rely on them to be available.
I can just check if the definition exists, if not, set it up and it will be used whenever an instance of that will be requested via DI.

But

Those pile up unnecessarily, I will have definitions my application won't need in order to function. Moreover, plugging/unplugging those can lead to errors.
Requires more code, getOrSet would be a nice shortcut.

Thanks for proposal. For 2.x we cannot apply it since it's in bugfix-only mode for a while. For 3.0 DI container is different: https://github.com/yiisoft/di


Report: Open Links in a new tab

in the current report, if there is any Link inside the details, the link will be opened in the same tab.  would be cool if this will be opened in a new tab.   example: EIDSCA.AP05 Links to GE

This is not recommended due to poor accessibility, see https://www.w3.org/TR/WCAG20-TECHS/H83.html
Users can easily achieve this by clicking the mouse wheel, ctrl + click, right click -> open new tab etc. depending on their browser.

I second @fflaten here and would avoid to go against W3 recommendations


Prefer creating larger denoms at the second step of CreateDenominated

This also means less smaller denoms (which could be good for privacy) and fixes a bug (not mixing all the available funds), see comments in code for more info.

Rebased on develop to fix merge conflicts (clean rebase, no actual code changes).

backported in https://github.com/dashpay/dash/pull/3670


SignFile should support RFC 3161 SHA256 timestamp

My understanding is, that Microsoft requires (maybe not enforcing yet) that both the digest and timestamp digest are SHA256 to be a valid Authenticode SHA-2 signature on Windows 10 as of May 2017.
The current SignFile task is not able to produce such signature, but produces a signature with SHA256 file digest, but SHA1 timestamp digest.
Expected
In my opinion SignFile should use RFC 3161 SHA256 timestamp (/td SHA256 /tr <RFC3161-timestamp-url>) when signing, or at least support it.
Actual
Currently the when using SignFile to sign with SHA256 digest (TargetFrameworkVersion >= v4.5), it only supports using SHA1 digest for the timestamp. (it uses /t <timestamp-url>).
https://github.com/Microsoft/msbuild/blob/a9f64ebd108702c3fc65339c66cb124217854524/src/Tasks/ManifestUtil/SecurityUtil.cs#L755-L756
Dual signing
It would also be really nice to be able to dual sign, which would require the /as signtool parameter

I just found out that this has been fixed in #4397 which has been released in MSBuild 16.2


Fix 2016 Repentance Day in Saxony

Fixes #168

could you please amend the changelog?

superseded by #183


Bug: Filtering transactions by user

This bug is related to url encodings when making requests to the be api. We basically need to replace the %.. before making a request.
// apps/explorer/src/app/hooks/use-txs-data.ts
export const getTxsDataUrl = ({ limit, filters }: IGetTxsDataUrl) => {
  const url = new URL(`${DATA_SOURCES.blockExplorerUrl}/transactions`);

  if (limit) {
    url.searchParams.append('limit', limit);
  }

  if (filters) {
    url.searchParams.append('filters', filters);
  }

  return url;
};

Fixed at some point in the past


Make tests robust and independent

This is a continuation from #521 that puts the branch in the upstream repo rather than my fork to enable windows test runners to access the GAMS_LICENSE secret. Please see the original PR for a description, to-dos, etc.
PR checklist

[ ] Continuous integration checks all ✅
[ ] Add or expand tests; coverage checks both ✅
[ ] Add, expand, or update documentation.
[ ] Update release notes.

I'm just noticing: the windows tests consume much more time than all others. This is not specific to this PR; it was true also for recent dependabot PRs. Not sure if that is worth investigating, definitely not high in priority, I'm afraid.

I'm rerunning these tests a few times now to see if any flakiness comes up. If it doesn't, I'm happy with the current state of the PR. Just to be sure: my current solution for the flakiness of the tutorial tests is to run these tests on the same worker, which might run counter to our desire to distribute the runs using pytest-xdist. However, the tutorial suite in ixmp is small enough so that this doesn't worry me. Most test on recent Python versions still finish in ~3 minutes, which I think is fine.

The second rerun already revealed something interesting: On windows-py3.10, both of the first tutorial tests timed out when trying to reach the platform, but the second ones where happy to start. Unfortunately, for the R tutorials, the first test is required as the second tutorial requires the scenario from the first tutorial to be available. I have now introduced the pytest-dependency plugin to skip the second tutorial if the first one failed, but this still doesn't explain why the cells couldn't connect to the platform in the first place.

https://github.com/iiasa/ixmp/issues/467 is also still open, despite there being a workaround in our code base: the tutorial tests are skipped on ubuntu because they're taking too long. Not sure if we need to keep this issue open, I'm not working on running the tutorials on ubuntu with this PR. We could still use it as a tracking issue if we want to remove the skips one day.

We discussed test_del_ts extensively in #521.
I'm now choosing to not dive deep into this or the tutorial-related failures since they may well be due to very specific technological details that are not relevant anymore once we transition to ixmp4. As far as I know, ixmp4 is currently only tests on ubuntu, but doesn't contain a single flaky test (despite having ~90% coverage). So for now, I think our best course of action is to mark these tests as flaky (rerunning them when they are failing) and migrating to ixmp4 as soon as we can.
I'd not take the same approach with message_ix or message-ix-models, which won't be replaced in the future as far as I can tell.

I have rerun the tests now four times and did not observe any flaky behaviour, not even reruns due to flakiness. One time, the setup-graphviz action failed due to the backend server being overloaded, but there's not a lot we can do about that in this PR.
I did notice, though, that consistently this happens: XPASS ixmp/tests/backend/test_jdbc.py::test_del_ts - https://github.com/iiasa/ixmp/issues/463
This test got excluded for all Python versions <= 3.10 because of an incompatibility with jpype 1.4.1 and Python 1.5.0, which does not seem to persist with pandas > 2.x. I'll remove the xfail mark for this and see what happens.


RUSTSEC-2021-0073 in prost 0.7

RUSTSEC-2021-0073 was issued recently against prost 0.7, which is used by the prost-codec feature flag. It's fixed in prost 0.8, which has some backwards-incompatible changes but not much.
I believe the version of prost can be updated without any code changes. Unfortunately it is technically a breaking change as public raft types implement prost::Message, and any clients who use this impl will therefore be broken by the prost update. This could be mitigated going forward by re-exporting prost itself and documenting that backward compatibility of the prost-codec feature requires clients who wish to use the prost::Message impls to use the re-exported prost. Or duplicate the types so the ones that implement prost::Message are private. But for this particular update, the technically-correct thing to do is update the version to 0.7.
The rationale for going ahead and doing this now is anyone using cargo-deny will by default get an error about this RUSTSEC.

We don't use timestamp types. As all fields of protobuf types are public, so users of 0.7 should be able to get around the issue without breaking changes.
raft-rs can upgrade prost to 0.8 in v0.7.0.

I have the same problem.


Adding java version for leetcode question 2 and 3

Hi, I'm preparing for interviews. Please review my pull request.


Add a way to cache a document

Loading your workflowy list takes some time, so it would be nice to have a simple way to serialize/deserialize a workflowy document.

Thank you for the issue. What use case did you have in mind?
Caching is possible with the current API already by instantiating the Client, fetching data manually (and caching them) and then instantiating the Document, exacly like here: https://github.com/karelklima/workflowy/blob/main/src/workflowy.ts#L40-L42
One problem with caching is that the WorkFlowy document will get out of sync rather quickly with the original. It is possible to write a synchronization logic, and I will likely do it in the near future, but the additional complexity may not be worth it.
Finally, it also depends whether you run a long-running script with this library, and then you can just leave the document in the memory, or whether you are building something like a CLI tool which runs end exits.

yeah this is linked with the design of sunbeam. Every page maps to a distinct process, so the data needs to be persisted between runs.

Alright, thanks for the info. I saw that you solved that by using localStorage, I was thinking of doing the same within this library.


Adding a rule for module display/show to correctly expand paths

The following addition may be required tocorrectly expand the path when executing module display toolname  :
[module-info mode load] || [module-info mode switch] || [module-info mode remove] || [module-info mode display]

same as issue #56 .


How to set normalize_embeddings

When I use Batchference for sentence embedding，How to set normalize_embeddings=False：
https://github.com/michaelfeil/infinity/blob/main/libs/infinity_emb/infinity_emb/transformer/embedder/sentence_transformer.py#L118

It's currently no possible. Generally, the magnitude of the embedding has little meaning - why are you interested? If you name a good reason, I'll maybe implement it.

@wlleiiwang Can you elaborate please?


Style guide violation

https://github.com/dfinity/motoko-base/blob/5c9145e79bf471fb662b9b4028937783e97e23d1/src/Buffer.mo#L68
As far as I understand the style guidelines this should be size_ not _size because _size suppresses a compiler warning.

True, but, FTR, haven't implemented that (unused identifier) warning yet ...


Create a example with native mongodb client

A example with native mongoclient, without mongoose will be very helpful.

Can you submit a pull request ?

Yes. I can.


Implementing MTLS/SSL

Hey!
I have a server which needs to be called using MTLS/SSL.
I tried doing the following:
http = HTTPX.with(
  ssl: {
    verify: true,
    verify_mode: OpenSSL::SSL::VERIFY_PEER,
    client_cert: OpenSSL::X509::Certificate.new(cert),
    client_key: OpenSSL::PKey::RSA.new(key)
  }
).get(url)

But it does not work, the application just hangs. The only log I get is the following:
resolver: answer AAAA url.com.br: [#<IPAddr>, #<IPAddr>]

Am I doing anything wrong? Does HTTPX supports SSL in the same way as Faraday?
This is the Faraday code I use and it is currently working:
connection = Faraday.new(url: base_url, ssl: {
  verify: true,
  verify_mode: OpenSSL::SSL::VERIFY_PEER,
  client_cert: OpenSSL::X509::Certificate.new(cert),
  client_key: OpenSSL::PKey::RSA.new(key)
}
) do |faraday|
  faraday.request :json
end

connection.get(url)

Hi,
httpx supports the same options as OpenSSL::SSL::SSLContext#set_params, in the ssl option, so you can use the docs as reference (https://docs.ruby-lang.org/en/master/OpenSSL/SSL/SSLContext.html#method-i-set_params)

Thanks for the quick reply, and for the clarification!
I was using the wrong name on the params, I managed to make it work with the following:
http = HTTPX.with(
  ssl: {
    verify_mode: OpenSSL::SSL::VERIFY_PEER,
    cert: OpenSSL::X509::Certificate.new(cert),
    key: OpenSSL::PKey::RSA.new(key)
  }
).get(url)

Great that worked. I'll keep this open though, as I'll address the hanging part when I find the time.

can't reproduce the hanging behaviour, wrongly outlied ssl options will result in a NoMethodError, as expected. closing until further reproduction.


When will the pretrained model be released?

Hi I'm super interested in this paper.
Currently we are retraining the MaskCLIP+ and the results do not seem so goos as in the paper. May I ask when will the pretrained models be released?

Any news on this?


[FLINK-33317][runtime] Add cleaning mechanism for initial configs to reduce the memory usage

What is the purpose of the change
This PR is FLINK-33317, it's a subtask of FLINK-33315. See FLINK-33315 and FLINK-33317 for details.
Brief change log
Clean up the StreamConfig#SERIALIZEDUDF after getStreamOperatorFactory.
Verifying this change
This change added tests and can be verified as follows:

Added StreamConfigTest#cleanSerializedUDF

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving): no
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
The S3 file system connector: no

Documentation

Does this pull request introduce a new feature? no

Hi @pnowojski @RocMarshal , would you mind helping take a look this PR in your free time? Thanks a lot

Hi @pnowojski @RocMarshal , some unit tests failed when getStreamOperatorFactory throwing an exception after SERIALIZED_UDF is removed.
I researched it for a while and found that it takes some time to refactor related tests. However, I don't have enough time this week because flink-kubernetes-operator will be released recently. And one autoscaler feature is nice to finished before this release, and this PR belongs to flink-1.19, it has enough time to develop. So I will work on flink-kubernetes-operator first, and back to this PR after that.
Sorry for the delay,  and thanks a lot for your quick review, have a good time to you.

Hey @pnowojski @RocMarshal , I have fixed the test, would you mind helping double check? Thanks

Hey @1996fanrui , my 👍 still stands :)

Hey @1996fanrui , my 👍 still stands :)

Thank you, I rebased master just now, let's wait for the CI. and I will merge it after CI is green.


Allow array parameter default value

Previously schema.default tag for schema.type === array was never processed and only schema.items.default were usable.

Do you mind adding a unit test?  Ideally one that fails prior to your change and then passes after.  Also, when preparing your commit run gulp to not only run the tests and lint the code but to also generate the browser files.


Style Utilities : Incomplete TOC

Could not provide custom height as of now so it is on Hold.

Problem :

Fill this blank space .

Duplicate issue exsit in #16
Issue status : Transfered to issue no. #16
App Info :

Key
Value

Fronend URL
http://wrappid.framework.s3-website.ap-south-1.amazonaws.com/guide/styles

Fronend Version
0.0.5

Desktop :

Key
Value

OS
Wisdows

Browser
Google Chrome

Version
120.0.6099.225 (Official Build) (64-bit)


artifacts.zip missing Setup.ps1

Within:
https://github.com/MicrosoftDocs/mslearn-developer-tools-power-platform/blob/master/fusion-developers/artifacts.zip
The README document has two item #1 entries, the second one says:

Alternatively, run the PowerShell script.
```powershell
./Setup.ps1
```

The ZIP file, however, does not contain a Setup.ps1 file.

It's there. I overlooked it.


floating witch animation added

A floating witch is added which moves across the page to increase interactivity

Great! Just rebase your branch with master and resolve the merge conflicts and put the javascript code in home.js. And if you could, try and see how the page look, if the witch animation is recurring.

@grusha02 I am opening a separate issue for what Diptanshu said, you can have a go on it.

I am sorry sir I am a little new to this so I didnt rebase my branch. I will try to solve the new issue


Clean custom OSL extensions, unneeded comments

Would make sense to squash the commits into one before merging.


generate_figure: ValueError: x and y must be the same size occurs if T1w and T2w subjects do not match

If the subjects don't match between T1w and T2w, the following error is raised:
Traceback (most recent call last):
  File "/Users/julien/miniconda3/bin/sg_generate_figure", line 11, in <module>
    load_entry_point('spinegeneric', 'console_scripts', 'sg_generate_figure')()
  File "/Users/julien/code/spine-generic/spine-generic/spinegeneric/cli/generate_figure.py", line 791, in main
    generate_figure_t1_t2(df, csa_t1, csa_t2)
  File "/Users/julien/code/spine-generic/spine-generic/spinegeneric/cli/generate_figure.py", line 555, in generate_figure_t1_t2
    label=vendor)
  File "/Users/julien/miniconda3/lib/python3.7/site-packages/matplotlib/pyplot.py", line 2841, in scatter
    None else {}), **kwargs)
  File "/Users/julien/miniconda3/lib/python3.7/site-packages/matplotlib/__init__.py", line 1599, in inner
    return func(ax, *map(sanitize_sequence, args), **kwargs)
  File "/Users/julien/miniconda3/lib/python3.7/site-packages/matplotlib/axes/_axes.py", line 4443, in scatter
    raise ValueError("x and y must be the same size")
ValueError: x and y must be the same size

An elegant solution would be to ignore the subject both for T1w and T2w, in case it is excluded in one of them.
~Another (quick) solution is to ignore the excluded subjects.~
Update: oopsi! i tried running the code without the -exclude flag and the same error happens 😞

Update: oopsi! i tried running the code without the -exclude flag and the same error happens 😞

actually, this was caused by corrupted csv files with duplicated subjects (process run several times):
Archive.zip
so, ignoring excluded subjects should work.

I have just tried to generate this figure with previous release of results (r20200830) and it fails on the same error.
When I go deeper inside, both csa-SC_T1w.csv and csa-SC_T2w.csv have same number of lines, i.e., same number sites. But when I do not use any exclusion, for Siemens I have totally 173 values (i.e., 173 subjects) for t1 (np.concatenate(CSA_dict[vendor + '_t1']), but for t2 I have 172 values (i.e., 172 subjects) (np.concatenate(CSA_dict[vendor + '_t2']). This causes discrepancy for length of x and y axis.
https://github.com/spine-generic/spine-generic/blob/4143cbfd7a206f06e3fd9f42485d5087129899d7/spinegeneric/cli/generate_figure.py#L549-L550
How is possible that Siemens has for t1 different number of subjects than for t2 when I do not use any exlude.yml?

How is possible that Siemens has for t1 different number of subjects than for t2 when I do not use any exlude.yml?

which site is missing?

Missing is sub-mountSinai03. The reason is because this site has in csa-SC_T2w.csv file value None:

It is reason why this site is listed in exclude.yml:
https://github.com/spine-generic/data-multi-subject/blob/6bda590b10ec08315265f40ca0eda6b9e9c7eb04/exclude.yml#L13
Current implementation of generate_figure.py obviously discards None and without using exclude.yml, discrepancy in length of t1 and t2 (i.e., x and y axis) happens.

An elegant solution would be to ignore the subject both for T1w and T2w, in case it is excluded in one of them.

This would be really elegant solution. I can implement it. Unfortunately, I will have time for it after weekend. Until that, exclude.yml file has to include subject under both csa_t1 and csa_t2 flags even if the problem is only in one contrast.


Add Hash and Eq to rdev::Key (this allows Key to be used as a key in a HashMap)

Already done: https://github.com/Narsil/rdev/commit/b2400b74b092de086807e6bf78c2a8b2da73de6a


[BUG] ARF predict_proba failing if first call to partial fit does not specify classes

Describe the bug
If in the first call to partial_fit in ARF we do not specify the classes, predict_proba might fail.
There are two possible errors that seem to have the same root.

It seems that predict_proba expects every array of votes to be of the same size, but that might not be true when the classes are not specified.

The error occurs in this line of predict_proba of ARF:
y_proba_mean = y_proba_mean + (y_proba - y_proba_mean) / (i+1)
And the error is:
ValueError: operands could not be broadcast together with shapes (100,) (100,2)

The second error also happens only when we do not specify the classes parameter in the first call to partial_fit. I did not completely understand what is happening, but the error is the following:

ValueError: setting an array element with a sequence.
And it happens when converting a variable to numpy array in the method check_array of the file utils/validation.py.
This method is called by normalize in the file preprocessing/data.py. And normalizeis called by ARF predict_proba.
To Reproduce
This code should reproduce the first error:
stream.prepare_for_use()
model_orig = AdaptiveRandomForest()
x_train, y_train = stream.next_sample(100)
model_orig.partial_fit(x_train, y_train)
print(model_orig.predict_proba(x_train))

And the second one:
stream = AGRAWALGenerator()
stream.prepare_for_use()
model_orig = AdaptiveRandomForest()
x_train, y_train = stream.next_sample(1000)
model_orig.partial_fit(x_train, y_train)
print(model_orig.predict_proba(x_train[:100]))

Both can also be seen here: https://colab.research.google.com/drive/1iRT2r9h3emgvEBtdkntjgWPHcEbGgBn9
Expected behavior
I think one of the possible ways to correct that is to add in ARF partial_fit, the same verification that we have on OnlineBoosting:
if self.classes is None:
    if classes is None:
        raise ValueError("The first partial_fit call should pass all the classes.")
    else:
        self.classes = classes

if self.classes is not None and classes is not None:
    if set(self.classes) == set(classes):
        pass
    else:
        raise ValueError("The classes passed to the partial_fit function differ from those passed earlier.")

But I'm not sure how that would affect other people's code.

Hi @luccaportes
This is indeed a bug. The root cause is that the implementation of predict_proba is not handling correctly the use of mini-batches. There are other details missing as well. The following fix deals with most of them:
    def predict_proba(self, X):
        if self.ensemble is None:
            self.init_ensemble(X)

        y_proba = []
        y_proba_mean = None
        n_rows, _ = get_dimensions(X)

        for i in range(n_rows):
            for j in range(self.n_estimators):
                y_proba_current = self.ensemble[j].predict_proba(np.asarray([X[i]]))
                if y_proba_mean is None:
                    y_proba_mean = y_proba_current
                else:
                    y_proba_mean = y_proba_mean + (y_proba_current - y_proba_mean) / (j + 1)
            y_proba.append(normalize(y_proba_mean, norm='l1').flatten())

        return np.asarray(y_proba)

I will do further tests and try to get a more generic approach so predict_proba can be called from predict.
I made some small changes to your MWE for testing:
from skmultiflow.data import AGRAWALGenerator
from skmultiflow.meta import AdaptiveRandomForest

stream = AGRAWALGenerator(random_state=1)
stream.prepare_for_use()
model_orig = AdaptiveRandomForest(random_state=1)

x_train, y_train = stream.next_sample(100)
model_orig.partial_fit(x_train, y_train)
x_test, y_test = stream.next_sample(10)
print(model_orig.predict_proba(x_test))

stream.restart()
model_orig.reset()

x_train, y_train = stream.next_sample(100)
model_orig.partial_fit(x_train, y_train)
x_test, y_test = stream.next_sample(100)
print(model_orig.predict_proba(x_test[:10]))

Upon further investigation, I found that the bug is indeed triggered only when classes are not defined in the first call of partial_fit.
The root cause is the way we return probabilities from the base estimators. When no classes are defined, the resulting array is missing some values until all classes are observed. This caused the array to be a 1D array of lists with different lengths instead of a 2D array.
The current implementation of predict_proba in ARF expects a 2D array but receives pseudo-1D arrays, thus raising the errors. I am uploading a fix for this corner-case as well as a streamlined version of predict based on predict_proba.


Just an idea: kool.yml on user level

Hey, guys! How about looking for kool scripts inside a user's home directory/subdirectory? Is it possible?

Awesine idea! so we can have global scripts available. Questions that arise:

Should we make kool run look for scripts on both local and user environment folders? What should we do if a same script exists in both? have precedence or explode an error?

Other questions?

I guess we should prioritize the project script. But it would be nice to warn the user that the project script was executed instead of his.

This has been implemented and merged onto master! Will be in the next release.


l_pos and r_pos empty in nt_xent loss?

When I run MolCLR, an error seems to stem from the following lines in the forward function:
>>> l_pos = torch.diag(similarity_matrix, self.batch_size)
>>> r_pos = torch.diag(similarity_matrix, -self.batch_size)
>>> positives = torch.cat([l_pos, r_pos]).view(2 * self.batch_size, 1)
RuntimeError: shape '[1024, 1]' is invalid for input of size 0

The similarity matrix appears to be calculated correctly, but l_pos and r_pos are empty tensors when printed out. Would appreciate any guidance here.

Update: Never mind, had the wrong batch size in config.


How can I change instruct-pix2pix to another  image-edited model?

This is a great work. It combines the instruct-pix2pix and NeRF models very well to achieve a very good effect. I noticed another work called HIVE, which is an improved version based on instruct-pix2pix. It adds RLHF on the basis of instruct-pix2pix, and the basic model structure remains unchanged. Now I want to try to replace the instruct-pix2pix component in instruct-nerf2nerf with HIVE to exercise my programming skills. However, since the HIVE model has not been released on Hugging Face like instruct-pix2pix, the method of loading the model needs to be changed. I have tried many methods, but none of them have been successful. This is very frustrating for me. Could you give some feasible suggestions based on your professional knowledge? Thank you very much.

Will close this issue as there hasn't been a response in a while.


sc-7855 Refactor Overview module component to reach at least 60%

Scope of changes
sc-7855 Refactor Overview module component to reach at least 60%
Type of change

[ ] bug fix
[ ] new feature
[ ] documentation
[x] test

Acceptance criteria
Describe how reviewers can test this change to be sure that it works correctly. Add a checklist if possible
Author checklist

[] I have manually tested the change and/or added automation in the form of unit tests or integration tests
[ ]  I have updated the dependencies list
[ ]  I have recompiled and included new protocol buffers to reflect changes I made
[ ]  I have added new test fixtures as needed to support added tests
[ ]   Check this box if a reviewer can merge this pull request after approval (leave it unchecked if you want to do it yourself)
[ ]  I have moved the associated Shortcut story to "Ready for Review"

Reviewer(s) checklist

[ ] Any new user-facing content that has been added for this PR has been QA'ed to ensure correct grammar, spelling, and understandability.

Codecov Report

Merging #697 (6e89391) into main (8c2a659) will decrease coverage by 13.99%.
The diff coverage is 78.78%.

:exclamation: Current head 6e89391 differs from pull request most recent head 1a71017. Consider uploading reports for the commit 1a71017 to get more accurate results

@@             Coverage Diff             @@
##             main     #697       +/-   ##
===========================================
- Coverage   44.04%   30.04%   -14.00%
===========================================
  Files         575      316      -259
  Lines       17181     2786    -14395
  Branches     1247      628      -619
===========================================
- Hits         7567      837     -6730
+ Misses       8430     1937     -6493
+ Partials     1184       12     -1172

Impacted Files
Coverage Δ

...-ui/src/components/CertificateManagement/index.tsx
0.00% <ø> (ø)

web/gds-user-ui/src/components/Metrics/index.tsx
33.33% <0.00%> (ø)

...src/components/NetworkAnnouncements/caroussels.tsx
7.69% <ø> (ø)

...r-ui/src/components/NetworkAnnouncements/index.tsx
50.00% <ø> (-16.67%)
:arrow_down:

.../gds-user-ui/src/components/NetworkMetricsTabs.tsx
0.00% <0.00%> (ø)

...er-ui/src/components/StatCard/StatCard.stories.tsx
0.00% <0.00%> (ø)

...ser-ui/src/components/TechnicalResources/index.tsx
0.00% <0.00%> (ø)

...ules/dashboard/certificate/lib/validationSchema.ts
11.76% <ø> (+0.65%)
:arrow_up:

...s-user-ui/src/modules/dashboard/overview/index.tsx
56.25% <0.00%> (+53.12%)
:arrow_up:

...ds-user-ui/src/components/NeedsAttention/index.tsx
100.00% <100.00%> (+50.00%)
:arrow_up:

... and 276 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Redundancy example in Lecture 1.pdf

In Lecture 1.pdf, on the 6th page, the slide talks about redundancy in columns DNAME and DMGRSSN, but I've noticed that DNUMBER is also redundantly corresponding to DNAME and DMGRSSN.

Greetings.

We emphasised here just one example for the redundancy. But department name can also be considered redundant, since it's number is redundant.


Kindle Cloud Reader broken

When Saka Key is enabled, the Kindle Cloud Reader never finishes loading ebooks, and the following error is raised:
TypeError: y.grid_style_container is undefined[Learn More] KindleLibrary-min.js:273:493.
Disabling the extension allows the reader to work again.

I also have this problem, but only with Firefox (67.0.1). I just tried it on Chrome and some Chrome-based browsers (Opera, Brave, and Vivaldi) and they all load. Disabling Saka Key within its interface doesn't work, only disabling the Add-On from the Firefox interface seems to help.
My current workaround is to disallow it in Private Windows and open one when I use the cloud reader (or use Brave).


Upgrade to Babel 6

https://babeljs.io/blog/2015/10/29/6.0.0/
We need to enforce Babel 6 (for devs wanting to use local version) as it's incompatible with 5.

I think we should defer this for a couple of weeks until Babel 6 settles a bit. I see a lot of issues on the Babel issue tracker that might affect us.

Just released babel-plugin-espower 2.0.0. Now power-assert works with Babel6!

This was resolved in v0.9.0.


Failures on Add tests for behavior of class modifiers through legacy libraries.

There are new test failures on Add tests for behavior of class modifiers through legacy libraries..
The tests
language/class_modifiers/trans_legacy/legacy_superdeclaration_test CompileTimeError (expected Pass)
language/class_modifiers/trans_legacy/legacy_superdeclaration_test Crash (expected Pass)
language/class_modifiers/trans_legacy/legacy_superdeclaration_test DartkCrash (expected Pass)

are failing on configurations
cfe-strong-linux
cfe-weak-linux
vm-mac-debug-arm64
vm-mac-release-arm64

The front end crashes on the non-error test, because a type parameter of a generic interface is "not indexed":
/==================================================================================================================\
| language/class_modifiers/trans_legacy/legacy_superdeclaration_test is new and failed (DartkCrash, expected Pass) |
\==================================================================================================================/

--- Command "vm" (took 02.000492s):
DART_CONFIGURATION=DebugARM64 xcodebuild/DebugARM64/dart --sound-null-safety -Dtest_runner.configuration=vm-mac-debug-arm64 --enable-experiment=class-modifiers --ignore-unrecognized-flags --packages=/opt/s/w/ir/.dart_tool/package_config.json /opt/s/w/ir/tests/language/class_modifiers/trans_legacy/legacy_superdeclaration_test.dart

exit code:
252

stderr:
Invalid argument(s): Type parameter TypeParameter(Sink.T) is not indexed
#0      TypeParameterIndexer.[] (package:kernel/binary/ast_to_binary.dart:3425:8)
#1      BinaryPrinter.visitTypeParameterType (package:kernel/binary/ast_to_binary.dart:2576:38)
...

or is referenced out of scope
/============================================================================================================================\
| cfe-weak-linux:language/class_modifiers/trans_legacy/legacy_superdeclaration_test is new and failed (Crash, expected Pass) |
\============================================================================================================================/

--- Command "fasta" (took 173ms):
(cd /b/s/w/ir/cache/builder/sdk/ ; DART_CONFIGURATION=ReleaseX64 ./out/ReleaseX64/dart ./pkg/front_end/tool/_fasta/compile.dart --verify --skip-platform-verification -o ./out/ReleaseX64/generated_compilations/cfe-weak-linux/tests_language_class_modifiers_trans_legacy_legacy_superdeclaration_test/out.dill --platform ./out/ReleaseX64/vm_platform_strong.dill -Dtest_runner.configuration=cfe-weak-linux --enable-experiment=class-modifiers --packages=./.dart_tool/package_config.json ./tests/language/class_modifiers/trans_legacy/legacy_superdeclaration_test.dart )

exit code:
253

stdout:
sdk/lib/core/sink.dart:17:14: Error: Verification of the generated program failed:
Target=vm, VerificationStage.afterModularTransformations: Type parameter 'TypeParameter(Sink.T)' referenced out of scope, owner is: 'Class(Sink)'.
  void add(T data);
             ^
sdk/lib/core/sink.dart:17:14: Error: Verification of the generated program failed:
Target=vm, VerificationStage.afterModularTransformations: Type parameter 'TypeParameter(Sink.T)' referenced out of scope, owner is: 'Class(Sink)'.
  void add(T data);
             ^
sdk/lib/async/stream.dart:2225:30: Error: Verification of the generated program failed:
Target=vm, VerificationStage.afterModularTransformations: Type parameter 'TypeParameter(StreamConsumer.S)' referenced out of scope, owner is: 'Class(StreamConsumer)'.
  Future addStream(Stream<S> stream);
                             ^
sdk/lib/async/stream.dart:2225:30: Error: Verification of the generated program failed:
Target=vm, VerificationStage.afterModularTransformations: Type parameter 'TypeParameter(StreamConsumer.S)' referenced out of scope, owner is: 'Class(StreamConsumer)'.
  Future addStream(Stream<S> stream);
                             ^

stderr:
Unhandled exception:
  Invalid argument(s): Type parameter TypeParameter(Sink.T) is not indexed
#0      TypeParameterIndexer.[] (package:kernel/binary/ast_to_binary.dart:3425:8)
#1      BinaryPrinter.visitTypeParameterType (package:kernel/binary/ast_to_binary.dart:2576:38)
...

Smaller reproduction found. Does not appear to be related to class modifiers at all.
// lib1.dart
abstract mixin class MixesInMixin = Object with Iterable<int>;

and
// lib2.dart
import "lib1.dart";

abstract base class MixesInMixesInMixin extends Object with MixesInMixin {}

void main() {}

Run dart lib2.dart and it crashes with
Invalid argument(s): Type parameter TypeParameter(Iterable.E) is not indexed
#0      TypeParameterIndexer.[] (package:kernel/binary/ast_to_binary.dart:3425:8)
#1      BinaryPrinter.visitTypeParameterType (package:kernel/binary/ast_to_binary.dart:2576:38)

Run pkg/front_end/tool/_fasta/compile.dart and it gives a much longer error:
sdk/lib/core/iterable.dart:169:27: Error: Verification of the generated program failed:
Target=vm, VerificationStage.afterModularTransformations: Type parameter 'TypeParameter(Iterable.E)' referenced out of scope, owner is: 'Class(Iterable)'.
  Iterator<E> get iterator;
                          ^
sdk/lib/core/iterable.dart: Error: Verification of the generated program failed:
Target=vm, VerificationStage.afterModularTransformations: Type parameter 'TypeParameter(Iterable.E)' referenced out of scope, owner is: 'Class(Iterable)'.
sdk/lib/core/iterable.dart:193:38: Error: Verification of the generated program failed:
Target=vm, VerificationStage.afterModularTransformations: Type parameter 'TypeParameter(Iterable.E)' referenced out of scope, owner is: 'Class(Iterable)'.
  Iterable<E> followedBy(Iterable<E> other) {
                                     ^
...

and so on for every Iterable member.
Switching both libraries to<EMAIL_ADDRESS>and removing the base and mixin keywords yields the same errors.
Putting both classes in the same library removes the error.

Putting both classes in the same library removes the error.

Thank you for the analysis, @lrhn! That observation was especially helpful. Turned out, the full mixin resolution transformation in the VM didn't account for the potential dependency of one mixin application to another, as it did so for a mixin application and its supertype. I added the missing bit in this CL: https://dart-review.googlesource.com/c/sdk/+/305320


Commits since nov 11,  does not work in Windows 7 with 32 bits

Hello, your last package compiles well, generates the .a file and my program that imports your packate compiles well also, but when executing, it hangs and produces the increment of CPU and I/O.
The connection to Database seems working well:
db, err := sql.Open("oci8", connection)
But it hangs when invoking to:
err = db.Ping()
I recovered one of the commits previous to november 11 and all is working as usual (very well), so it seems that the changes made to your code in nov 11 are the origin of the described problem.
Best regards.

Should be fixed


GOSA17 - Make User login in firebase

Connect user login UI to firebase and connect the Register Here to registration UI by using pushNamed (use the static variable routeName  in the class).

assign me

assign me

@rithik-dev work on this

@rithik-dev Your Pull request is successfully merged. Thank you for the contribution and you have recieved 10 points for the contribution


Refactoring tests

I've refactored some tests with assertFileExists and assertFileNotExists, assertFalse and assertNull methods.

@carusogabriel Thanks for looking into this!
@WyriHaximus Not sure what's going on here, all tests are failing due to timeouts?

@clue not sure yet, looking into them now. But aside from that these changes LGTM 👍

RTM? :)


Merge release20.11-SNAPSHOT to release20.11

Co-authored-by: labkey-jeckels
Rationale
Merge for 20.11.5

Hmm, I thought this change was merged for 20.11.4


InfluxDB goes unresponsive

Bug report
System info: [Include InfluxDB version, operating system name, and other relevant details]
Version: 0b4528b
OS: FreeBSD
Steps to reproduce:

???

Expected behavior: [What you expected to happen]
Keep working
Actual behavior: [What actually happened]
Stops responding. Process still running, but no longer responds to any queries.
Additional info: [Include gist of relevant config, logs, etc.]
Happens at random, but it seems the more activity, the more likely the issue will occur. Haven't identified a pattern yet.
Here's the output of SIGQUIT.
https://gist.githubusercontent.com/phemmer/251eab87914681d30f0e0435c664e9f7/raw/e79a847b896a7533586f6b384bd2aeb4c4c98083/log

As an additional note, it seems that when this happens, and I restart InfluxDB, one of my measurements will become corrupt. Any attempt to insert into the measurement returns something like [shard 17] field type conflict. And even when I drop the measurement and try to recreate it, the error still occurs.

It just happened again. Working fine all day, and then just a minute ago it went unresponsive. The only thing I was doing (in addition to what's been going on all day long) was that I was skipping backwards in time a lot in grafana (the little left arrow thing next to the time selector in the top).
Here's another SIGQUIT dump: https://gist.githubusercontent.com/phemmer/251eab87914681d30f0e0435c664e9f7/raw/a63f6cdf6c54d89771630764a13d42aa07318f88/log2
Note however that this time I was running InfluxDB on a completely different host and OS. This time it's on Linux, and I was running a7ca7b5. I also had a completely brand new database as the crash last night completely corrupted my old one and I couldn't recover it.

@phemmer Are you seeing any panics in your logs?

no

@phemmer Could you try #8518 to see if that helps?

Perhaps. I'll see if I can reproduce it in an isolated environment. I rolled back to v1.2 because the issue was happening every few minutes, and was constantly corrupting my database. Though given how frequently it was occurring, it shouldn't be hard.

I could drop the entire measurement drop measurement foo and the issue would still occur. show measurements would indicate that it's not there, yet I would still get that error trying to recreate it.

I just upgraded my system to 1.3.0 (downloaded from https://dl.influxdata.com/influxdb/releases/influxdb-1.3.0.x86_64.rpm)
and then ran into troubles.  Things run fine for ~1 min, then the whole system locks up and dies.  I have to shutdown the aws instance and bring it back up to get in.  I can't find any logging that clearly says what happened -- but can try to dig further if this is helpful.
I then tried upgrading to the nightly 1.4.0~n201706240800 / e014dd012d9a66cce540f4a021ee0a4a177960a7  and I get the same behavior.
I then put back the 1.2.4 and things are OK again
sudo rpm -Uvh --oldpackage influxdb-1.2.4.x86_64.rpm
I don't know if this is related...   let me know if there is any more info you want (and where to find it!)

@ryantxu it looks like 1.4.0~n201706240800 would contain the suggested possible fix in https://github.com/influxdata/influxdb/issues/8518, which means that #8518 probably hasn't done the trick.
Does the server only lock up under write load or can you literally start it up and leave it to lock up without any writes or queries going in?

@phemmer did you have the same problems when trying #8518 or did you not get around to that?

The setup has active writes and continuous queries running.  It would take some work to test without that... but if useful, let me know.

@phemmer did you have the same problems when trying #8518 or did you not get around to that?

I have not had a chance. I'm not keen on doing this with my main InfluxDB and corrupting my DB again, so I need to duplicate the server and all the clients, which will take some work.

@ryantxu I'm trying to think of a way for us to be able to locally reproduce your situations so we can track down the deadlock. I figured as much that writes/queries were needed. If not, we probably would have seen this within our tests. @phemmer are you running CQs by chance?

No CQs

@phemmer @ryantxu I just realised I didn't ask—do you know if profile endpoints are responding on 8086? A blocking profile could be useful.. curl -o blocking.txt "http://localhost:8086/debug/pprof/block?debug=1"
@ryantxu if you're using 1.3 onwards then the new archive profile endpoint would be available to you: curl -o profiles.tar.gz "http://localhost:8086/debug/pprof/all"

I must say that as an InfluxDB user, the handling of this issue is concerning. With this issue, and #8533, we now have several people confirming the issue.
I created this issue well before InfluxDB 1.3.0 was released. An issue where at the very least InfluxDB becomes unresponsive, and at the worse results in DB corruption. How did a stable release get out without this issue being investigated and addressed? What is the purpose of the release candidates if they're not used to validate stability before general release?

@phemmer I can assure you that we're looking into this. We had hoped that #8518 would fix the issue, but it appears not according to @ryantxu's testing of the 1.4.0~n201706240800 nightly. You mention that the server goes unresponsive but I just wanted to get a couple of clear clarifications to help me try and isolate the issue:

does the entire 8086 write/query endpoint become unresponsive, or is it just related to the database in question?
if it's not the entire port, are you able to get profiles when the database does become unresponsive? As I mentioned previously, a blocking profile may provide some insight.

I understand we're working on it now, and I appreciate that, but how come it wasn't worked on until after 1.3.0 was released? That's my concern. As a user, I should be comfortable using stable releases.
I only had 1 database, so I was unable to determine if other databases were affected.
Other operations were responsive, such as I can confirm the ping endpoint was responsive.
I'm working on reproducing this in a lab right now. Hopefully I'll be able to do so.

@phemmer Are you able to share your log files related to the hang?

@phemmer We are running about 100 production instances of 1.3.0 and have been running 1.3.0 shadows internally for weeks; we have never reproduced this error. We are giving this issue a very high priority - it is blocking our 1.3 GA build and announcement as it stands. I apologize for the frustration it has caused you.

When I last tried, i was in a bit of panic mode trying to get things back to working :)
If you guys are going to look at it, i can get things to reproduce again -- if you want, i can give you direct access if that helps your debugging

@ryantxu I just merged in a potential fix here: https://github.com/influxdata/influxdb/pull/8547
Are you able to pull that down and build it? I'd be interested to know if you can reproduce with 1.3 but not that branch.

is there an rpm i can install with it?  or do i need to wait for the nightly to get built and try tomorrow?

testing now with
"2f1134708925fd54a36b45feeaf48553cdc274e3",
"1.4.0~n201706290800"
Still freezes after a few mins -- this time, the last few lines of the logs before crash say something interesting:
[I] 2017-06-29T17:57:12Z Snapshot for path /mnt/db/data/_internal/monitor/14227 written in 1.295712536s engine=tsm1
[I] 2017-06-29T17:57:13Z beginning level 1 compaction of group 0, 2 TSM files engine=tsm1
[I] 2017-06-29T17:57:13Z compacting level 1 group (0) /mnt/db/data/_internal/monitor/14227/000000915-000000001.tsm (#0) engine=tsm1
[I] 2017-06-29T17:57:13Z compacting level 1 group (0) /mnt/db/data/_internal/monitor/14227/000000916-000000001.tsm (#1) engine=tsm1
[I] 2017-06-29T17:57:16Z compacted level 1 group (0) into /mnt/db/data/_internal/monitor/14227/000000916-000000002.tsm.tmp (#0) engine=tsm1
[I] 2017-06-29T17:57:16Z compacted level 1 2 files into 1 files in 2.698048352s engine=tsm1
[I] 2017-06-29T17:57:16Z beginning level 2 compaction of group 0, 2 TSM files engine=tsm1
[I] 2017-06-29T17:57:16Z compacting level 2 group (0) /mnt/db/data/_internal/monitor/14227/000000913-000000002.tsm (#0) engine=tsm1
[I] 2017-06-29T17:57:16Z compacting level 2 group (0) /mnt/db/data/_internal/monitor/14227/000000916-000000002.tsm (#1) engine=tsm1

I was wondering if it might be compaction related. This might help explain why my db was getting corrupted.

I don't think my db is getting corrupted -- and the downgrade back to 1.2.4 seems to work without needing to reload the data.  My problem is that the whole system freezes and i need to reboot the EC2 instance to get back on

Yeah, there might be something else going on in my environment that in combination triggers the corruption. Something like a new series being created, measurement being dropped & re-created, etc.

I'm trying to get the output of blocking.txt and profiles.tar.gz for you...
This latest reboot and restart never got to a point I can query :(  The system freezes after:
[I] 2017-06-29T18:13:09Z /mnt/db/data/usgs/autogen/12701 opened in 11.167452ms service=store
[I] 2017-06-29T18:13:09Z opened service service=subscriber
[I] 2017-06-29T18:13:09Z Starting monitor system service=monitor
[I] 2017-06-29T18:13:09Z 'build' registered for diagnostics monitoring service=monitor
[I] 2017-06-29T18:13:09Z 'runtime' registered for diagnostics monitoring service=monitor
[I] 2017-06-29T18:13:09Z 'network' registered for diagnostics monitoring service=monitor
[I] 2017-06-29T18:13:09Z 'system' registered for diagnostics monitoring service=monitor
[I] 2017-06-29T18:13:09Z Starting precreation service with check interval of 10m0s, advance period of 30m0s service=shard-precreation
[I] 2017-06-29T18:13:09Z Starting snapshot service service=snapshot
[I] 2017-06-29T18:13:09Z Starting continuous query service service=continuous_querier
[I] 2017-06-29T18:13:09Z Starting HTTP service service=httpd
[I] 2017-06-29T18:13:09Z Authentication enabled:false service=httpd
[I] 2017-06-29T18:13:09Z Listening on HTTP:[::]:8086 service=httpd
[I] 2017-06-29T18:13:09Z Starting retention policy enforcement service with check interval of 30m0s service=retention
[I] 2017-06-29T18:13:09Z Listening for signals
[I] 2017-06-29T18:13:09Z Storing statistics in database '_internal' retention policy 'monitor', at interval 10s service=monitor
[I] 2017-06-29T18:13:09Z Sending usage statistics to usage.influxdata.com
[I] 2017-06-29T18:13:10Z beginning full compaction of group 0, 2 TSM files engine=tsm1
[I] 2017-06-29T18:13:10Z compacting full group (0) /mnt/db/data/_internal/monitor/14227/000000897-000000005.tsm (#0) engine=tsm1
[I] 2017-06-29T18:13:10Z compacting full group (0) /mnt/db/data/_internal/monitor/14227/000000916-000000004.tsm (#1) engine=tsm1

@ryantxu could you try SIGQUITing the process when it freezes?
$ kill -s SIGQUIT <pid of influxd>
Or, if you're running influxd in the foreground just hit ctrl \

@ryantxu could you try SIGQUITing the process when it freezes?

When it freezes, i can't type in my terminal window.  I can't reconnect via ssh, the only thing i can do is reboot the ec2 instance!

I am unable to login to the ec2 instance now...  i will need to restore and try again.  Maybe this afternoon, but more likely tomorrow morning.  Let me know if there is an .rpm i should try.

@ryantxu maybe the system is becoming unhealthy due to limited RAM or other resources?  Have you tried keeping a second terminal open tailing kern.log or running top?

Yes, it appears that 1.4.0~n201706290800 pushes this machine outside its memory
top - 22:48:27 up  3:41,  2 users,  load average: 2.61, 0.77, 0.28
Tasks: 108 total,   2 running, 106 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.1%us,  0.6%sy,  0.0%ni, 14.1%id, 85.1%wa,  0.0%hi,  0.0%si,  0.1%st
Mem:   7660676k total,  7544632k used,   116044k free,      192k buffers
Swap:        0k total,        0k used,        0k free,     2616k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
  658 root      20   0     0    0    0 R  7.2  0.0   1:10.62 kswapd0
 3361 influxdb  20   0 31.5g 7.1g    0 D  6.7 96.6   1:36.44 influxd
 2645 grafana   20   0  716m  10m    0 S  2.0  0.1   3:41.31 grafana-server
 3304 ec2-user  20   0 15308  524  204 R  0.9  0.0   0:00.27 top
    1 root      20   0 19636  316    0 S  0.0  0.0   0:01.66 init
    2 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kthreadd
    3 root      20   0     0    0    0 S  0.0  0.0   0:00.02 ksoftirqd/0

When I revert back to 1.2.4, things run fine and look like this:
top - 23:28:38 up 33 min,  2 users,  load average: 2.59, 2.56, 5.72
Tasks: 107 total,   1 running, 106 sleeping,   0 stopped,   0 zombie
Cpu(s): 45.7%us,  0.1%sy,  0.0%ni, 53.9%id,  0.0%wa,  0.0%hi,  0.0%si,  0.3%st
Mem:   7660676k total,  4187248k used,  3473428k free,    61340k buffers
Swap:        0k total,        0k used,        0k free,  1594792k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 2966 influxdb  20   0 25.2g 3.5g 1.2g S 200.1 47.4   5:00.83 influxd
 2380 root      20   0  4372   88    0 S  0.3  0.0   0:01.14 rngd
    1 root      20   0 19632  824  508 S  0.0  0.0   0:01.00 init
    2 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kthreadd
    3 root      20   0     0    0    0 S  0.0  0.0   0:00.00 ksoftirqd/0
    4 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kworker/0:0
    5 root       0 -20     0    0    0 S  0.0  0.0   0:00.00 kworker/0:0H

So i guess this is really a memory use issue.  Perhaps it is getting stuck on a one time upgrade?  Or do you think the memory requirements really changed that much between 1.2.4 and 1.3?
this is on a c3.xlarge/us-east-1a

I'm still unable to reproduce the hanging issue. But I still haven't reproduced the original environment exactly, just mostly :-/
I am however seeing a plethora of other issues (won't go into right now). But I can confirm the memory thing @ryantxu is seeing. I have a test box where InfluxDB is completely idle (no queries, and no writes), and I can watch the memory usage go from ~1gb up to ~7gb, while using around 100% CPU. It plateaus at ~7gb, and sits there for for a few minutes, still using 100% CPU, and then drops back down to ~1gb and starts the process over again.

Lol, I dropped a measurement that I didn't need any more to try and remove some data from InfluxDB to relieve some stress from it, and my disk usage went up. Wut?
Before:
# df -h .
Filesystem                Size  Used Avail Use% Mounted on
/dev/mapper/sys-influxdb   30G   16G   13G  57% /mnt/influxdb

After:
# df -h .
Filesystem                Size  Used Avail Use% Mounted on
/dev/mapper/sys-influxdb   30G   23G  5.8G  80% /mnt/influxdb

@phemmer can you open another issue with the .tmp compaction problems? This is something that we identified internally on Friday but have not yet found the root cause of. I would love to hear more about the data you have in this instance and any other information you can provide.

So the hanging thing has started happening for me again.
Version e3918e0
SIGQUIT: https://gist.githubusercontent.com/phemmer/251eab87914681d30f0e0435c664e9f7/raw/f4d6a9d33a680bd7ab3eef2dcbf13d8df9ca0135/log5 (unfortunately truncated at ~10k lines due to logging configuration issue)
curl /debug/pprof/block: https://gist.github.com/phemmer/251eab87914681d30f0e0435c664e9f7/raw/f4d6a9d33a680bd7ab3eef2dcbf13d8df9ca0135/log5-blocking
Version 8550fab
SIGQUIT https://gist.github.com/phemmer/251eab87914681d30f0e0435c664e9f7/raw/52859f853ca16ae42bcbd45b789eb86b56625e16/log6
curl /debug/pprof/block: https://gist.github.com/phemmer/251eab87914681d30f0e0435c664e9f7/raw/52859f853ca16ae42bcbd45b789eb86b56625e16/log6-blocking
There are no panics in the log prior to this happening. Completely normal POST calls with 204 responses, and then 500s & timeouts.

@phemmer syncErr is sent to the syncWaiters channel and then written to here
However, if the syncWaiters channel is full (1024), it will fail, but the closure will return an error and thus return early here

Ah, I misread line 424. I skimmed the code and read it as a read from the chan, not sending the chan to another chan o.O. Sorry bout that :-)
Something is causing it to block though.

@phemmer yes something is up. I'm looking at it as a priority.

@phemmer by any chance do you have the server logs from either of the above two SIGQUITs?

Unfortunately not as the logs have already rotated out (it's a personal box, and I don't permanently archive the logs).
I can however tell you that for a long time prior, and after the issue started, it was nothing but writes with 204 responses, and then writes with 500 responses. There weren't even any queries being executed around the time. It's possible there might have been something way in the past (several minutes before the issue started), but nothing near the time of the event.

OK no worries. Was the box under significant write load? Roughly how many writes per second do you think? And typically what would your batch sizes be?

Average WPS is around 6. The writer flushes every 100ms, and when its buffer is full. Normal batch size is pretty small, probably less than 20 points. However once every 60 seconds a batch (or multiples if it can't fit in the write buffer, buffer size is 512KB) will come through that is about 4k points.

@phemmer What kind of disks do you have?  SSDs? HDD?  Can you run iostat -xdz 1 30 when you are getting the timeouts?
Would you be able to test #8567, it may fix the timeouts.
For the higher memory usage, if you can grab a memory profile via:
curl -o heap.txt "http://localhost:8086/debug/pprof/heap?debug=1"

that might help identify what may be consuming more memory.

Hybrid disks.
I don't think this is a disk issue. When it happens, influxdb stays stuck until I restart it. It does not recover own its own.
I'll see about reproducing this again (I downgraded back to 1.2.4 which does not have the issue), but probably not tonight. Maybe tomorrow.

Can you elaborate on "Hybrid disks"?  What are the disks and how are they setup?  Are they RAIDed? Directly attached?

Each disk is a combination of SSD & spinning. There's 2 of them in raid-1. Yes, direct attach.

@phemmer could you clarify something for us? Throughout all of the this ticket have you been running InfluxDB on the same HW (with the raid-1 hybrid drives), and could you also confirm:

if you've been building influxdb or using nightlies/releases.
if you have been building influxdb: with what version of Go?

I just want to get a better understanding of your environment. Cheers.

Yes, same hardware.
I've been building myself. Go 1.8.1.

#8577 should fix the case where writer is block indefinitely on the syncErr channel.

@phemmer All of the pending fixes are on the 1.3 branch and master now.  Would you be able to test off of 1.3 and see if the issue is resolved?

@stuartcarnie and I have found the cause for the initial deadlock on Measurement reported in the issue description.  We'll get a fix up for that as well.

Would you be able to test off of 1.3 and see if the issue is resolved?

I might be able to tomorrow night. I'll be intermittently available to test things as starting yesterday I'm on vacation through next week. But that one should be really easy to test. It generally occurred within a few minutes of starting InfluxDB.

stuartcarnie and I have found the cause for the initial deadlock on Measurement reported in the issue description. We'll get a fix up for that as well.

🎆  I'm eager to see that PR, as when I skimmed the code, I couldn't see any obvious explanation for it. Everywhere I could see a lock being taken, all code paths released it.

It's here:  https://github.com/influxdata/influxdb/blob/master/tsdb/index/inmem/meta.go#L333
The RLock is not released.

Ahha. And that makes sense why I'm able to trigger the issue so easily. I have InfluxDB hooked up to splunk, so splunk can query InfluxDB directly. Splunk is quite font of killing searches while they're in progress. I didn't even think of that atypical behavior.


remove con_outputs fro model_io

should we get rid of con_outputs from model IO? it is only used for OCP, and really defined by the OCP, and selected from the existing model outputs. So a model developer really isn't aware of this!
What makes things a bit more complicated is that for efficiency, the compiled model will only have this 'con_outputs' as outputs, and with derivatives taken on it. So for the compiled model, it does need to be aware of con_outputs!
Some guiding principles:

someone who writes a model really shouldn't care what outputs are used as constraints
the model only cares about its execution, it's the OCP that should care about what NLP functions the model provides to the OCP
at the same time, we'd want to be able to change to a model written in a different software, but can be executed in python, and run OCP with it, and that should only change the 'model'
ideally one should be able to change which outputs to put constraints on, even after a model is constructed
clean up the logistics on what indices do the model and the ocp need to manage respectively
we would still like to keep the OCP agnostic of the backend of choice and hence all the transformations used

Some challenges:

con_outputs names are used quite widely, and deeply, to construct variable indices, orders, flat inputs for the implicit functions, etc
it is also used for construction model algebra jacobian structures
if we create indices for array variables which also have names, then we can't use arbitrary subsets every time! Although for dict inputs and outputs, the indices and orders are less important
as we start to decouple ocp indexing and model indexing, we need to remove and avoid using model indexing inside OCP
information management of constraints composition of the stage_con is definitely a week point that has hard-coded assumptions at lots of places=

Some questions:

should we keep con_outputs somewhere, or just make it a variant instance of outputs?
who should be the owner of the implicit method and its associated derivatives? The model owns it now, but would require inputs from OCP to determine what outputs to use as constraint outputs. The OCP could also own it, but in that case, how do we bring in external model?

#IO definition
StateSpaceModel
forward
states, inputs -> states_dot, outputs
forward_with_array
states, input -> states_dot, output, outputs might be just a subset, as it is much cheaper to collect the subset before the outputs array is formed than after
implicit
states, inputs, states_dot, con_outputs -> residuals
flat implicit
as implicit, but with a flat array as inputs


Telegraph-DS, Chronograf K8S dashboards not working

I must be doing something not quite right.  I deployed Telegraph-DS to my cluster, and in my host list in Chronograf, I can see my cluster worker nodes (status and cpu/load is not displaying).  I can click on a node and see the metrics associated with it just fine.
If I go to the dashboard in Chronograf and try to view any of the K8S dashboards, I'm greeted with:
To enable this graph, you will need to configure the Telegraf Kubernetes Input plugin.
I thought Telegraph-DS enables the Kubernetes Input Plugin by default.
My config map for telegraph-ds has the input:
[[inputs.kubernetes]]
bearer_token = "/var/run/secrets/kubernetes.io/serviceaccount/token"
insecure_skip_verify = true
url = "https://:10250$HOSTIP"
And from the pod console, $HOSTIP resolves properly an an environment variable.
Any thoughts on what I'm missing?  Thanks.

This was on me and not understanding how it works.  I was having Telegraf dump metrics into a different database just for itself.  So when I logged into Chronograf, it was using my cluster metrics that were coming into InfluxDB from Prometheus, not from Telegraf.
Since the measurements from Telegraf didn't overlap measurements from Prometheus, I moved Telegraf data into the same database as Prometheus metrics so when I change data sources, it's all relevant data to the Cluster I'm looking at.


CardInserted event

Is there any way to get an event when de card is removed from the card reader?
What I am looking for is to create a 'CardStatusChanged' event with 'Inserted' and 'Removed' statusses.
I have been looking through the code.  The events 'Connected' and 'Removed' seem to apply to the reader itself.
thx

That implementation depends on the specifics of your device. The intent of this library is to be generic. You will need to write the code to detect card insertion and removal.


Reported Bug

Issue reported by<EMAIL_ADDRESS>Web component version: 1.28.0
Angular/React version: 5.3.0
Expected Behaviour
The GoADropdown list should either open up or down within the field.
Actual Behaviour
The GoADropdown list list appear far away from the field, user has to scroll down the page to see the list.
Replication Steps
The issue is discussed in the #design-system-support by another team:
https://goa-dio.slack.com/archives/C02PLLT9HQ9/p1731600318618909
Additional Info
The issue is discussed in the #design-system-support by another team:
https://goa-dio.slack.com/archives/C02PLLT9HQ9/p1731600318618909

Closing as this was resolved by setting the relative property


html content had been parsed correctly except youtube video links

I am trying to parse this page http://almawqea.net/news.php?id=2547 which contains youtube link like youtube-nocookie.com/xxxx`. the article content has been retrieved exept the div contains the youtube video link

- فيديو :

i am using the following rules:
return array(
   'grabber' => array(
       '%.*%' => array(
           'test_url' => 'http://almawqea.net/news.php?id=2547',
           'body' => array(
               '//div[@class="main-content-column-1"]',
           ),

           'strip' => array(
               '//*[contains(@class, "title")]',
               '//*[@class="legend-default"]',
               '//*[contains(@class, "img-thumbnail")]',
               '//*[contains(@class, "tag")]',
               '//*[contains(@class, "addthis_sharing_toolbox")]',
           )
       )

   )
);

Any help please

The iframe src get filtered out because it's not whitelisted by the filter. You need to add http://www.youtube-nocookie.com and https://www.youtube-nocookie.com to the whitelist. Look at the function get_iframe_whitelist() in models/config.php.

it works .. thanks a lot


cmd/link: openshift build fails due to another problem with too far branches on ppc64le

Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (go version)?
go 1.9
What operating system and processor architecture are you using (go env)?
Ubuntu 16.04
What did you do?
Built upstream openshift binaries, which builds using packages from Kubernetes.
What did you expect to see?
Binaries built that work.
What did you see instead?
Some binaries now fail due to a new linker limit being exceeded, which causes the GNU linker to generate plt stubs for long calls.  The stub generated in this case by the GNU linker expects that R2 contains the address of the TOC (according to the PPC64LE v2 ABI) but code generated by golang in this case does not maintain that value in R2.  As a result the generated address is bad and attempts to branch to a bad address.
I know trying to set up and maintain R2 in this case is not the best solution.  I believe the solution as requested here https://github.com/golang/go/issues/17917 is the right answer but I need to test it out more.  (I had some concerns about this solution which is why I wasn't able to get it into go 1.9).
To reproduce this, clone the repo from here:
https://github.com/openshift/origin.git
cd origin; make all
Try to run one of the tests in:
_output/local/bin/linux/ppc64le with the -h option and the result is a SEGV.

The failure occurs the first time main.init tries to call a plt_branch sequence like this:
000000001380448c <00000038.plt_branch.36:b9ef>:
1380448c:   ff ff 82 3d     addis   r12,r2,-1
13804490:   10 4e 8c e9     ld      r12,19984(r12)
13804494:   a6 03 89 7d     mtctr   r12
13804498:   20 04 80 4e     bctr
But even though this is a dynamic executable built by the GNU linker, the code is not generated to load R2 or maintain it.  As a result, this sequence expects R2 to be valid when it is not, and jumps off to a bad address, or 0.
This is the first time I've seen a plt_branch to handle this situation, if the branch offset is not too large then it is just a long_branch and that works fine.
I am able to build valid binaries successfully by using -buildmode=pie.
In reading the PPC64 v2 ABI, it sounds like the linker should know whether or not the procedure has initialized R2 and generate the call stub based on that information?  If that is true then it shouldn't be creating this type of call stub unless it doesn't have correct information about the procedure making the call.  I'm trying to track that down.

I have hit this too(Fedora27/26, go1.8.3), with bins(origin/genman,origin/openshift) ~300M+ in size and was able to get (at least partially) functional binaries using linkmode internal(for non shared bins, haven't tried shared at all), but I haven't verified that binaries are generally correct(yet). They don't segfault when executed(which  seems to be according to your notes).

The workaround for this is to use -bulidmode=pie when building the executables.  I've tested with it and seems to work fine.  It will ensure that r2 is valid as needed.
A better fix is planned for Go 1.9 and is being tested to verify no regressions are introduced.

Actually internal linker is not a workaround(at least not in full static Go/default build, surprisingly it is when you have intermediate files left from failed build with external linker, haven't investigated further this path yet), it fails with relocation overflow. With build mode pie(without -linkshared) and go1.8.3 I'm now hitting illegal instruction in genman, investigating further.
seems as jump to ctr which point in to data

│0x242e1ec0                      .long 0x0

#0  0x00000000242e1ec0 in ?? () at /usr/lib/golang/src/runtime/asm_ppc64x.s:109
#1  0x00000000242e1db4 in runtime.rt0_go () at /usr/lib/golang/src/runtime/asm_ppc64x.s:17
#2  0x00003fffb7d2291c in generic_start_main (main=0x242e49b0 , argc=, argv=0x3fffffffeed8, auxvec=0x3ffffffff040, init=, rtld_fini=,
stack_end=, fini=) at ../csu/libc-start.c:289
#3  0x00003fffb7d22b18 in __libc_start_main (argc=, argv=, ev=, auxvec=, rtld_fini=, stinfo=,
stack_on_entry=) at ../sysdeps/unix/sysv/linux/powerpc/libc-start.c:102
#4  0x0000000000000000 in ?? ()

Can you provide directions on how you build genman?  This was reported to me when building openshift using the directions above, and I don't see how to build genman.  If I build openshift with -buildmode=pie using go 1.8.3 it works AFAICT, and it certainly doesn't die in __libc_start_main.
Maybe you noticed, the upstream patch was submitted this morning, and we are testing the go 1.8.3 patch now so that should be available very soon.

I will be changing my solution for this problem as a result of the discussion in #20492.

@laboger genman is built using hack/generate-docs.sh, I did some tracing and it seem that following will build genman(invoked in tree of failed build of OS).
export GOPATH=origin/_output/local/go
go build -x -ldflags='-linkmode=internal' -pkgdir origin/_output/local/pkgdir/linux/ppc64le -o genman -tags 'include_gcs include_oss containers_image_openpgp ' github.com/openshift/origin/tools/genman

(path adjustment needed)
Recently I have done most of the investigation by hacking common.sh like this way(which is our proposed workaround)
+        linkshared=""
+      if [[ "${platform}" == "linux/ppc64le" ]]; then
+        local_ldflags+=" -buildmode=pie"
+        linkshared="-linkshared"
+      fi
+
       if [[ ${#nonstatics[@]} -gt 0 ]]; then
-        GOOS=${platform%/*} GOARCH=${platform##*/} go install \
+        GOOS=${platform%/*} GOARCH=${platform##*/} go install $linkshared \
           -pkgdir "${OS_OUTPUT_PKGDIR}/${platform}" \
           -tags "${OS_GOFLAGS_TAGS-} ${!platform_gotags_envvar:-}" \
           -ldflags="${local_ldflags}" \

A few notes on the workaround.  Most of my testing had been done upstream, and I made the wrong assumption that go 1.8 would behave the same.  But I've since discovered an upstream fix (not in go 1.8)  that is needed to make this workaround work in all cases.  Sorry about that confusion. @jcajka


⚠ Bofrost.lu has degraded performance

In 813f8d6, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1194 ms

Resolved: Bofrost.lu performance has improved in bd35100.


Many lines showing extraneous "PL {name}" (Phase Line) Label (instead of correct abbreviation)

Using the latest mil2525d.stylx with the DictionaryRenderer, about 15% of the line symbols now show an extaneous "PL {Name}" Label. Ex "PL T" (all of the uniquedesignation attributes have a value of "T") in the ControlMeasuresAll package from this repo:

For example the Bridgehead Line above should have a "BL" label instead.

I think this is fixed. I went into the Label Rules and removed the extraneous PL's. @csmoore verify with the latest stylx i sent you today.

This looks fixed - I'll verify with the next PR

verified (just forgot to close, oops)


/submap_query

I use services “/submap_query” to get data from submaps, data is one-dimensional, how to convert one-dimensional data into two-dimensional map matrix?
Through this service, I can save a cells as well as height and width. But the product of height and width is not equal to the number of elements in the cells.How to convert one-dimensional data into two-dimensional map matrix?

I recommend to check in the source code how this data is assembled.


Runnning paginate with orderBy gives me an error

Here is the error message:
"select count(*) as \"total\" from \"payout_transactions\" where \"userId\" = $1 order by \"id\" desc - column \"payout_transactions.id\" must appear in the GROUP BY clause or be used in an aggregate function"

here is the code:
const transactions = yield PayoutTransaction.query().orderBy('id', 'desc').where('userId', userId).paginate(page, pageSize})

This issue happened to me, i had to create a custom paginator, the good thing is that the query works when you use forPage, something like this:
 const totalRecords = yield model.query().count(options.sortField)
 const totalPages = Math.ceil(totalRecords[0].count / recordsPerPage)

 let records = yield model.query()
    .orderBy(sorting.sortField, sorting.sortDirection)
    .forPage(page, recordsPerPage)
    .fetch()
 records = records.toJSON()

 const pageData = {
      page: page,
      pageSize: recordsPerPage,
      totalPages: totalPages,
      totalRecords: records.length,
      data: records
 }

I hope this gets fixed.

The issue is duplicated on the adonis-lucid repo. I am going to close this issue and will keep the other one updated