Filter out private regex IDs from generated constants file

This makes OtherFilesGen skip private regexes from the generated constants file. Private regexes don't correspond to a token (they can't be referred to from expansions), and trying to match tokens with their ID can never succeed. So having an ID for them may just cause bugs, for example: https://github.com/pmd/pmd/pull/2260/files#r371882523

@new-javacc What is your position on this PR?


New FDK/Reconstructor interface

from cil.processors import reconstructors

ad = AcquisitionData(...)
ig = ImageGeometry(...)

# creates a default FDK with Ram-Lak
fdk = reconstructors.create_FDK(backend='tigre') # default
# or pass AcquisitionData in the instantiation
fdk = reconstructors.create_FDK(ad)
# or optionally pass the image geometry they want
fdk = reconstructors.create_FDK(ad, ig)

# possible one-liner if default is sufficient
volume = reconstructor.create_FDK(ad).run()
# or optionally to reconstruct on a specific image geometry
volume = reconstructor.create_FDK(ad, ig).run()
volume = reconstructor.create_FDK().set_data(ad).set_ImageGeometry(ig).run()

# specify the backprojector?

# optional additional configuration of the FDK reconstructor
fdk.set_filter('hann')
filt = fdk.get_filter_array()  # returns a numpy array with the filter that can be modified and passed back
# get filter by name
filter_name = fdk.get_filter # returns hann, ram-lak, custom etc
fdk.set_filter(filt)
# set the fft order for the filter
fdk.set_fft_order = N
# set the reconstruction volume geometry
fdk.set_ImageGeometry(ig)
# set the AcquisitionData
fdk.set_data(ad)

# FINALLY run the algorithm
volume = fdk.run()

# or pass the output
fdk.run(out = volume)

# Iterative reconstructors

tikhonov = reconstructors.create_TikhonovLeastSquares(ad, backend='tigre') # i.e. CGLS
# Operator for the regulariser
tikhonov.set_smoothing('identity') # default to be decided about the method name
# or
tikhonov.set_smoothing('gradient')
tikhonov.set_regularisation(alpha)
# optional
tikhonov.set_ImageGeometry(ig)

# run
num_iter = 100
volume = tikhonov.run(num_iter)
# or
tikhonov.run(num_iter, out=volume)
tikhonov.get_iteration_num # returns the iteration number

tikhonov.get_objective() # returns numpy array with objective values vs iteration number

Add also the explicit PDHG with TV that it's used in MantidImaging. Possibly SPDHG later.


fix(deps): update dependency @shoelace-style/shoelace to v2.18.0

This PR contains the following updates:

Package
Type
Update
Change

@shoelace-style/shoelace
dependencies
minor
2.17.1 -> 2.18.0

Release Notes

shoelace-style/shoelace (@shoelace-style/shoelace)
v2.18.0
Compare Source
Commits

761e062: remove title (Cory LaViska)
1fb72ad: Fix typo in usage.md (#2198) (J Wong) #2198
61f2fdb: Fix textarea resizing bug (#2182) (Konnor Rogers) #2182
4e09bac: fix select padding; fixes #2194 (#2197) (Cory LaViska) #2197
8806672: Added Finnish translations. (#2211) (Jouni Männistö) #2211
53aae15: Added ./dist/translations directory to exports to allow dynamic importing of directory contents (#2210) (Zac Lovoy) #2210
4f78930: add more resilient lazy loading to sl-select (#2204) (Konnor Rogers) #2204
5a3d46a: Add ability to focus sl-radio-group dynamically (#2192) (Christian Schilling) #2192
2c66c85: More resilient directionality checks (#2189) (Cory LaViska) #2189
b144988: sl-tab-group throws error if unmounted too fast (#2218) (Susanne Kirchner) #2218
b4a3b32: add changelog entry (#2219) (Konnor Rogers) #2219
8ca1fc7: SlTooltip call super.disconnectedCallback() (#2226) (Daniel Liebner) #2226
f238fca: update changelog (Cory LaViska)
e89cbc7: Merge branch 'next' of https://github.com/shoelace-style/shoelace into next (Cory LaViska)
bb7e840: Issue 2117: Fix sl-select throws exception if a sl-input in the same form receives a new value with autofill in MS Edge (#2221) (dhellgartner) #2221
ee79a46: update changelog (Cory LaViska)
more responsive carousel pagination dots (#2220) #2220 (Alessandro)
b21f51d: update version (Cory LaViska)
c7b0440: 2.18.0 (Cory LaViska)

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Enabled.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

⚠ Artifact update problem
Renovate failed to update an artifact related to this branch. You probably do not want to merge this PR as-is.
♻ Renovate will retry this branch, including artifacts, only when one of the following happens:

any of the package files in this branch needs updating, or
the branch becomes conflicted, or
you click the rebase/retry checkbox if found above, or
you rename this PR's title to start with "rebase!" to trigger it manually

The artifact failure details are included below:
File name: pnpm-lock.yaml
Segmentation fault


11111:Get Started with Oracle Cloud Infrastructure Operations Insights

Hi,
Please approve the Pull request. The lab was updated with new feature SQL Insights.
Get Started with Oracle Cloud Infrastructure Operations Insights
Workshop ID : 11111
-Vivek

@oravivek
Hello,
Please click the "Resolve conflicts" button and mark all files as resolved after you have gotten rid of the conflicts. Then we will approve your PR as soon as we can. Thanks!

@kaylien
Hi,
I have resolved the conflict.
Please approve.
Thanks.


[WIP]Feat/use select row

Hey,
Here is a new useSelectRow hook.

Adds a isSelected prop to the row object.
Adds a getSelectRowToggleProps to the global API. This function return an object with a onChange event and checked boolean. This is meant to be passed to a checkbox component (I tested it with both native input like in the CodeSandbox or with a fully custom component both can work).

Here is a CodeSandbox example https://codesandbox.io/s/react-table-v7-sandbox-u87y8

@lishine regarding your comment there we should better keep talking here instead.
Please tell me if I understand you well:
You can't have access to the selected rows from outside the Table components, is that correct?

But the click on the checkbox gets registered as well, need to be excluded, this is what I will add to my code now.

Regarding your click issue, that's up to you to stopPropagation in your click callback or do any other event handling. The hook doesn't do anything about this.

@lishine regarding your comment there we should better keep talking here instead.
Please tell me if I understand you well:
You can't have access to the selected rows from outside the Table components, is that correct?

Yes, and I solved it like this, I wrapped the useTable hook and use it my component, so I have access to rows.
import React, { useState, useEffect } from 'react'
import {
    useTable,
    useColumns,
    useRows,
    useFilters,
    useSortBy,
    useExpanded,
    usePagination,
    useFlexLayout,
    useSelectRow,
    useTableState,
} from '../react-table/index'

export const useRTable = ({ tableStateInit, tableProps }) => {
    const [tableState, setTableState] = useTableState(tableStateInit)
    const [totalCount, _setTotalCount] = useState(0)

    const tableInstance = useTable(
        { state: [tableState, setTableState], ...tableProps },
        useColumns,
        useRows,
        useFilters,
        useSortBy,
        useExpanded,
        usePagination,
        useFlexLayout,
        useSelectRow
    )
    const { rows } = tableInstance

    const setTotalCount = totalCount => {
        _setTotalCount(totalCount)
        const { pageSize } = tableState
        setTableState(old => ({
            ...old,
            pageCount: Math.ceil(totalCount / pageSize),
        }))
    }

    return {
        tableInstance,
        totalCount,
        setTotalCount,
        tableState,
        rows,
    }
}

export const Signs = () => {
    const loading = useLocalStore('loading')
    const load = useLocalActions('load')
    const data = useLocalStore('data')

    const tableInstanceProps = useRTable({
        tableStateInit: { pageCount: 0, pageSize: 3 },
        tableProps: {
            data,
            columns,
            manualSorting: true,
            manualFilters: true,
            manualPagination: true,
            disableMultiSort: false,
            disableGrouping: true,
            disableSorting: true,
            disableFilters: true,
        },
    })
    const { setTotalCount, tableState, rows } = tableInstanceProps
    const { sortBy, filters, pageIndex, pageSize } = tableState

    useEffect(() => {
        load({ tableState, setTotalCount, columns })
    }, [sortBy, filters, pageIndex, pageSize])

    return (
        <RTable
            {...{
                loading,
                onRowClick,
                ...tableInstanceProps,
            }}
        />
    )
}

I added to my useRTable, calculation of { rowsSelected, totalRowsSelected }, see bellow.
And for it to work your useSelectRow hook should return new reference for rows, something like this:
    const mutatedRows = useMemo(() => {
        rows.forEach(row => {
            row.isSelected = selected.includes(row.index)
            row.toggleSelected = () => toggleSelected(row.index)
        })

        return [...rows]
    }, [rows, selected, toggleSelected])

(There is some option for useSelectRow to use state [selected, setSelected]  received externally, that way selected can be seen outside)
import React, { useState, useEffect, useMemo } from 'react'
import {
    useTable,
    useColumns,
    useRows,
    useFilters,
    useSortBy,
    useExpanded,
    usePagination,
    useFlexLayout,
    useSelectRow,
    useTableState,
} from '../react-table/index'

export const useRTable = ({ tableStateInit, tableProps }) => {
    const [tableState, setTableState] = useTableState(tableStateInit)
    const [totalCount, _setTotalCount] = useState(0)

    const tableInstance = useTable(
        { state: [tableState, setTableState], ...tableProps },
        useColumns,
        useRows,
        useFilters,
        useSortBy,
        useExpanded,
        usePagination,
        useFlexLayout,
        useSelectRow
    )
    const { rows } = tableInstance

    const { rowsSelected, totalRowsSelected } = useMemo(() => {
        let totalRowsSelected = 0
        let rowsSelected = []
        rows.forEach(({ isSelected, index }) => {
            rowsSelected[index] = isSelected
            if (isSelected) {
                totalRowsSelected += 1
            }
        })
        return { rowsSelected, totalRowsSelected }
    }, [rows])

    const setTotalCount = totalCount => {
        _setTotalCount(totalCount)
        const { pageSize } = tableState
        setTableState(old => ({
            ...old,
            pageCount: Math.ceil(totalCount / pageSize),
        }))
    }

    return {
        tableInstance,
        totalCount,
        setTotalCount,
        tableState,
        totalRowsSelected,
        rowsSelected,
    }
}

But the click on the checkbox gets registered as well, need to be excluded, this is what I will add to my code now.

Regarding your click issue, that's up to you to stopPropagation in your click callback or do any other event handling. The hook doesn't do anything about this.

Thanks for the idea!
So the onRowClick with stopPropagation looks like this.
Clicking on row is also usual use case so code section like this can make itself into the official sandbox example
    const renderRow = (row, index, style = {}) => {
        if (!row) {
            return (
                <Row {...{ style, even: index % 2 }}>
                    <Cell>Loading more...</Cell>
                </Row>
            )
        }
        prepareRow(row)
        return (
            <Row
                onClick={() => onRowClick({ index })}
                {...row.getRowProps({ style, even: index % 2, selected: row.isSelected })}
            >
                {row.cells.map(cell => {
                    return (
                        <Cell
                            {...cell.getCellProps()}
                            onClick={e => {
                                if (cell.column.id === 'selectRow') {
                                    e.stopPropagation()
                                }
                            }}
                        >
                            <span>{cell.render('Cell')}</span>
                        </Cell>
                    )
                })}
            </Row>
        )
    }

@tannerlinsley, @Grsmto Question related to this one:
I've noticed that usePagination + useExpanded(paginateSubRows: false) changes the structure of rows and does not flatten them. With these settings it seems like useSelected won't work properly for nested rows. (I suppose that checking parent row should check all nested rows and vice versa.)

Decided to go with a more internal implementation of this feature. Turns out I needed to add a few more core features to make it work, but I'm happy with it. Thanks for your input guys!

@tannerlinsley v7 already looks great, thank you for making it happen!


Does Japronto Support HTTP/2?

Curious if this project supports HTTP/2 and if it's planned to in the future.

It does not but it's something that I am very interested in doing. There is more important stuff that I have to address first though so I cannot promise any dates.


WebDriverConnect.driver_exists? and activate_driver fail if passed driver name is a string with embedded spaces or mixed case

DESCRIPTION:
WebDriverConnect.driver_exists? and activate_driver methods fail to find existing driver if the passed driver name is a string with embedded spaces or has mixed case - i.e. 'Moose and Squirrel'
EXPECTED BEHAVIOR:
WebDriverConnect.driver_exists? and activate_driver methods should convert string passed as a driver name to a symbol with lower case letters and spaces replaced with underscores, which is how WebDriverConnect.initialize_web_driver saved the driver name.

Fixed in version 4.5.1 of gem


Add .pre-commit hooks

Closes #121

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:
:loudspeaker: Thoughts on this report? Let us know!.


[opentelemetry] Evaluate use cases for eBPF

Context
As of now (12/2024) it seems like most components using eBPF in OpenTelemetry are still in a early stage. With this issue we want to keep track of the changes and re-evaluate where necessary if and how we can leverage eBPF in our landscape.
The following components can to be evaluated:

https://github.com/open-telemetry/opentelemetry-network
https://github.com/open-telemetry/opentelemetry-ebpf-profiler

References

As of now, the https://github.com/open-telemetry/opentelemetry-network can collect telemetry data (mainly metrics) from three sources:

kernel events
k8s events
cloud events

But k8s events are already retrieved via the k8sevents-receiver and cloud events are only supported for GCP and AWS


test(Go SDK): Set up integration tests

add a success case of integration test for two runtimes: go1.x and provided.al2
fix the Go SDK to aws.lambda end time match aws.lambda.invocation end time (Close(t ...time.Time) instead of Close())
do not execute Array.from on options.layersConfig if options.layersConfig is undefined (node/test/lib/create-core-resources.js)

Additionally, wouldn't it be good to configure CI for that? (but that can handled in next PR)


.findOrAdd() should not trigger "add" event if object can be found

Having a go at this one.

Ok looks like there's a few ways this could be fixed.
Problem is that findOrAdd is wrapped by eventify but a find operation doesnt need to be wrapped.
My PR passes the API methods via the state so the findOrAdd is using the external API.
There's a few other ways it could be fixed but it's pretty dodge so lets talk about it?

Doing some catch up...
Where is the PR? Is it merged or maybe closed?
Is your discussion somewhere to follow up? :)

Hey Zoe,
Sorry I forgot to follow this up. :P
We had a discussion on Slack about this:
https://hoodie-community.slack.com/messages/hoodie-pouch/
(http://hood.ie/chat/ if future travellers want to join and read the converstation)
nick [10:22 PM]
My PR techinically fixes the problem but I'm not happy with that

gregor [10:22 PM]
it's not too much code

gregor [10:22 PM]
but we expose the methods separately, including findOrAdd

nick [10:22 PM]
Make something explicit rather than magic?

nick [10:23 PM]
So you call the emitter manually?

gregor [10:23 PM]
yes

gregor [10:23 PM]
It's the only method that is affected by it

gregor [10:23 PM]
maybe we can share some code between the findOrAdd method that can be exposed by itself and the one that triggers events, but I think we should implement it twice

gregor [10:24 PM]
more code, but it's not too bad I think.

gregor [10:24 PM]
yeah it's really a tough one

gregor [10:24 PM]
or: we don't expose it like the other APIs

gregor [10:24 PM]
so you can't do `require('pouchdb-hoodie-api/find-or-add')`

gregor [10:25 PM]
and only use it internally like the `.on`, `.off` methods

I think what I'm going to try is to change the eventify wrapper so we're not wrapping methods and explicitly call the event emitter within the relevant API methods.
@gr2m does that sound like a good way to go forwards?

no, let's leave the emitter out of the separate methods. I wouldn't put the logic into all the methods just because of findOrAdd. I'd rather find a solution for findOrAdd only. Even if the soluition is to implement it twice, or to not expose it as separate method like add etc

Fixed with 07bd9dd


[Feature] adding a random number generator

A nice feature would be to add a random number generator of a certain length with the possibility of declaring an initial combination of numbers / digits. It should works as an integral and separate function of the splin.
Example:
generate random number with length 4 starts from 01 (or range) -> 0123, 0145, 0178 etc.

@wojciechsobon it is now possible by using the --pattern / -p option. Look at examples below.

splitn -p "01\d{2}" outputs sequence of numbers beginning with "01" and ending with two randomly generated
splitn abc -p "a .*" outputs split sequences beginning with "a " ie. a bc and a b c, but not ab c

LGTM, thx!


Extension for Reducer.State/Action causes a compilation error with the new @Reducer macros

Description
Hello! I have updated to ComposableArchitecture 1.4.2 and am trying to use the new macros.
And if I use extensions for "Reducer.State/Action" I get compilation errors, you can see this in the Steps to reproduce section.
Checklist

[X] I have determined whether this bug is also reproducible in a vanilla SwiftUI project.
[X] If possible, I've reproduced the issue using the main branch of this package.
[X] This issue hasn't been addressed in an existing GitHub issue or discussion.

Expected behavior
The code should compile without errors.
Actual behavior
The code doesn't compile.
Steps to reproduce
I have a few extensions for Reducer.State in my code like:
struct Test: Reducer {
    struct State {}
    struct Action {}

    func reduce(into state: inout State, action: Action) -> Effect<Action> {
        .none
    }
}

extension Test.State {
    static let initial: Self = ...
}

And if I replace the Reducer protocol with the @Reducer macro, I get the following compilation errors:
@Reducer                  // Error: Circular reference resolving attached macro 'Reducer'
struct Test {             // Error: Circular reference
    struct State {}
    struct Action {}

    func reduce(into state: inout State, action: Action) -> Effect<Action> {
        .none
    }
}

extension Test.State {    // Error: 'State' is not a member type of struct 'Test'
    static let initial: Self = ...
}

If move static let Initial to struct State everything is ok:
@Reducer
struct Test {
    struct State {
        static let initial: Self = ...
    }

    struct Action {}

    func reduce(into state: inout State, action: Action) -> Effect<Action> {
        .none
    }
}

The Composable Architecture version information
1.4.2
Destination operating system
macOS 13
Xcode version information
Xcode 15.0.1
Swift Compiler version information
swift-driver version: 1.87.1 Apple Swift version 5.9 (swiftlang-<IP_ADDRESS>.108 clang-15<IP_ADDRESS>)
Target: arm64-apple-macosx14.0

Hi @rock88, this is an issue with the Swift compiler unfortunately, not this library. We have documented this issue here.

@mbrandonw Oh I missed this info, sorry


Cannot invoke XCTAssertNotNil with JSON

Maybe I didn't get it correctly, but here is my issue:
I'm unit testing if my custom model --> json generator works correctly. So I'm checking the json dictionary directly afterwards. And for some properties I just want to make sure that a json attribute was serialized and should not be empty. I don't care about the value it contains. So my test code looks something like this:
func testVerifyOld(dict: NSDictionary) {
    XCTAssertNotNil(dict["id"]) // works fine before
}

func testVerifyNew(dict: JSON) {
    XCTAssertNotNil(dict["id"]) // Error: Cannot invoke 'XCTAssertNotNil' with an argument list of type '(JSON)'
}

now I need to change it to the following. which is a bit more to write than the traditional NSDictionary style.
func testVerifyCorrectNew(dict: JSON) {
    XCTAssert(!dict["id"].stringValue.isEmpty)

    XCTAssertGreaterThan(count(dict["id"].stringValue), 5, "must be filled with something")
}

Is there a better way in SwiftyJSON to check for NotNil and NotEmpty?

That's a bug in Swift; you can't check structs for nil. Use these methods instead:
/*
	Workaround extension for asserting nil on a struct.
	See http://stackoverflow.com/questions/26684207/how-can-one-use-xctassertnil-with-optional-structs for further reference.
*/
extension XCTest {

	public func XCTAssertNil<T>(@autoclosure expression: () -> T?, _ message: String = "",
		file: String = __FILE__, line: UInt = __LINE__) {

		XCTAssert(expression() == nil, message, file: file, line: line);
	}

	public func XCTAssertNotNil<T>(@autoclosure expression: () -> T?, _ message: String = "",
		file: String = __FILE__, line: UInt = __LINE__) {

			XCTAssert(expression() != nil, message, file: file, line: line);
	}
}

Try
XCTAssertNotNil(dict["id"].string)

Feel free to reopen this if you have more questions


No data for Tables/Columns

Hi, I'm using your Azure Function and I noticed that the default value for "$getInfoDetails" in "Fetch - Catalog.ps1" is:
$getInfoDetails = "lineage=true&datasourceDetails=true&getArtifactUsers=true&datasetSchema=false&datasetExpressions=false"
"datasetSchema=false" and "datasetExpressions=false" is excluding tables/columns details right?
I don't have the PBIMONITOR_CatalogGetInfoParameters Application Setting in the Azure Function Configuration, would that be the reason? If so, what value should I use? "lineage=true&datasourceDetails=true&getArtifactUsers=true&datasetSchema=TRUE&datasetExpressions=TRUE"?
Thanks for your solution btw, It's been really helpful!!!
Vanessa

You are correct, by default I dont collect schema & expressions.
But you can override that setting using the setting ' PBIMONITOR_CatalogGetInfoParameters'


target msbuild.exe directly

This is more or less a cosmetic change, but the goal is to reduce the effort to clone and build (and run) the Gallery on a fresh PC.
Instead of relying on the correct PATH-Setup (which might be not in place), I enhanced the build.cmd to directly target the msbuild.exe which ships with VS2015.
With this in place you can just open a command promt and hit build. With the previous "@msbuild" it didn't work for me.

The previous version required a VS command prompt (with MSBUILD in the PATH variable). Good change though, I have merged it into dev.


Clarify between 'error' and 'warning' in GIthub Issue

Right now its either 'success' and 'error'. But sometimes the error isn't really an error...

Fred Slater confirmed that 5xx and all 4xx result in no-deployment. We either succeed (2xx) or fail (5xx/4xx). There is no 'warning, but succeed'.


'Open With Atom' Missing in Right Click Context Menu on Ubuntu 18.04

Description
Open With Atom option is missing in Ubuntu 18.04. Yesterday i installed a fresh Ubuntu 18.04 and installed Atom Editor there. I tried both the Snap package and also downloaded the .deb version from the website. None of the cases i found this option.
Earlier on Ubuntu 16.04 this option was available and i also use this on Windows 10.
Steps to Reproduce

Install Ubuntu 18.04
Download Atom from the website
Install the Atom
Open Nautilus file browser and click right button of the mouse on any project

Expected behavior: [What you expect to happen]
Open the context menu with an option 'Open with Atom'
Actual behavior: [What actually happens]
Context menu opened but the 'Open with Atom' option was not there.

Thanks for the report - I just checked 18.04 and it looks like you have to set it up first?  I.e. right-click on a file, click Open With Other Application, click View All Applications,  click Atom and select it.  After doing this, right-clicking on a file of that type has "Open With Atom" available by default.
Going to go ahead and close but if you have any trouble with this or I'm mis-understanding (do you think the "Open With Atom" should be available by default?), let me know.

I tried both the Snap package and also downloaded the .deb version from the website.

Oh also, the Snap package is unofficial, the only officially supported installs are documented in the Flight Manual (the .deb from https://atom.io and the official package repos).

What do you see on right-click with files you've setup to open with Atom?  This is what I see on Ubuntu 18.04 after setting up Atom with .md files:

The menu was like the yours on my Ubuntu 16.04 but now Open with Atom is not available on my Ubuntu 18.04.
I downloaded the package from atom.io and followed the Flight manual again and still facing the same issue.

I am now using this two step option to open the directory on Ubuntu 18.04. But it would be great to have this feature like it was previously on Ubuntu 16.04

I think the issue should remain open and you can consider this as a feature request.


"cargo artifacts not found for target crate" with nightly-2024-01-18

I updated my nightly and suddenly trunk stopped working (trunk 0.18.5) with the following error.
2024-01-18T09:57:13.242985Z  INFO fetching cargo artifacts
2024-01-18T09:57:13.326641Z ERROR ❌ error
error from build pipeline

Caused by:
    0: HTML build pipeline failed (1 errors), showing first
    1: error from asset pipeline
    2: cargo artifacts not found for target crate
Error: error from build pipeline

Caused by:
    0: HTML build pipeline failed (1 errors), showing first
    1: error from asset pipeline
    2: cargo artifacts not found for target crate

I tested on a minimal yew project (no workspace involved)
I did a bisect and found out it works for nightly-2024-01-17 but stopped working for nightly-2024-01-18

side remark, 01-18 also breaks rust-analyzer, which reports failure to resolve proc-macros. My guess is that it made significant changes to the structure of the generated artifacts

I think using nightly is always a risk. Without further information on "why" and "what" breaks, I am not sure we can do much.

Note sure there's something that trunk can do.


Cannot decode some QRCode after fix for #709

After this commit  which fixes #709 , python wrapper with default Binarizer can no longer decode the following QRCode.
Although it can be decoded with Binarizer.FixedThreshold, since it looks quite standard, the implementation probably can be improved somehow since it is the default.
This QRCode contains some custom encrypted data with magic header S12

I can't reproduce your issue: the demo_reader.py and python module from current HEAD decode this just fine.

Here's the correct output with binarizer=zxingcpp.Binarizer.FixedThreshold
Found barcode:
 Text:    "S12<NUL><SOH><DC1> IḞéCp<DEL>JFFS<ESC>ṗ-r<U+8E>Ä9ä<EOT>N<CAN>ṫtD<U+81><DC4>ṫÿex"
 Format:   BarcodeFormat.QRCode
 Content:  ContentType.Binary
 Position: 123x123 900x123 900x900 123x900

Yeah, so it does not work on Windows building with MSVC but works on Linux (WSL) building with GCC

Arghh, that is very unpleasant news because I will have a hard time debugging this. Can you do me a favor and also test if the ZXingReader binary from the example directory displays this behavior (MSVC vs GCC)?

Another bad news, on windows (msvc) this works in debug build but not in release build. On Linux this works on all builds
I am pretty sure it's this algorithm since in the previous commit which does not have this new algorithm works (I use git bisect to find this commit). Previous commit does not have this issue (same for the version on pypi.)

Interesting... Have you seen my edited comment about the #define? Your git bisect makes this test almost unnecessary but not quite.

Yes, I can confirm that use new algorithm cannot detect in this case in release build, comment it out and it can detect just fine

Interesting, if I enable the generation of thresholds_new.pnm file in HybridBinarizer.cpp, it works fine

It seems you are on track to debug this all by yourself. Very much appreciated. 😁

So the cause is definitely the optimize of ThresholdImage function.
I tried two workarounds that are working, one is I surround it with #pragma optimize( "", off ), the other is insert something into this function, in my case a std::string and it also works

Oh my. This might very well turn out to be a compiler bug. :-/. To get some idea about what might go wrong, we'd need to have a look at the calculated BitMatrix. Could you maybe compile your release build with -DPRINT_DEBUG? Then execute ZXingReader -format qrcode -noscale -noinv <image-file.jpg>. You should then find a qr-log.pnm in your current directory. Could you recode this into a png (just for file-size management) and add it here?

Sure. The log file that I get for release build is completely black, while for debug build I get the original image back, so I guess something happens on parameter passing due to optimization?

Can you tell me what exactly you mean by "insert a std::string into the function"? I'm wondering what the least invasive workaround could be. I would especially want to avoid a workaround that has a significant runtime performance impact and the #pragma optimize one seems like it could be quite detrimental if ThresholdBlock gets inlined and then not vectorized.

Hmm, for some reason I cannot use std::string as workaround on my laptop. Currently the only thing works is to disable optimization for this function.

Ok, so the issue is vectorization of this loop,
If I add #pragma loop(no_vector) it can produce the correct result

Ahh I go it, ThresholdBlock function's threshold should be of type T_t.
static void ThresholdBlock(const uint8_t* __restrict luminances, int xoffset, int yoffset, T_t threshold, int rowStride,
						   BitMatrix& matrix)

Looks like with threshold being an int MSVC is making some incorrect vectorization


iOS - Add value prop behind remote config feature flag

Description
LEARNER-8109
In this PR I have added the value prop feature behind the remote config feature flag and remove its dependency on local config.
How to test this PR
We have updated the flag from this place
https://console.firebase.google.com/u/2/project/openedx-mobile-stage-2a1bd/config

@saeedbashir
I tested this on the first run it would not show value prop but once the value has fetched from remote config then on refresh the collection view it will start showing the value prop. FirebaseRemoteConfig is a shared object so once its value will update after fetching from remote config it will enable value prop

PR is ready for another pass.


Bump ttf-parser

...and keep such bumps from updating the lockfile manifest format and hence prematurely bumping our MSRV.

FYI I'm tentatively planning on a minor version release after this to bring Harfbuzz port parity to 10.1.0.


[Feature]: disable compile_commands rule on CI or pipeline

#66 disable compile_commands rule on CI or pipeline


Larger dictionary terms can subsume smaller ones

In GitLab by @nwalczak on Jul 29, 2019, 15:52
We have terms for MG1655 and MG1655_WT.  It looks like currently we won't link against MG1655 even if it's an exact match, instead picking to match MG1655_WT.
This was first reported in #77.

In GitLab by @nwalczak on Jul 31, 2019, 11:53
Looks like I may have been mistaken.  There is no MG1665 in the dictionary.  MG1665_WT is the only match that would work for MG1665, the rest are much longer.
There could be a point of discussion.  Presumably, a reference to MG1665 in the text should link to something, but it 's getting cut-off by the 75% match criteria.  At 6 characters, because of how we round (ie, just truncate), anything that's over 9 characters won't be allowed to match.
Between this and #83, it seems like there is evidence that evaluating the partial match based off of the dictionary term instead of the whole matching world may not be what we want.  If we computed the partial match percentage off of the length of the matching word, MG1655_* would match at almsot 100% for MG1655.
The problem then would be how do we handle overlapping matches.  This would lead to >10 overlaps (as there are a lot of dict terms that start with MG1655_).  How do we handle these > 10 possible links?
Just for clarification, here's a list of some of the possible matches:
  "MG1655_WT": "https://hub.sd2e.org/user/sd2e/design/MG1655_WT/1",
  "MG1655_PhlF_Gate": "https://hub.sd2e.org/user/sd2e/design/MG1655_PhlF_Gate/1",
  "MG1655_pJS007_LALT__I1__IcaRA": "https://hub.sd2e.org/user/sd2e/design/MG1655_pJS007_LALT__I1__IcaRA/1",
  "MG1655_NAND_Circuit": "https://hub.sd2e.org/user/sd2e/design/MG1655_NAND_Circuit/1",
  "MG1655_pTACmin": "https://hub.sd2e.org/user/sd2e/design/MG1655_pTACmin/1",
  "MG1655_IcaR_Gate": "https://hub.sd2e.org/user/sd2e/design/MG1655_IcaR_Gate/1",
  "MG1655_pBADmin": "https://hub.sd2e.org/user/sd2e/design/MG1655_pBADmin/1",
  "MG1655_pJS007_LALT__backbone": "https://hub.sd2e.org/user/sd2e/design/MG1655_pJS007_LALT__backbone/1",
  "MG1655_pJS007_LALT__P3__PhlF": "https://hub.sd2e.org/user/sd2e/design/MG1655_pJS007_LALT__P3__PhlF/1",
  "MG1655_empty_landing_pads": "https://hub.sd2e.org/user/sd2e/design/MG1655_empty_landing_pads/1",

In GitLab by @nwalczak on Jul 31, 2019, 13:07
This idea of how to handle multiple links to similar dict terms was first hit upon in #62.

In GitLab by @nwalczak on Aug 7, 2019, 08:00
changed the description

In GitLab by @nwalczak on Aug 7, 2019, 08:01
Another update.  Originally, I had been typing MG1665, when I should have been looking at MG1655 (65 vs 55).

In GitLab by @nwalczak on Aug 7, 2019, 11:03
I was finally able to reproduce this, after fixing #85.  It looks like MG1655 is in fact in the dictionary and the issue was just that my formatted json version of the item_map was out of date.
Ultimately, this comes down to #62: both MG1655 and MG1655_WT match, and whichever one gets picked is based upon which happens first in the result set.  Now, since we are using multiprocessing this is a bit of a race condition.  The solution is to define arbitration rules for ties (ie, #62).  I'm closing this as a duplicate.

In GitLab by @nwalczak on Aug 7, 2019, 11:03
closed


Onelogin not returning encrypted response

I've created an integration with Onelogin that works without encryption, but after attempting to turn on encryption, login fails because the response from onelogin is not encrypted. What is the proper way to enable this?
Related settings:
# onelogin.saml.properties
###
onelogin.saml2.strict = true
onelogin.saml2.debug = false
onelogin.saml2.security.nameid_encrypted = true
onelogin.saml2.security.authnrequest_signed = true
onelogin.saml2.security.logoutrequest_signed = true
onelogin.saml2.security.logoutresponse_signed = true
onelogin.saml2.security.want_messages_signed = true
onelogin.saml2.security.want_assertions_signed = true
onelogin.saml2.security.sign_metadata = true
onelogin.saml2.security.want_assertions_encrypted = true
onelogin.saml2.security.want_nameid_encrypted = true
urn:oasis:names:tc:SAML:2.0:ac:classes:urn:oasis:names:tc:SAML:2.0:ac:classes:Password
onelogin.saml2.security.requested_authncontext = urn:oasis:names:tc:SAML:2.0:ac:classes:urn:oasis:names:tc:SAML:2.0:ac:classes:Password
onelogin.saml2.security.onelogin.saml2.security.requested_authncontextcomparison = exact
onelogin.saml2.security.want_xml_validation = true
onelogin.saml2.security.signature_algorithm = http://www.w3.org/2000/09/xmldsig#rsa-sha1
###

And onelogin.saml2.sp.entityid is configured to an endpoint which uses this properties file.

When you enable
onelogin.saml2.security.want_assertions_encrypted = true

SAMLResponses with unencrypted Assertions will be rejected.
You may use at Onelogin a connector with the encryption enabled.
If you are using a SAML-Test-Connector, you will need to replace it by a SAML Test Connector (IdP) w/encrypt, that will allow you encrypt the assertion.
If you are using a custom app, you will need to contact Onelogin Help Center open a ticket, provide the ID of your connector and ask them to turn on encryption for that connector if possible.

@pitbulk Thanks. I wasn't sure if this might be an issue with OneLogin itself or this library. I'm using a real connector, though, so I'll continue onward with them.


Hide horizontal scrollbar from verbatim elements when it is not needed

The current version always shows horizontal scrollbars for verbatim elements. This change makes them optional to improve visual appearance and to remove unnecessary clutter when the elements are completely visible.

Thanks. Out of curiosity, what browser displayed the unnecessary scroll bars? I haven't generally noticed them, though they did come up in epub readers that I was experimenting with.

Firefox shows the scrollbars.

On what platform? Firefox is my everday browser and I never saw them unless the listing really was too wide.
(FYI: I've accepted the change and rolled it into a big set of updates that will hopefully be out this weekend.)

Firefox 106.0.3 on Windows 10 version 22H2 build 19045.2130. Strangely, the scrollbars are always visible to me with the original CSS.
Shall I close the pull request?

Interesting, thanks. No, don't close it, I'll close it with the PR that fixes it.

Apparently I forgot to close this PR.


Add capacity management

@pettersv and (@simonekman, since we are a BizDevOps company :sunglasses:)
I felt inspired during my run and decided to upload the content of my brain to GitHub. :smile:
This document raises more questions than it brings answers, but I feel it is a good conversation starter. I would like to make this document public, for full transparency. I would also like to bring this document forward to the next available arch meeting (Nov 4th) to iron out the logic.
The document includes a recommendation to operators, but without taking a decision. @pettersv Can you spot it? :smile:

@lucianvlad @viktor-f PTAL. Thanks!

Before downscale we should:

wait for more then 3  5 days and actually evaluate the capacity trends in last 3 to 6 months and take decision based on that (this is where Timescaledb will help) --> maybe capacity usage is varying in different times of the month or of the year...  more traffic at beginning and end of the month and less traffic/usage in the middle
ask customer if the reduce in capacity usage is not due to quiet period or vacation periods
as customer if they foresee any increase in capacity due to new app releases or new apps additions or something that will require more resources

No point in down-scaling if we need to upscale again in a few days or weeks.


Force C/C++ code to be formatted by GitHub Actions

Trying to dip our toes into automatic formatting. This new workflow will automatically run clang-format on all our C/C++ files and commit the result to the PR. Later, we can add Python autoformatting to this workflow.

Also, if CI does not trigger CI when linting the code, do we always need to trigger CI by pushing empty commit after formatting?

Does clang-format apply to new C files as well?

Yes, it applies to the whole repository except dependencies/ and tests/gemmini/gemmini-rocc-tests/ (i.e. third party code modulo small patches).

Also, if CI does not trigger CI when linting the code, do we always need to trigger CI by pushing empty commit after formatting?

Yes, but that's not a problem. The alternative is to error when the code is improperly formatted, but then you have to do the work of running clang-format and pushing that, anyway. If the code is properly formatted to begin with, no new commits will be created. So no matter what, if the code is improperly formatted, then you will need to push something (either an empty commit or a fix commit). This way is just less work (git commit --allow-empty -m "trigger gha").

Is it not possible to trigger CI automatically after formatting? (no worries if that's difficult)

Is it not possible to trigger CI automatically after formatting? (no worries if that's difficult)

It is potentially dangerous if a bug in clang-format is non-idempotent on a source file.


feed figure included image styles missing

non-public. URL Sorry!
version: 3.2
My feeds have possible html "figure/figcaption" tags around some images for comments.
It does need the following addition to the reader.css;
.item figure {
    display: inline;
}

and change the max-width of:
.item img {
  max-width: 50%;
  height: auto;
  vertical-align: middle;
}

to
.item img {
  max-width: 100%;
  height: auto;
  vertical-align: middle;
}

which works well with all others too.
Thanks!

Thanks for the suggestion. I think this might not work well for however. The idea was, I think, for text to flow around images located on the left side. A few feeds have small thumbnail sized images that shouldn't take the full width.

No, I don't think so.
It (the .item img { max-width: 100% ...) would just expand to the full image width, surrounded and influenced by its parent container(s); In my case the figure element.  The feed viewer should not try to be more clever than the the blogger and size images to 50% of their value.
So, if you think for smaller left sided images (which was mine too) you should as well do for images that are bigger (which was also true for mine).

I guess you are right. I am by no means an CSS expert. Looking https://deathclawdesu.tumblr.com/rss the text doesn't flow around the image and the image is squashed. Removing the max-width makes it look better. Please open a PR, maybe @nt1m can take a look.


lib: os: printk hook cleanups

Make _char_out static, cleanup the console driver code.
Will follow up on __stdout_hook_install separately (https://github.com/zephyrproject-rtos/zephyr/pull/76234).

Thanks for the summary Andy - yes I bumped into this because I was looking into  figuring out the various interaction between logging with backends, logging minimal, logging with the shell backend etc... there's few combinations, not all of which necessarily lead to useful results. :-) And for the console it's odd that you can select multiple backends and they race for which one is the last setting the hook, was considering to convert the hooks into an iterable section, but then bumped into the early printk stuff as well, figured I'd just clean the headers for now.
This whole thing is a bit overwhelming for what it does.

moved the header under sys/

added a typedef for the hook argument

Typo in the last commit message insteadl.

Thanks, fixed.


🐛 [firebase_auth] In verifyPhoneNumber, codeSent seems to be called back without a resendToken

Bug report
When I call verifyPhoneNumber as documented here, the codeSent callback is triggered with a non-null verificationId but a null forceResendingToken, which prevents me from triggering a resend of the code in case the user didn't receive it.
And I didn't find anywhere in the doc reasons why this argument might be null.
Steps to reproduce
await _auth.verifyPhoneNumber(
        phoneNumber: _phoneNumber.toString(),
        verificationCompleted: (PhoneAuthCredential credential) async {},
        verificationFailed: (FirebaseAuthException e) {},
        codeSent: (String verificationId, int resendToken) {
          //here resendToken is null, but why?
        },
        codeAutoRetrievalTimeout: (String verificationId) {},
      );
    }

Flutter doctor
Run flutter doctor and paste the output below:
Click To Expand
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, 1.22.5, on macOS 11.1 20C69 darwin-x64, locale fr-FR)

[!] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    ! Some Android licenses not accepted.  To resolve this, run: flutter doctor --android-licenses
[✓] Xcode - develop for iOS and macOS (Xcode 12.3)
[!] Android Studio (version 4.1)
    ✗ Flutter plugin not installed; this adds Flutter specific functionality.
    ✗ Dart plugin not installed; this adds Dart specific functionality.
[✓] IntelliJ IDEA Ultimate Edition (version 2020.3.1)
[✓] VS Code (version 1.51.1)
[✓] Connected device (1 available)
    ! Error: Séjour is busy: Waiting for Device. Xcode will continue when Séjour is finished. (code -10)

! Doctor found issues in 2 categories.

Flutter dependencies
Run flutter pub deps -- --style=compact and paste the output below:
I get this weird error message I don't know how to solve. So I will add a copy of my pubspec.yaml too.
Click To Expand
The ".dart_tool/package_config.json" file is not recognized by "pub" version, please run "pub get".
pub finished with exit code 65

pubspec.yaml
name: blindly
description: A new Flutter project.

publish_to: 'none' # Remove this line if you wish to publish to pub.dev

version: 1.0.0+1

environment:
  sdk: ">=2.7.0 <3.0.0"

dependencies:
  flutter:
    sdk: flutter
  flutter_localizations:
    sdk: flutter
  intl: 0.16.1

  cupertino_icons: ^1.0.0
  firebase_core: 0.5.3
  firebase_auth: ^0.18.4+1
  firebase_database: ^4.4.0
  firebase_analytics: ^6.3.0
  firebase_crashlytics: ^0.2.4
  firebase_performance: ^0.4.3
  cloud_firestore: ^0.14.4
  google_fonts: ^1.1.1
  logger: ^0.9.4
  flutter_bugfender: ^1.2.2
  url_launcher: ^5.7.10
  flutter_markdown: ^0.5.1
  intl_phone_number_input: #^0.6.0
    git:
      url: git://github.com/natintosh/intl_phone_number_input.git
  pin_code_fields: ^6.0.2
  json_annotation: ^3.1.1
  validators: ^2.0.1
  form_validators: ^0.2.1
  uuid: ^2.2.2

dev_dependencies:
  flutter_test:
    sdk: flutter
  build_runner: ^1.10.11
  json_serializable: ^3.5.1

flutter:
  generate: true
  uses-material-design: true

  # To add assets to your application, add an assets section, like this:
  assets:
    - assets/images/

Hi @sarbogast
Does this not work for you?
    PhoneCodeSent codeSent =
        (String verificationId, [int forceResendingToken]) async {
      print('verificationId: $verificationId');
      print('forceResendingToken: $forceResendingToken');
    };

Thank you

Hi @sarbogast
Does this not work for you?
    PhoneCodeSent codeSent =
        (String verificationId, [int forceResendingToken]) async {
      print('verificationId: $verificationId');
      print('forceResendingToken: $forceResendingToken');
    };

Thank you

What do you mean by "work"? The code you are mentioning only declares a function. In my case, codeSent is called back with a null forceResendingToken.

What do you mean by "work"? The code you are mentioning only declares a function. In my case, codeSent is called back with a null forceResendingToken.

I mean if you replace the codeSent function with this signature, is forceResendingToken still null?

I mean if you replace the codeSent function with this signature, is forceResendingToken still null?

Yes, same thing with this code signature, forceResendingToken is still null.
By official example, do you have a specific sample project in mind?

Yes, same thing with this code signature, forceResendingToken is still null.
By official example, do you have a specific sample project in mind?

I just mean the example project included in the plugin source code here.

I just mean the example project included in the plugin source code here.

Maybe it's a problem related to the Flutter version you are using. Have you tried with the stable branch?

Maybe it's a problem related to the Flutter version you are using. Have you tried with the stable branch?

It works just fine on the latest stable 1.22.5 on Android as well. I was able to reproduce this on iOS 14.3.

flutter doctor -v
[✓] Flutter (Channel master, 1.26.0-2.0.pre.179, on macOS 11.1 20C69 darwin-arm, locale en-GB)
    • Flutter version 1.26.0-2.0.pre.179 at /Users/nevercode/development/flutter_master
    • Framework revision a293c82251 (3 hours ago), 2021-01-05 04:04:03 -0500
    • Engine revision b7352132cd
    • Dart version 2.12.0 (build 2.12.0-179.0.dev)

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at /Users/nevercode/Library/Android/sdk
    • Platform android-30, build-tools 30.0.3
    • Java binary at: /Users/nevercode/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/201.6953283/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 12.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.3, Build version 12C33
    • CocoaPods version 1.10.0

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.1)
    • Android Studio at /Users/nevercode/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/201.6953283/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      � https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      � https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)

[✓] Connected device (4 available)
    • Nevercode’s iPhone (mobile) • b668e524315069f3db3661ac11ff1f66afafebdb • ios            • iOS 14.3
    • iPhone 12 Pro Max (mobile)  • 09D9E93F-A98D-4735-B6E6-6E57F9CD01D8     • ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)             • macos                                    • darwin-arm64   • macOS 11.1 20C69 darwin-arm
    • Chrome (web)                • chrome                                   • web-javascript • Google Chrome 87.0.4280.88

• No issues found!

It works just fine on the latest stable 1.22.5 on Android as well. I was able to reproduce this on iOS 14.3.

flutter doctor -v
[✓] Flutter (Channel master, 1.26.0-2.0.pre.179, on macOS 11.1 20C69 darwin-arm, locale en-GB)
    • Flutter version 1.26.0-2.0.pre.179 at /Users/nevercode/development/flutter_master
    • Framework revision a293c82251 (3 hours ago), 2021-01-05 04:04:03 -0500
    • Engine revision b7352132cd
    • Dart version 2.12.0 (build 2.12.0-179.0.dev)

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at /Users/nevercode/Library/Android/sdk
    • Platform android-30, build-tools 30.0.3
    • Java binary at: /Users/nevercode/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/201.6953283/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 12.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.3, Build version 12C33
    • CocoaPods version 1.10.0

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.1)
    • Android Studio at /Users/nevercode/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/201.6953283/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      � https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      � https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)

[✓] Connected device (4 available)
    • Nevercode’s iPhone (mobile) • b668e524315069f3db3661ac11ff1f66afafebdb • ios            • iOS 14.3
    • iPhone 12 Pro Max (mobile)  • 09D9E93F-A98D-4735-B6E6-6E57F9CD01D8     • ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)             • macos                                    • darwin-arm64   • macOS 11.1 20C69 darwin-arm
    • Chrome (web)                • chrome                                   • web-javascript • Google Chrome 87.0.4280.88

• No issues found!

All I know is that I am receiving a null forceResendingToken along with an non-null verificationId and I'm wondering what circumstances can lead to that. I have just tried with the latest versions of all the FlutterFire libraries and I'm still seeing this, which means I cannot trigger a resend of the code. If this code is always supposed to be returned, shouldn't there be some non-nullable and assert logic in the code somewhere to detect that case?

All I know is that I am receiving a null forceResendingToken along with an non-null verificationId and I'm wondering what circumstances can lead to that. I have just tried with the latest versions of all the FlutterFire libraries and I'm still seeing this, which means I cannot trigger a resend of the code. If this code is always supposed to be returned, shouldn't there be some non-nullable and assert logic in the code somewhere to detect that case?

At the very least the documentation simply states

When Firebase sends an SMS code to the device, this handler is triggered with a verificationId and resendToken.

so I don't think this should be happening.

At the very least the documentation simply states

When Firebase sends an SMS code to the device, this handler is triggered with a verificationId and resendToken.

so I don't think this should be happening.

I'm back at it, trying to trace what is going on inside the plugin code itself and figure out where this forceResendingToken could get lost.
And in this line, it seems that's where the codeSent callback is called, and I see that verificationId is passed into arguments but not forceResendingToken.
Yet, according to this, this is the name of the argument the Flutter side is expecting.
In fact, when I search forceResendingToken in this repository, I can only find places where it's read, but no place where it is set.
Is that normal?

I double checked the documentation for the native Firebase Auth SDK, and indeed it seems like this resend token functionality is supported on Android but not on iOS. I think the documentation there should reflect that and explain why on iOS, this leads to a null resendToken being forwarded to codeSent.


Support Block Kit

https://api.slack.com/block-kit

Are you working on this, or should I give it a try?

@dashorst I've not started working on this. It would be appreciated if you could work on this.


Cloudflare SRV lookup seems invalid

One of the projects uses your library. But some servers in minecraft, which proxy the server through cloudflare, do not see the server status
https://github.com/AuroraTeam/Launcher/blob/44564e9c7864a996376dfb2c8d55b2a866158e09/src/main/scenes/ServersList.ts#L35C28-L35C62

Sorry, I'm having a very hard time understanding your issue. The first screenshot of JSON - where is that coming from? What is the context for this JSON? Also, what is the context of the second screenshot? It doesn't really tell me anything, I need to see the actual result from the fetchServerInfo promise.
To be clear, what I need are specifics for this scenario:

What version of the library are you using? npm ls minestat-es
What outputs are you expecting from fetchServerInfo? Provide a JSON object.
What outputs are you actually getting from fetchServerInfo? Provide a JSON object.

Also, why is there a warning exclamation triangle next to the SRV record in your DNS screenshot? What does it tell you if you hover over / click on that?

The launcher uses JSON to specify the path to the client libraries, as well as to specify the address of the game server, more details here https://docs.aurora-launcher.ru/basic/clients.html#настроика-профиля . So the client part of the launcher uses your library in version 1.1.7, the problem is that the developer of the launcher himself cannot give a specific answer why, when fixing through the server, Cloudflare does not want to display the status not through ip:port and through A and SRV records (I checked on server monitoring, they see the server status). About the exclamation mark in DNS records Cloudfalre complains that SRV reveals the IP of the VDS server. About fetchServerInfo it is no longer used anywhere and pings servers independently

It looks like the problem may be on the server side or a flaw in the script.

By the way, another issue I wanted to mention but didn't have time to, is the requirement to specify the “_minecraft._tcp” subdomain to allow SRV writes. This is not mentioned in the Readme, and it is not handled in any way in the code. You need to pick one of the options and fix it.

@JoCat Yes, that could certainly be an issue with the server. It could be caused by CloudFlare too, but I don't plan on supporting CloudFlare explicitly if they do some weird munging of the packets for Minecraft traffic.
The error you screenshotted is thrown when we receive less than 6 bytes back in response to the 0xfe, 0x01 preamble. That indicates that we got some invalid/incomprehensible reply from the server. Look into this code for more debugging help - log out that info variable to see what the server is actually returning.

By the way, another issue I wanted to mention but didn't have time to, is the requirement to specify the “_minecraft._tcp” subdomain to allow SRV writes. This is not mentioned in the Readme, and it is not handled in any way in the code. You need to pick one of the options and fix it.

Not sure what this is saying. Are you asking for the library to prepend _minecraft._tcp to the hostname parameter for SRV lookups? The tone of this message comes across as very disrespectful and arrogant to me, can you please clarify what you are trying to say here?

Not sure what this is saying. Are you asking for the library to prepend _minecraft._tcp to the hostname parameter for SRV lookups? The tone of this message comes across as very disrespectful and arrogant to me, can you please clarify what you are trying to say here?

Disrespect or arrogance is out of the question. I just wanted to mention that SRV records have a certain format, but your code or documentation does not take this into account in any way.

@JoCat Sorry, I had assumed it would be obvious that it would pass the string directly to the SRV resolver, and that the user would be prefixing _minecraft._tcp. in a real world scenario. I'll update the documentation and prepend the string if it's not already the start of the hostname string and release a v1.2.0 in the near future.
As to the actual issue reported here, please let me know when you've inspected the info variable and can see the bytes that are coming back from the server.

The first log is the output of the raw variable
Second - variable info

I think I've realized what the error is, your delimiters are counted by 3 zeros, although the correct way is 2 (I took the information from wiki.vg) and if you make this change - everything will work.

Implemented a more correct algorithm for reading a package based on information from wiki.vg
// create working copy of buffer
const infoBuffer = Buffer.alloc(raw.readInt16BE(1) * 2, 0, 'binary');
// Copy data into buffer
raw.copy(infoBuffer, 0, 3);
// swap bytes (convert big endian to little endian for reading)
infoBuffer.swap16();

// read buffer to string
const info = infoBuffer.toString('utf16le').split('\x00');

// attempt to parse data from server
const version = info[2];
const motd = info[3];
const players = parseInt(info[4], 10);
const maxPlayers = parseInt(info[5], 10);

Example of work:

I made a fork to test my concept. Everything seems to work, the only thing is that due to a small change in the parsing logic, the tests that catch the parsing error will need to be updated. If there are any changes or requirements from your side, please let me know. I will try to make them as soon as possible. Or you can use the sample code I posted above and refine it yourself (you can also look at the first version of my comment, the code there is different, maybe it will work better for you).

I'm working with the code you've provided, but my "valid parse" test case is broken at the moment and I don't have any more time this weekend for personal projects. I'll try to take a look this week and get something merged in.

@JoCat Please update to 2.1.0 and let me know if you continue to have issues. Thanks for your contribution.

The byte you had to change in the tests, that was captured from a real server so it should be correct

strange, it seems that the packet is either longer (several bytes are missing, it seems to be in MOTD), or there is a wrong packet length specified in principle


Fix #2235

Description
Fixes issue where OSQP solver data could become erroneous.
Issue link (if applicable): #2235
Type of change

[ ] New feature (backwards compatible)
[ ] New feature (breaking API changes)
[x] Bug fix
[ ] Other (Documentation, CI, ...)

Contribution checklist

[x] Add our license to new files.
[x] Check that your code adheres to our coding style.
[ ] Write unittests.
[x] Run the unittests and check that they’re passing.
[x] Run the benchmarks to make sure your change doesn’t introduce a regression.

@SteveDiamond Actually, let me quickly push a test.

Closing because this is not an issue in the latest OSQP


Feature/Question - Remove reference to the furnace module

The module Furnace is been deprecated for FoundryVTT 0.8.6 , its successor for the advanced macro mechanism is Advanced Macros.
Any reference to the Furnace module should be removed, in favor of the new one?

Hum, i thought I had, it appears that chagne didnt make it through. will fix

Nono excuse me I expressed myself very badly, I meant only the reference on the readme.md in the code i dont' see (at least for me) any reference to the furnace!

Yeah, that's what I understood. It should be gone now


Confusion matrix metrics for multi- vs single-task models

The confusion matrix that gets stored in the model metadata is internally inconsistent - for multitask models it is like:
'confusion_matrix': [[50, 3], [0, 4]]
and for a single task model it is embedded inside another set of brackets:
'confusion_matrix': [[[220, 25], [11, 17]]]
I'm not sure why the inconsistency is there, but it messes up perf_plots.plot_confusion_matrices().
For now I'm checking in a quick fix to plot_confusion_matrices but we also probably need to update the metadata code.

Weird - if anything it should be the other way around. I'll look at the metadata code and see how it's handling metrics in general for multitask models.

Updated perf_plots to check for > 1 task in https://github.com/ATOMScience-org/AMPL/commit/ed92c46647a55a2056f3697a97d42f29724f59f2. If we update the metadata code, this should be reverted.


drop support of Visual C++ 2010 from version 1.58.0 of Boost?

Available Visual C++ Express 2010 edition doesn't support x64. There is a workaround using Windows SDK 7.1 but it's hardly compatible with Windows 8.1 and Visual C++ Express 2010 SP1.

possible solution: ship only 32-bit libraries. NuGet package description/tags should contain this information.

I was not able to install proper NuGet package to Visual C++ Express 2010 SP1 so I can't test Boost NuGet packages.
codeplex download page can't be bigger than 10000 symbols which is not enough to have references on all nuget packages. Possible solutions:

generate several CodePlex Download pages
make GitHub site sergey-shandar.github.io/getboost and place all download pages there.

Decision: only Win32 is supported for Visual Studio 2010.


Added support for OpenRouter API

Addressing #46

@FreedomCoder-dev I have found the root cause of this problem, and while your changes are working, they are treating the symptoms, not the root cause. Thanks for your efforts though.

The root cause of app crashing has been addressed in https://github.com/Renset/macai/pull/49
SSE comment filtering was added as well in top-level function


Make git commands much more efficient

When the branch is set, we only need to pull down that branch:
Adding --single-branch to the clone command tells git to only pull down the branch we need
When grabbing a specific ref, we only need the one ref:
Instead of pulling down 50 commits from each branch, initialize an empty git repo, setup the remote, and pull down the PR ref.
In the context of one of the apps I am working on, this lowers the git download from 140MB to about 1.6MB. While I realize everyone won't save that much, this should still help a lot.
For another reference, this would save about 140MB of download bandwidth per current rails PR build

What blocks this PR from being merged?

What blocks this PR from being merged?

Merge conflicts, apparently, at the very least.

Are there any downsides to adopting this approach?

I can resurrect this PR and work on it, or I can wait for @albus522 to redo it.
Either way, this PR is stale because we moved code around, unfortunately, so I'm closing this one.


Card Name Shortcuts

Rather than writing out the card name in every case, there could be support for a shortcut that would be autofilled with a card name. For example, {c1} would be replaced with the name of the card in column 1.
Could potentially interfere with dymanic card ordering, but the displayed ordering doesn't  necessarily have to match the order in the sheet.
Also runs into potential issues with legends whose names are supposed to be abbreviated, we might need to make a list of exceptions, or just handle those manually?

Not needed anymore with the backend


Failed to open file "CRA_test1_mgatk/temp/temp_bam/barcodes.8.temp0.bam" : No such file or directory

Hi,
I am running mgatk tenx as advised in the documentation:
mgatk tenx -i $folder_bam/possorted_bam.bam  -n CRA_test1 -o CRA_test1_mgatk -c 12 -bt CB -b $folder_bam/filtered_peak_bc_matrix/barcodes.tsv
I get the following error:
Error in checkGrep(grep(".A.txt", files)) :
Improper folder specification; file missing / extra file present. See documentation
Calls: importMito -> checkGrep
Execution halted
(myenv)jovyan@jupyter-bio-2druxandra-2dtesloianu:~$ [E::hts_open_format] Failed to open file "CRA_test1_mgatk/temp/temp_bam/barcodes.8.temp0.bam" : No such file or directory
[Sat Nov  7 14:38:19 2020]
Error in rule process_one_slice:
jobid: 0
output: CRA_test1_mgatk/qc/depth/barcodes.8.depth.txt, CRA_test1_mgatk/temp/sparse_matrices/barcodes.8.A.txt, CRA_test1_mgatk/temp/sparse_matrices/barcodes.8.C.txt, CRA_test1_mgatk/temp/sparse_matrices/barcodes.8.G.txt, CRA_test1_mgatk/temp/sparse_matrices/barcodes.8.T.txt, CRA_test1_mgatk/temp/sparse_matrices/barcodes.8.coverage.txt
RuleException:
SamtoolsError in line 93 of /home/jovyan/my-conda-envs/myenv/lib/python3.7/site-packages/mgatk/bin/snake/Snakefile.tenx:
'samtools returned with error 1: stdout=, stderr=samtools sort: can't open "CRA_test1_mgatk/temp/temp_bam/barcodes.8.temp0.bam": No such file or directory\n'
File "/home/jovyan/my-conda-envs/myenv/lib/python3.7/site-packages/snakemake/executors/init.py", line 2252, in run_wrapper
File "/home/jovyan/my-conda-envs/myenv/lib/python3.7/site-packages/mgatk/bin/snake/Snakefile.tenx", line 93, in __rule_process_one_slice
File "/home/jovyan/my-conda-envs/myenv/lib/python3.7/site-packages/pysam/utils.py", line 75, in call
File "/home/jovyan/my-conda-envs/myenv/lib/python3.7/site-packages/snakemake/executors/init.py", line 560, in _callback
File "/home/jovyan/my-conda-envs/myenv/lib/python3.7/concurrent/futures/thread.py", line 57, in run
File "/home/jovyan/my-conda-envs/myenv/lib/python3.7/site-packages/snakemake/executors/init.py", line 546, in cached_or_run
File "/home/jovyan/my-conda-envs/myenv/lib/python3.7/site-packages/snakemake/executors/init.py", line 2264, in run_wrapper
Exiting because a job execution failed. Look above for error message
[E::hts_open_format] Failed to open file "CRA_test1_mgatk/temp/temp_bam/barcodes.4.temp0.bam" : No such file or directory
When I do ls -lR on the CRA_test1_mgatk directory, it seems like it completely lacks the temp directory
(myenv)jovyan@jupyter-bio-2druxandra-2dtesloianu:~/CRA_test1_mgatk$ ls -lR
.:
total 12
drwxr-sr-x 2 jovyan users 4096 Nov  7 13:05 final
drwxr-sr-x 4 jovyan users 4096 Nov  7 13:11 logs
drwxr-sr-x 3 jovyan users 4096 Nov  7 14:38 qc
./final:
total 120
-rw-r--r-- 1 jovyan users 121446 Nov  7 14:31 chrM_refAllele.txt
./logs:
total 16
-rw-r--r-- 1 jovyan users 1392 Nov  7 14:37 base.mgatk.log
-rw-r--r-- 1 jovyan users  602 Nov  7 14:37 CRA_test1.parameters.txt
-rw-r--r-- 1 jovyan users    0 Nov  7 14:37 CRA_test1.snakemake_tenx.log
drwxr-sr-x 2 jovyan users 4096 Nov  7 14:26 filterlogs
drwxr-sr-x 2 jovyan users 4096 Nov  7 13:11 rmdupslogs
./logs/filterlogs:
total 96
-rw-r--r-- 1 jovyan users 26 Nov  7 14:38 barcodes.10.filter.log
-rw-r--r-- 1 jovyan users 26 Nov  7 14:38 barcodes.11.filter.log
-rw-r--r-- 1 jovyan users 26 Nov  7 14:38 barcodes.12.filter.log
-rw-r--r-- 1 jovyan users 24 Nov  7 14:25 barcodes.13.filter.log
-rw-r--r-- 1 jovyan users 24 Nov  7 14:26 barcodes.14.filter.log
-rw-r--r-- 1 jovyan users 24 Nov  7 14:26 barcodes.15.filter.log
-rw-r--r-- 1 jovyan users 24 Nov  7 14:26 barcodes.16.filter.log
-rw-r--r-- 1 jovyan users 24 Nov  7 14:26 barcodes.17.filter.log
-rw-r--r-- 1 jovyan users 24 Nov  7 14:25 barcodes.18.filter.log
-rw-r--r-- 1 jovyan users 24 Nov  7 14:25 barcodes.19.filter.log
-rw-r--r-- 1 jovyan users 26 Nov  7 14:38 barcodes.1.filter.log
-rw-r--r-- 1 jovyan users 24 Nov  7 14:25 barcodes.20.filter.log
-rw-r--r-- 1 jovyan users 24 Nov  7 14:26 barcodes.21.filter.log
-rw-r--r-- 1 jovyan users 24 Nov  7 14:26 barcodes.22.filter.log
-rw-r--r-- 1 jovyan users 24 Nov  7 14:26 barcodes.23.filter.log
-rw-r--r-- 1 jovyan users 24 Nov  7 14:26 barcodes.24.filter.log
-rw-r--r-- 1 jovyan users 26 Nov  7 14:38 barcodes.2.filter.log
-rw-r--r-- 1 jovyan users 26 Nov  7 14:38 barcodes.3.filter.log
-rw-r--r-- 1 jovyan users 26 Nov  7 14:38 barcodes.4.filter.log
-rw-r--r-- 1 jovyan users 26 Nov  7 14:38 barcodes.5.filter.log
-rw-r--r-- 1 jovyan users 26 Nov  7 14:38 barcodes.6.filter.log
-rw-r--r-- 1 jovyan users 26 Nov  7 14:38 barcodes.7.filter.log
-rw-r--r-- 1 jovyan users 26 Nov  7 14:38 barcodes.8.filter.log
-rw-r--r-- 1 jovyan users 26 Nov  7 14:38 barcodes.9.filter.log
./logs/rmdupslogs:
total 0
./qc:
total 4
drwxr-sr-x 2 jovyan users 4096 Nov  7 13:11 quality
./qc/quality:
total 0

hey sorry @bio-ruxandra-tesloianu somehow I missed seeing this-- did this get resolved somehow?

hey sorry @bio-ruxandra-tesloianu somehow I missed seeing this-- did this get resolved somehow?

no :(

Could you cat the *logs/filterlogs* files?
we recently made some tweaks to tenx mode-- could you try running with version 0.6.0 (latest on Pypi) and let me know if anything gets better?

Hi @caleblareau,
I got the same error while running with version 0.6.1. Here is the output for cat the *logs/filterlogs*:
Kept 715654
Removed 5450
Kept 670028
Removed 5264
Kept 681359
Removed 5555
Kept 790015
Removed 6727
Kept 662046
Removed 6125
Kept 773335
Removed 6794
Kept 659653
Removed 5663
Kept 697615
Removed 6022
Kept 729339
Removed 6202
Kept 666933
Removed 5954
Kept 646504
Removed 5592
Kept 686152

And you're still getting an error associated with
SamtoolsError in line 93 of /home/jovyan/my-conda-envs/myenv/lib/python3.7/site-packages/mgatk/bin/snake/Snakefile.tenx:
'samtools returned with error 1: stdout=, stderr=samtools sort: can't open

?
It looks like the filtering is working as we would expect but the resulting bam file isn't being output... Can you try the same execution with the bcall mode and see if anything changes?

Hmm, so I also am running into this after installing mgatk (0.6.1) on a new system.
Error in rule process_one_slice:                                                                                jobid: 0                                                                                output: mgatk_out/qc/depth/barcodes.29.depth.txt, mgatk_out/temp/sparse_matrices/barcodes.29.A.txt, mgatk_out/temp/sparse_matrices/barcodes.29.C.txt, mgatk_out/temp/sparse_matrices/barcodes.29.G.txt, mgatk_out/temp/sparse_matrices/barcodes.29.T.txt, mgatk_out/temp/sparse_matrices/barcodes.29.coverage.txt                                                                                RuleException:                                                                                OSError in line 113 of /home/chang/miniconda3/envs/venv/lib/python3.7/site-packages/mgatk/bin/snake/Snakefile.tenx:                                                                                No such file or directory: 'mgatk_out/temp/ready_bam/barcodes.29.qc.bam'                                                                                File "/home/chang/miniconda3/envs/venv/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 2352, in run_wrapper                                                                                File "/home/chang/miniconda3/envs/venv/lib/python3.7/site-packages/mgatk/bin/snake/Snakefile.tenx", line 113, in __rule_process_one_slice                                                                        File "/home/chang/miniconda3/envs/venv/lib/python3.7/site-packages/pysam/utils.py", line 61, in __call__                                                                                File "pysam/libcutils.pyx", line 293, in pysam.libcutils._pysam_dispatch                                                                                File "/home/chang/miniconda3/envs/venv/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 569, in _callback                                                                                File "/home/chang/miniconda3/envs/venv/lib/python3.7/concurrent/futures/thread.py", line 57, in run                                                                                File "/home/chang/miniconda3/envs/venv/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 555, in cached_or_run                                                                                File "/home/chang/miniconda3/envs/venv/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 2364, in run_wrapper

Run command
 mgatk tenx -i possorted_bam.bam -b filtered_peak_bc_matrix/barcodes.tsv -bt CB -o mgatk_out -c 48 --snake-stdout --keep-temp-files

I'm running bcall right now, but it's definitely taking much longer than usual.

Ok here is the bcall error:
Traceback (most recent call last):                                                                                File "/home/chang/miniconda3/envs/venv/lib/python3.7/site-packages/mgatk/bin/python/oneSample.py", line 82, in <module>                                                                                pysam.index(outputbam)                                                                                File "/home/chang/miniconda3/envs/venv/lib/python3.7/site-packages/pysam/utils.py", line 61, in __call__                                                                                save_stdout=kwargs.get("save_stdout", None))                                                                                File "pysam/libcutils.pyx", line 293, in pysam.libcutils._pysam_dispatch                                                                                OSError: No such file or directory: 'mgatk_out/temp/ready_bam/ACGCCACGTTGTGGTT-1.qc.bam'

MissingOutputException in line 21 of /home/chang/miniconda3/envs/venv/lib/python3.7/site-packages/mgatk/bin/snake/Snakefile.Scatter:                                                                             Job Missing files after 5 seconds:                                                                                mgatk_out/temp/ready_bam/ACGCCACGTTGTGGTT-1.qc.bam                                                                                mgatk_out/temp/ready_bam/ACGCCACGTTGTGGTT-1.qc.bam.bai
mgatk_out/qc/depth/ACGCCACGTTGTGGTT-1.depth.txt                                                                                mgatk_out/temp/sparse_matrices/ACGCCACGTTGTGGTT-1.A.txt
mgatk_out/temp/sparse_matrices/ACGCCACGTTGTGGTT-1.C.txt                                                                                mgatk_out/temp/sparse_matrices/ACGCCACGTTGTGGTT-1.G.txt                                                                                mgatk_out/temp/sparse_matrices/ACGCCACGTTGTGGTT-1.T.txt                                                                                mgatk_out/temp/sparse_matrices/ACGCCACGTTGTGGTT-1.coverage.txt                                                                                This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait.                                                                                Job id: 0 completed successfully, but some output files are missing. 0                                                                                File "/home/chang/miniconda3/envs/venv/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 584, in handle_job_success                                                                             File "/home/chang/miniconda3/envs/venv/lib/python3.7/site-packages/snakemake/executors/__init__.py", line 259, in handle_job_success                                                                           Exiting because a job execution failed. Look above for error message

So bcall will take much longer in general than tenx mode. If you want to trouble shoot, just supply a shorter list of barcodes (e.g. head filtered_peak_bc_matrix/barcodes.tsv > short.tsv; then supply -b short.tsv
Anywho, I'm not sure that I see a consistent error here compared to what it previously was. You verified java, which is great. Can you send ls -lR of the mgatk output folder?

Log.txt
Here's the output dir.

just to verify @cnk113 did the pipeline work on the test data? It looks liek the per-sample processing just stopped randomly at some point in the cell processing, which is extremely unusual. I don't think that I've seen this behavior before...

Oops, that was my old one, here's the one with a few bcs.
Log.txt

I am quite late here but my error was solved by doing snake stdout at the end of the command

I ran it with snake stdout. I've previously solved an earlier issue like that. This is new.

So I've been trying to debug this, and tried this multiple different Ubuntu versions and python version/env.
From what I understand, everything is fine until filterClipBam which ends up not printing the bam file (I can see it run in htop but almost instantaneously fails)

Ok I found the issue:
Exception in thread "main" picard.PicardException: This program requires input that are either coordinate or query sorted (according to the header, or at least ASSUME_SORT_ORDER and the content.) Found ASSUME_SORT_ORDER=null and header sortorder=unsorted                                                                                at picard.sam.markduplicates.MarkDuplicates.doWork(MarkDuplicates.java:294)                                                                                at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:295)                                                                                at picard.cmdline.PicardCommandLine.instanceMain(PicardCommandLine.java:103)                                                                                at picard.cmdline.PicardCommandLine.main(PicardCommandLine.java:113)

Pretty much it seems the header showed unsorted even though it was sorted through pysam sort.
I just added ASSUME_SORTED=true to your picard command and it works.
However I now get a different error towards the end (almost there!)
Traceback (most recent call last):
File "/media/chang/HDD-6/chang/mg/atac/Adult_DSP_MG_multi_atac_cr/outs/mgatk/mgatk/bin/python/variant_calling.py", line 88, in <module> base_coverage_dict = load_mgatk_output(MGATK_OUT_DIR)
File "/media/chang/HDD-6/chang/mg/atac/Adult_DSP_MG_multi_atac_cr/outs/mgatk/mgatk/bin/python/variant_calling.py", line 30, in load_mgatk_output fwd_base_df[missing_pos] = 0  # fill in missing positions
File "/home/chang/miniconda3/envs/venv3.6/lib/python3.6/site-packages/pandas/core/frame.py", line 2935, in __setitem__                                                                                self._setitem_array(key, value)
File "/home/chang/miniconda3/envs/venv3.6/lib/python3.6/site-packages/pandas/core/frame.py", line 2966, in _setitem_array                                                                                key, axis=1, raise_missing=False
File "/home/chang/miniconda3/envs/venv3.6/lib/python3.6/site-packages/pandas/core/indexing.py", line 1553, in _get_listlike_indexer keyarr, indexer, o._get_axis_number(axis), raise_missing=raise_missing
File "/home/chang/miniconda3/envs/venv3.6/lib/python3.6/site-packages/pandas/core/indexing.py", line 1640, in _validate_read_indexer raise KeyError(f"None of [{key}] are in the [{axis_name}]")
KeyError: "None of [Int64Index([    1,     3,     4,     6,     8,     9,    10,    11,    12,\n               14,\n            ...\n            16551, 16555, 16558, 16560, 16562, 16563, 16565, 16566, 16568,\n            16569],\n           dtype='int64', length=11090)] are in the [columns]"
MissingOutputException in line 163 of /media/chang/HDD-6/chang/mg/atac/Adult_DSP_MG_multi_atac_cr/outs/mgatk/mgatk/bin/snake/Snakefile.tenx:
Job Missing files after 5 seconds:                                                                                mgatk_out/final/mgatk.variant_stats.tsv.gz                                                                                mgatk_out/final/mgatk.cell_heteroplasmic_df.tsv.gz                                                                                mgatk_out/final/mgatk.vmr_strand_plot.png                                                                                This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait.                                                                                Job id: 0 completed successfully, but some output files are missing. 0                                                                                File "/home/chang/miniconda3/envs/venv3.6/lib/python3.6/site-packages/snakemake/executors/__init__.py", line 589, in handle_job_success
File "/home/chang/miniconda3/envs/venv3.6/lib/python3.6/site-packages/snakemake/executors/__init__.py", line 259, in handle_job_success

Uploading 2021-05-28T214711.286892.snakemake.log…

Ok so the latest error seems to be sample specific, I'm assuming no variant was confidently detected or there wasn't enough MT coverage.

I just wanted to add that I get the same error with failing to find the bam file.

Implemented this fix in v0.6.3-- hopefully this solves things, but please comment on thread if not!

Hi @caleblareau - thanks for making this great tool. I am encountering the same issue using v0.6.4 where the files in mgatk_out/temp/ seem to be called after they are deleted. I read through this thread but the implemented solution does not seem to carry over for me - please excuse me if I am missing a key detail.
Using:

OS: Ubuntu LTS 20.04
python: 3.7.12
R: 4.1.2
cellranger-atac-2.0.0

I've included all of the outputs of mgatk below - I am happy to supply any other potentially useful information.
Command:
mgatk tenx --input outs/possorted_bam.bam -bt CB -b outs/filtered_peak_bc_matrix/mini.tsv --mito-genome hg38 --ncores=14

Error message

[Thu Dec 23 21:22:31 2021]
Error in rule process_one_slice:
    jobid: 0
    output: mgatk_out/qc/depth/barcodes.8.depth.txt, mgatk_out/temp/sparse_matrices/barcodes.8.A.txt, mgatk_out/temp/sparse_matrices/barcodes.8.C.txt, mgatk_out/temp/sparse_matrices/barcodes.8.G.txt, mgatk_out/temp/sparse_matrices/barcodes.8.T.txt, mgatk_out/temp/sparse_matrices/barcodes.8.coverage.txt

RuleException:
OSError in line 113 of /home/mvinyard/.anaconda3/envs/mgatk/lib/python3.7/site-packages/mgatk/bin/snake/Snakefile.tenx:
No such file or directory: 'mgatk_out/temp/ready_bam/barcodes.8.qc.bam'
  File "/home/mvinyard/.anaconda3/envs/mgatk/lib/python3.7/site-packages/mgatk/bin/snake/Snakefile.tenx", line 113, in __rule_process_one_slice
  File "/home/mvinyard/.anaconda3/envs/mgatk/lib/python3.7/site-packages/pysam/utils.py", line 61, in __call__
  File "pysam/libcutils.pyx", line 323, in pysam.libcutils._pysam_dispatch
  File "/home/mvinyard/.anaconda3/envs/mgatk/lib/python3.7/concurrent/futures/thread.py", line 57, in run
Exiting because a job execution failed. Look above for error message
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: /home/mvinyard/data/test_sample/.snakemake/log/2021-12-23T212219.494102.snakemake.log

find ./mgatk_out/ produces the following file structure:

mgatk_out
mgatk_out/final
mgatk_out/final/chrM_refAllele.txt
mgatk_out/qc
mgatk_out/qc/quality
mgatk_out/logs
mgatk_out/logs/mgatk.parameters.txt
mgatk_out/logs/rmdupslogs
mgatk_out/logs/base.mgatk.log
mgatk_out/logs/mgatk.snakemake_tenx.log
mgatk_out/logs/filterlogs
mgatk_out/logs/filterlogs/barcodes.5.filter.log
mgatk_out/logs/filterlogs/barcodes.7.filter.log
mgatk_out/logs/filterlogs/barcodes.6.filter.log
mgatk_out/logs/filterlogs/barcodes.3.filter.log
mgatk_out/logs/filterlogs/barcodes.4.filter.log
mgatk_out/logs/filterlogs/barcodes.2.filter.log
mgatk_out/logs/filterlogs/barcodes.10.filter.log
mgatk_out/logs/filterlogs/barcodes.8.filter.log
mgatk_out/logs/filterlogs/barcodes.1.filter.log
mgatk_out/logs/filterlogs/barcodes.9.filter.log

cat base.mgatk.log produces:

Thu Dec 23 21:22:17 UTC 2021: Starting analysis with mgatk
Thu Dec 23 21:22:17 UTC 2021: Processing samples with 14 threads
Thu Dec 23 21:22:17 UTC 2021: mgatk successfully processed the supplied .bam files
Thu Dec 23 21:22:26 UTC 2021: Successfully created final output files
Thu Dec 23 21:22:26 UTC 2021: Intermediate files successfully removed.

 cat filterlogs/* produces:

Kept 35386
Removed 25
Kept 23104
Removed 11
Kept 51542
Removed 19
Kept 42683
Removed 39
Kept 86645
Removed 37
Kept 18473
Removed 11
Kept 211905
Removed 136
Kept 66058
Removed 38
Kept 90319
Removed 53
Kept 33776
Removed 31

cat mgatk.parameters.txt produces:

input_directory: 'outs/possorted_bam.bam'
output_directory: 'mgatk_out'
script_dir: '/home/mvinyard/.anaconda3/envs/mgatk/lib/python3.7/site-packages/mgatk'
fasta_file: '/home/mvinyard/.anaconda3/envs/mgatk/lib/python3.7/site-packages/mgatk/bin/anno/fasta/hg38.fasta'
mito_chr: 'chrM'
mito_length: '16569'
name: 'mgatk'
base_qual: '0'
remove_duplicates: 'True'
handle_overlap: 'False'
low_coverage_threshold: '10'
barcode_tag: 'CB'
umi_barcode: 'MU'
alignment_quality: '0'
emit_base_qualities: 'False'
proper_paired: 'False'
NHmax: '1'
NMmax: '4'
max_javamem: '8000m'

cat mgatk.snakemake_tenx.log  produces nothing.
Probably unnecessary but just in case, cat mgatk_out/final/chrM_refAllele.txt | head -10 produces:

1       G
2       A
3       T
4       C
5       A
6       C
7       A
8       G
9       G
10      T

I think that's everything - I will keep trouble-shooting in the meantime and provide an update if I find a solution. Thanks in advance for your time and any potential help!

Can you try with --snake-stdout ?

Can you try with --snake-stdout ?

Ah this seems to clear up the issue.
I saw it mentioned above

I ran it with snake stdout. I've previously solved an earlier issue like that. This is new.

but thought this would be a separate case. I'm not sure why the snakemake workflow requires that but it's enough for me. I am currently testing the outputs in downstream workflows. Thanks for your help, Caleb!

I have no idea either but glad it's working now :)


⚠ Bofrost.lu has degraded performance

In 98f32b6, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1060 ms

Resolved: Bofrost.lu performance has improved in 17ef127.


gopherberry, boisterously

This pull request was generated by the 'mq' tool
[test]
flake rate: 0.4
logical conflict every: 100
sleep for: 300s
[pullrequest]
requests per hour: 30
deps=[b,g]

/trunk merge


certain exceptions at serialization seem to prevent sending messages to the server

given the following code both raven 5.1.1 as well as the latest pypi version (5.8.1 at the time of writing) will no longer send a message to the server
"""
contrived example, simply the first variant that broke sending a sentry message for me

"""

import raven

client = raven.Client(dsn='<dsn>')

class BreakMe:
    def __getattr__(self, name):
      raise BreakMe, name

def fun():

    breakme = BreakMe()

    raise ValueError, breakme

try:
    fun()
except:
    client.captureException()

while that code looks quite horrible, its what will happen when one has to deal with suds,
and it can happen in a slightly different fashion when dealing with sqlalchemy mapped instances in the expired /detached state
unfortunately making suds reasonable is practically impossible

Given the code you just provided I cannot reproduce this on master.  It sent it just fine.  What version of Python and which interpreter are you using?

The interpreter in question is one of the 2.7 series
I can collect more info next week back at work

closing as cant reproduce and api is deprecated


Production release

@XhmikosR I'd rather not release https://github.com/MaxCDN/bootstrap-cdn/pull/913 until we have @jdorfman's approval from a product perspective, as such, I'll let him thumb this one.

@XhmikosR merge when ready


Identify storage location for break glass accounts for GCP

Describe the Issue
There is a need to determine break glass process for org admin capabilities.  In the event those with the permission are not available.
Additional Context
Add any other context, attachments or screenshots
Acceptance Criteria

[x] Investigate current practice for MS services w/ OCIO SecOps
[ ] Identify appropriate contacts (emails, etc)
[ ] Identify other considerations (recovery codes, tokens, mfa applicability, etc)

.
.
.
.
.
.
.
.
.
.
.
Extra Template Info
Definition of Ready
These set of conditions will need to be met in order to bring a ticket into the sprint and start work. Protects the team from unclear requirements.
Issues (Task/Story/Spike) aligned to an EPIC and linked appropriately.
Assigned to the appropriate CPF MEMBER (and is not left blank in ZenHub) at the start of sprint (1st day of the sprint)
Acceptance has been defined for the issue and has been reviewed with CPF team and approved by the necessary approver.
Has been sized and estimated by the delivery team.
Detailed breakdown of the steps required to complete the story in the additional context
Any additional specifications have been documented, reviewed with CPF and approved by the necessary approver.
Acceptance Criteria
A set of pre-defined requirement that need to be met in order to mark the user story as “done”.
It should be testable with no room for interpretation
It should be either “pass” or “fail”
It should be clear enough for business stake holders to understand
As part of the user story, it should be written from the user perspective
A well written acceptance criteria is great for

managing expectations
defining scope
reducing ambiguity
establishing testing criteria for QA
Given (Pre-condition)
When (Action)
Then (Outcome)

Definition of Done
These set of conditions are met for work to be considered as complete on an issue.
All acceptance criteria(s) have been validated.
All the necessary documentation for the issue has been uploaded to Teams.
Functionality has been tested when applicable.
Functionality has been demonstrated to the relevant stakeholders (where applicable).
Story has been scheduled for a Sprint Demo/community update and impacted teams invited to the Demo.
Stories accepted by PO and documented for potential sharing with impacted users.

Documented = Captured in Community Update (CU) deck
Release notes = aspirational goal of providing a bulleted list of features and changes that our users can touch. Was going to store them in our Cloud Pathfinder repo. Can also display to users in the CU

Waiting for feedback from SecOps/ADMS

Current practice by other groups is local password safe or secure USB.

Process documented.  Just need to work with Warren to create IAM accounts for AWS and GCP accounts and then test.
Vault mount point has been created for secrets storage.


No longer available in Play Store?

Today I tried to install OI File Manager on a new phone, only to find that https://play.google.com/store/apps/details?id=org.openintents.filemanager was returning a "Not Found" page. I had to download the .apk from GitHub instead and turn on the "Allow app installation from unknown sources" option in order to get OI File Manager installed.
Is there some security reason why the app was removed from the Play Store? Or is it simply that it hasn't had a release in two years?

Google Play suddenly didn't like that there is a link to the contribute page on openintents.org

Fix is planned for Monday or Tuesday.
We will focus on alternative distribution method in the future. We don't want to miss the Huawei users.

Hi @friedger ,
Do you have an update on when OI File manager will be available again in the Play Store, or if you found an alternative distribution method?

@friedger any update on when OI File manager will be available again in the Play Store?

@wbrunette @maprehensive I just pushed version 2.3.1 out...

@friedger any updates when OI file manager will make it back into the play store?

@friedger any updates when OI file manager will make it back into the play store?


Use https instead of git:// for source url

git:// is no longer supported by GitHub (https://github.blog/2021-09-01-improving-git-protocol-security-github/)

Thank you for the fix @felixonmars


Multiple WebSocketClients in one thread

I need to connect from one application to a hundred websocket servers and receive messages from them. How do I do this, without creating a thread for each client?

Hi, this is currently not possible without creating many threads. There are no plans to implement this as it would require rewriting most of the client code and we don't have enough contributors :P
I'm closing this as completed, let me know if there is anything I can help with.

Thank you. I found solution on c++: https://github.com/zaphoyd/websocketpp

Hi, this is currently not possible without creating many threads. There are no plans to implement this as it would require rewriting most of the client code and we don't have enough contributors :P
I'm closing this as completed, let me know if there is anything I can help with.

So, the correct way to use this library is to perform the creation operation of each WebSocket in its own thread, right?


New website changes

the containerd website has been moved into https://github.com/containerd/containerd.io. I think this pr should be filed into containerd.io repo. 😄

@fuweid My bad. I misread the instructions provided by @dmcgowan. I've changed this PR to simply remove website-related materials from this repo and will re-target the initial PR to the right repo.

This is a bit confusing as many of these files were already removed under #2572; maybe your local git clone has not been updated in a long time?
I don't think any changes are needed in this repo unless I missed a few files; but this PR is now also removing actual documentation and manpages that were not part of the website, so is incorrect as it stands.  This was one of the reasons we moved the website to containerd.io repo because we had mixed content with manpages, regular docs, and website content all together.

should the netify file be removed as well?

Codecov Report

Merging #2715 into master will decrease coverage by 2.87%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #2715      +/-   ##
==========================================
- Coverage   43.54%   40.67%   -2.88%
==========================================
  Files         101       70      -31
  Lines       10734     9392    -1342
==========================================
- Hits         4674     3820     -854
+ Misses       5326     5011     -315
+ Partials      734      561     -173

Flag
Coverage Δ

#linux
?

#windows
40.67% <ø> (ø)
:arrow_up:

Impacted Files
Coverage Δ

snapshots/native/native.go
1.78% <0%> (-41.52%)
:arrow_down:

archive/tar.go
16.99% <0%> (-26.15%)
:arrow_down:

metadata/snapshot.go
21.53% <0%> (-24.28%)
:arrow_down:

content/local/writer.go
55.2% <0%> (-1.05%)
:arrow_down:

gc/scheduler/scheduler.go
66.34% <0%> (-0.97%)
:arrow_down:

oci/spec_opts.go
20.49% <0%> (-0.28%)
:arrow_down:

mount/temp_unix.go

sys/reaper_linux.go

services/server/server_linux.go

sys/env.go

... and 27 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0146a48...af1c76e. Read the comment docs.

Yeah, looks like I was branching off of a much older commit without doing standard merge diligence. Given the preference for targeting the containerd/containerd.io repo instead, this PR appears to be unnecessary.


Deprecate unused geo codebase

Remove profile edit js and references.
Remove all mapbox HTML references. Remove unused location fields.
Remove all mapbox related config entries.
Remove location forms from views.
Remove unused form class and reverse_geocode method.
Remove unused CSS entries.
Remove deprecated tests.
Remove geo lookup functionality.

Coverage decreased (-0.06%) to 92.947% when pulling 76c52459216ed38ddfa8134f63a5ff1296952984 on johngian:parsys-18-mapbox-legacy into c80c303ef4ce342fb56ef61c373dd3aecebb88d2 on mozilla:master.

r+wc


Remove wiki tab during bootstrapping

We have too many places to look already—the wiki should be disabled by default.

Done in https://github.com/exercism/request-new-language-track/commit/2162a4618ab3cd9b3eef49750a2886c269441523