Add installation for macOS

If there are no specific dependencies, it would be great to add a package for macos users to install from homebrew.

Hi! I'm not familiar with Homebrew and macOS ecosystem in general, but I'll see what I can do

Logalize isn't yet ready to be added to the main homebrew-core repository because we have less than 75 stars. I've created my own Homebrew tap, so now you can install Logalize on macOS like this: brew install deponian/tap/logalize

Logalize isn't yet ready to be added to the main homebrew-core repository because we have less than 75 stars. See details here
I've created my own Homebrew tap, so now you can install Logalize on macOS like this: brew install deponian/tap/logalize


zulip Social Medien Login and Register

how can I integrate a social media login and register into zulip

Have you already looked through https://zulip.readthedocs.io/en/latest/production/authentication-methods.html ?


models.BleachField's form field set to forms.BleachField

Description
Describe: models.BleachField's form field automtically set to forms.BleachField in ModelForm
References
Fix #13
Checklist

[x] I have ran tox to ensure tests pass
[ ] Usage documentation added in case of new features
[x] Tests added / I have not lowered coverage from 100%

@Alirezaja1384 This needs tests to ensure a model form uses the correct field.
Likely best done by updating the form tests to use the bleach form and a new ModelForm with a model in the test project.

Ok, I'll do both soon.
Thanks for your review.

I added another test for models.py and its coverage is now 100%. but I have no idea about forms.py. coverage report says line 35->37 are missed, but there are test_default_widget and test_custom_widget tests in test_settings.py.

@Alirezaja1384 thank you.
I forget if its this package or another where the coverage from the github action isn't ran against all files in the app.
Essentially I need to find some time to look over things 😂👍


Loan Payment Calculator

Enter calculator name -
Loan Payment Calculator
Describe calculator functionality -
Create a calculator to calculate the monthly payment amount and total interest of a loan. The input parameters include the loan amount, interest rate, and loan term. The loan amount refers to the total amount of the loan, the interest rate refers to the annual interest rate of the loan, and the loan term refers to the repayment period of the loan. The output results include the monthly payment amount and total interest. The monthly payment amount can be calculated using the formula for equal monthly installments, and the total interest can be calculated using the following formula: Total Interest = Monthly Payment Amount * Loan Term - Loan Amount.
Under which category you wish to place it -

[ ] Computer Science
[ ] Mathematics
[ ] Physics
[ ] Chemistry
[ ] Biology
[ ] Art
[x] Finance
[ ] Scientific
[ ] Health
[ ] Sports
[x] Let project owner decide
[ ] Suggest a new category - (Answer here)

Kindly assign this issue to me @SarthakKeshari

Hey @1520229462,
Glad to see your eagerness for contributing 🤩.  This issue has already been addressed under #61.
Hope to see you come up with more innovative issues.
Happy contributing!! 🎉


Make the Homebrewery meet Web Content Accessibility Guidelines(WCAG)

As the title says, i think it would be epic if we made the homebrewery more accessible to people with disabilities, the WCAG international standard provides the necessary information on how to do so.
As per 2023, the most recent stable version is WCAG 2, so i suggest that standard is the one used for this issue.

For future reference - WCAG 2 guidelines: https://www.w3.org/TR/WCAG20/

Adding here some issues that should be solved, related to this:
Editor:

[ ] Add pointer cursor icon to the entire foldgutter element
[ ] Add highlight to the foldgutter element
[ ] Add not-allowed cursor icon to snippetbar functions if they are not available (brew history stuff)
[ ] Remove draggability from thumbnail image preview
[ ] Normalize text input look (border-radius, padding) in properties editor

Preview:

[ ] Add ew-resize cursor to  toolbar zoom function
[ ] Increase width of page number to at least 4 digits (4ch + padding)
[ ] Add not-allowed cursor icon to disabled buttons

Navigation:

[ ] Add wait cursor to saving element while saving
[ ] Add hover effect to storage and new save button

Some of these can be done globally:
*:disabled { cursor:not-allowed; }


AES encryption/decryption optimized with Intel AES-NI.

We may want to support AES encryption/decryption optimized with Intel AES-NI in case of common server platform in case we dont already use an hsm. Algorithm proposed:

Detect if the system has AES-NI enabled
In case it has, use the AES-NI primitive
Otherwise switch back to software AES.
https://software.intel.com/sites/default/files/article/165683/aes-wp-2012-09-22-v01.pdf

This issue was https://issues.apache.org/jira/browse/MILAGRO-37 in Apache Jira - that issue will be closed presently

This is mine. I will catch up this next week while in London.


Error installing via Homebrew

When I attempt to install via homebrew, I get the following: Authentication failed for https://github.com/srkmodo/homebrew-tap/.

It seems you have mistyped my username as srkmodo instead of srkomodo, run this command to install it properly: brew install srkomodo/tap/shadowfox-updater
After that just run shadowfox to open the updater.

🤦♂
Thank you


code-gen tests should be compiled with the appropriate feature flags and platform

We need to extend the new code-gen test machinery to:

use the features specified in the #[target_feature = "+...,+..."] attribute when compiling the test (or otherwise the proper assembly won't be generated and the test will fail, --release is not enough, the python script inferred this from the test name)
specify a target platform for the tests to be compiled (e.g. the arm tests do not run on my machine anymore, the python script inferred this from the test name)

ping: @alexcrichton

I think the first point is fixed b/c it uses the literal intrinsic as a function pointer, right? That is, the intrinsic itself surely has to have the right #[target_feature] annotation, so the test I think tests that literal function so should be good.
For ARM and such it's planned! If you'd like to add an couple examples in the repo I can rig up a harness to test it on Travis as well.

I think the first point is fixed b/c it uses the literal intrinsic as a function pointer, right? That is, the intrinsic itself surely has to have the right #[target_feature] annotation, so the test I think tests that literal function so should be good.

Indeed! I got a lot of failures on my system and thought this was not working, but the problem was that the new tests weren't querying the exact instructions that are generated. I've fixed those in #29 , cargo test --release should be green after that.

If you'd like to add an couple examples in the repo I can rig up a harness to test it on Travis as well.

I've added the tests for arm but note that #[target_feature] there is not used yet. I know that the right instructions are generated when:

arm::v6 is compiled for the target arm-unknown-linux-gnueabihf
arm::v7 is compiled for the target armv7-unknown-linux-gnueabihf
arm::v8 is compiled for the target arch64-unknown-linux-gnu

I've posted in the arm pull-request for help querying llc attributes for those targets. Maybe llc supports attributes for those features that can be passed via #[target_feature] or maybe not. I don't know.

Thanks for fixing up OSX! I can work on implementing ARM CI today.

I won't have much time till Saturday but I hope I can look into the
required arm features till then. Those llc dumps would help :)
On Thu 21. Sep 2017 at 16:10, Alex Crichton<EMAIL_ADDRESS>wrote:

Thanks for fixing up OSX! I can work on implementing ARM CI today.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/rust-lang-nursery/stdsimd/issues/33#issuecomment-331167840,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA3NpqGKHNFrZnUwJSojfkJeZCODutlDks5skm4IgaJpZM4PeZIn
.

Ok I've added ARM CI in https://github.com/rust-lang-nursery/stdsimd/pull/36


Amplify Gen 2 - Lambda Resolver Breaks When timeoutSeconds or memoryMB set

Environment information
System:
  OS: macOS 14.3.1
  CPU: (16) arm64 Apple M3 Max
  Memory: 328.11 MB / 48.00 GB
  Shell: /bin/zsh
Binaries:
  Node: 18.18.0 - ~/.nvm/versions/node/v18.18.0/bin/node
  Yarn: undefined - undefined
  npm: 9.8.1 - ~/.nvm/versions/node/v18.18.0/bin/npm
  pnpm: undefined - undefined
NPM Packages:
  @aws-amplify/backend: 0.13.0-beta.4
  @aws-amplify/backend-cli: 0.12.0-beta.4
  aws-amplify: 6.0.19
  aws-cdk: 2.127.0
  aws-cdk-lib: 2.127.0
  typescript: 5.3.3
AWS environment variables:
  AWS_STS_REGIONAL_ENDPOINTS = regional
  AWS_NODEJS_CONNECTION_REUSE_ENABLED = 1
  AWS_SDK_LOAD_CONFIG = 1
No CDK environment variables

Description
Any time I change my lambda functions runtime or memoryMB values my lambda throws this error:
{
    "errorType": "Error",
    "errorMessage": "No context value present for amplify-backend-namespace key",
    "stack": [
        "Error: No context value present for amplify-backend-namespace key",
        "    at _Node.getContext (file:///var/task/index.mjs:356:17)",
        "    at getBackendIdentifier (file:///var/task/index.mjs:431952:39)",
        "    at createDefaultStack (file:///var/task/index.mjs:432251:72)",
        "    at new BackendFactory (file:///var/task/index.mjs:432392:43)",
        "    at defineBackend (file:///var/task/index.mjs:432438:20)",
        "    at file:///var/task/index.mjs:433367:15"
    ]
}

here's my resource def:
import {defineFunction, secret} from '@aws-amplify/backend';

export const loginMutation = defineFunction({
    timeoutSeconds: 60,
    memoryMB: 256
});

if I just export:
export const loginMutation = defineFunction({});

My function works.
Just for some added context I think the error is being thrown from this code:
      /**
       * Retrieves a value from tree context if present. Otherwise, would throw an error.
       *
       * Context is usually initialized at the root, but can be overridden at any point in the tree.
       *
       * @param key The context key
       * @returns The context value or throws error if there is no context value for this key
       */
      getContext(key) {
        const value = this._context[key];
        if (value !== void 0) {
          return value;
        }
        if (value === void 0 && !this.scope?.node) {
          throw new Error(`No context value present for ${key} key`);
        }
        return this.scope && this.scope.node.getContext(key);
      }

any thoughts or advice on how to solve this let me know.

Hi @alexwhb, so far I am unable to reproduce this. Are you using npx amplify sandbox or npx amplify pipeline-deploy to deploy the backend?

@edwardfoyle hey, thanks for looking into it. im using npx sandbox.

Taking a closer look at your stack trace, it looks like you might be running the library in some sort of bundled context? eg The stack trace shows things like file:///var/task/index.mjs:432438:20. Can you share more about how you have installed / built the @aws-amplify/backend package and how you're executing the CLI?

@edwardfoyle Ya absolutely. So I'm building a service that connects to shopify. This particular Lambda logs into Shopify using a customers user name and password, which ideally will populate a dynomoDB table with some info related to the user:
import type {AppSyncResolverHandler} from "aws-lambda";
import {ResolverContext} from "../auth/handler";
import {z} from "zod";
import {getShopifyClient} from "../../utils/shopify-utils";
import {createStorefrontApiClient} from "@shopify/storefront-api-client";
import {createHash} from 'crypto';
// import {docClient} from "../../utils/dynomo-utils";
// import {UserTableName} from "../../utils/backend-utils";
// import {GetCommand, PutCommand} from "@aws-sdk/lib-dynamodb";
import {v4 as uuidv4} from 'uuid';
import {validateInput} from "../../utils/input-validation";

export const handler: AppSyncResolverHandler<
    ResolverArgs,
    ResolverResult
> = async (event, context) => {

    // TODO I need to figure out how I can pass in the app code in here since this should be an unauthorized route.
    const customContext = context.clientContext?.Custom as ResolverContext

    console.log(context)
    console.log(event)

    // this is our salt.
    const appCode = "SOME_CODE" // todo update this to be dynamic

    const schema =
        z.object({
            username: z.string().email(),
            password: z.string()
                .min(5)
                .max(40)
        })

    validateInput(schema, event.arguments)

    // todo figure out where to get store code header from
    const client = getShopifyClient(appCode)
    const accessToken = await getAccessToken(client, event.arguments.username, event.arguments.password)
    const customerData = await getCustomerInfo(client, accessToken)
    // const userID = sha256(`${customerData.customer.id}${appCode}`)
    // const sessionId = await createOrUpdateUser(userID, appCode);

    return {id: customerData.customer.id, sessionId: 'test121212', email: customerData.customer.email}
};

async function getCustomerInfo(client: ReturnType<typeof createStorefrontApiClient>, accessToken: string): Promise<CustomerResponse> {
    const {data, errors, extensions} = await client.request<CustomerResponse>(customerQuery, {
        variables: {
            customerAccessToken: accessToken
        },
    });

    if (errors || data == undefined) {
        console.error(errors)
        throw Error(`Error logging user in: ${JSON.stringify(errors)}`) // todo this should have a code so we can check on the app side
    }

    return data
}

async function getAccessToken(client: ReturnType<typeof createStorefrontApiClient>, username: string, password: string): Promise<string> {
    const {data, errors, extensions} = await client.request<CustomerAccessTokenCreateResponse>(loginMutation, {
        variables: {
            input: {
                email: username,
                password: password
            }
        },
    });

    if (errors || data == undefined) {
        console.error(errors)
        throw Error(`Error logging user in ${JSON.stringify(errors)}`) // todo this should have a code so we can check on the app side
    }

    return data.customerAccessTokenCreate.customerAccessToken.accessToken
}

note I've commented out my DynomoDB imports. Those may have been causing issues. But I will say even with them commented out, when I change the timeout and lambda RAM values I get that error, so I'm not really sure what's up.

Hmm, I'm still not seeing anything out of place. I wonder if you have an older version of the CLI in your package lock? Can you try running npm install -D @aws-amplify/backend-cli@beta @aws-amplify/backend@beta to ensure you've got the latest beta version of those packages?

@edwardfoyle I think I got it!!! 🎉 Your suggestion on the package lock made me think maybe something node related got out of sync... So I deleted  my node_module, my package lock, and my .amplify file and re set everything. Seems to be working now! I think there must have been a dependency that was not correct or something. I so appreciate your help.

I'm re-opening this issue. So This lambda works no problem until I import and setup AWS SDK V3 for dynamodb. As soon as I do that I get this same error again. I think I can use the V2 API just fine. I've done it in the past, but I'm trying to move everything into using the V3 API to stay away from a deprecated or soon to be deprecated system.

@alexwhb @edwardfoyle
Perhaps the following import may be the cause.
import {UserTableName} from "../../utils/backend-utils";

If you have defined UserTableName in backend-utils.ts as follows. The backend.ts gets bundled into the code that is deployed to the lambda and an error occurs when the lambda is called.
import { backend } from "../backend";

export const todoTableName = backend.data.resources.tables["Todo"].tableName;

I think it is better to specify the name of the DynamoDB table in an environment variable. But now, as per https://github.com/aws-amplify/amplify-category-api/issues/2217, there seems to be a problem to get the DynamoDB table for the schema defined in defineData.

Ah, that's a good call. And yes that would definitely cause the problem you are seeing. The backend definition in the amplify folder cannot be referenced in lambda handler code.
If you are trying to read/write to a table that is created as part of your GraphQL data model, we recommend accessing the table from your lambda via the GraphQL API. You can read about how to configure the lambda to have access to your API here.

@fossamagna your absolutely right this was the issue. Sorry for the late reply. Removing this code solved the problem. 🙌🎉
Thanks so much for the help


UDC shouldn't stop if pin not found

After the recent changes to the pin tip centring plan, UDC stops running if Hyperion can't find the pin tip. Instead, GDA should catch the pin tip centring failure and move onto the the next sample. This is how it behaved previous to v8.10.0
Acceptance criteria

Use a different exception when pin tip failed, GDA catches this and moves to next sample.
Test this behavior, mocking the GDA part

Specifically the exception that Hyperion should be wrapping it in is a WarningException e.g. like when the pin is too short/long


Load specific files from url

My problem is that I have too many files on the server and I'm looking to get only some specific files based on the user demand.
When I was having only it was working bit now I think that he's loading all the data, so he can't handle them and he crashes.
Any help please ? And thank you in advance

@Alaa-Nebli  thank you for opening this issue,
We have bug with opening from direct link and we're going to fix it.
We will let you know as soon as it fixed.

Dear @Alaa-Nebli, we fixed this bug in recent release. Please check it out. In case you face other problems please create separate issue. Thank you.


Performance will be improved by setting input strides=output strides for Clip in DirectMLX

I am investigating for the performance of
MobileNet V2 from TFLite models with "nhwc" layout and
MobileNet V2 from ONNX models with "nchw" layout on the implementation with DirectML and DirectMLX API.
I find that nhwc MobileNetV2 model has lots of Clip after Conv2d, the Clip will cost much time on inference. I guess that the Clip will do memory copy and hasn't be optimized in compilation stage.
I have a workaround to resolve this problem:
set Clip's input strides same as its' output strides by changing this line to TensorDesc outputTensor = inputTensor in DirectMLX.h, the Clip will be optimized just like fused into Conv2d, and then the inference time will be significantly reduced to be as same as nchw MobileNetV2.
When building nhwc MobileNetV2 model, we need append Identity after each Conv2d to transpose output tensor from default nchw to nhwc, then transpose this output tensor from nhwc to nchw as the next Conv2d's input tensor. In my opinion, I suppose that the Identity and Reinterpret can be optimized by DML in this model like:
Conv0->Identity(nchw->nhwc)->Reinterpret strides(nhwc->nchw)->Conv1
just like transpose sinking in OpenVINO backend.
I guess that the Identity and Reinterpret sinking may be blocked when there is Clip like:
Conv0->Identity(nchw->nhwc)->Clip->Reinterpret strides(nhwc->nchw)->Conv1 .
I verified that if I remove Identity to run
Conv0->Reinterpret strides(nchw->nhwc)->Clip(input strides = output strides)->Reinterpret strides(nhwc->nchw)->Conv1 , the inference time will be much lower than before.
So in conclusion, I suggest setting Clip's input strides same as its' output strides by changing this line  to TensorDesc outputTensor = inputTensor in DirectMLX.h.

Hi mingmingtasd,
By default, DMLX will produce output striding in packed NCHW order. As you've noticed, this means that if you want the model to run using NHWC inputs, you'll need to insert copies after every operator to convert from NCHW to NHWC. To solve this, DMLX allows you to customize the output strides that get produced. If you produce output strides in NHWC order (and remove the identities), the performance difference should disappear.
To set a custom output striding in DMLX, create an instance of the TensorPolicy class and supply it to Graph::SetTensorPolicy or the constructor of the graph. For NHWC (i.e. interleaved channel) layout, a TensorPolicy::InterleavedChannel() is provided for you which will cause DMLX to produce output strides in NHWC order.
For an example of how to use tensor policies in DMLX, you can find a sample here.
Hope this helps!

Thanks very much! It's very helpful! One more question is:
In the sample here, I find that the strides has been transposed to align with nhwc layout by the TensorProperties ComputeInterleavedChannel(), but why need not we transpose the sizes to nhwc accordingly?
If I understood correctly, then the model may perform as:
Conv0->output(nchw sizes and nhwc strides)->Conv1,
I am confused by the sizes and strides with different layout.
Could you help to explain this? Thanks! @adtsai

I also can't understand how do the output tensor strides of each operator affect the result. For my understanding, set strides for input tensor is enough when do m_device->CreateOperator(&opDesc, IID_PPV_ARGS(&op)), then DML will use the input tensor to compute to get a nchw default tensor. How do the the output tensor strides affect a operator's result? I am very confused.
I reproduced the alogrithem of TensorProperties ComputeInterleavedChannel() to calculate strides. For example, transpose nchw(1,3,2) to nhwc(1,2,3)  (w or h =1):
Memory:
ABCDEF

nchw shape:1,3,2 + default nchw strides: 6,2,1 will produce:

AB
CD
EF

nchw shape 1,2,3 + nhwc strides: 6,1,2 will produce:

ACE
BDF

But according to TensorProperties ComputeInterleavedChannel() :
nchw shape 1,3,2 + nhwc strides calculated by tensor policy : 6,1,3 will produce:

AD
BE
CF

So how will the input(nchw shape 1,3,2 + tensor policy strides: 6,1,3 (nhwc)) ->Operator->output(nchw shape 1,3,2 + tensor policy strides: 6,1,3 (nhwc)) be computed in DML implementation? Could you help to explain it? Thanks very much! @adtsai

One thing to remember is that in DML, a tensor's sizes for something like Convolution are always expressed in NCHW order. So for example, if supplying an input tensor of sizes [1, 2, 3, 4] to Convolution, C is always equal to 2 no matter what layout is used. DML's strides are used to map from the logical dimensions (e.g. NCHW) to a physical memory layout (e.g. nchw, nhwc, or anything else).
This page has a more complete description of using strides in DML: Using strides to express padding and memory layout
In the above example, with shape [1, 3, 2], the channel count is 3. Let's label the channels R, G, and B. Here's what NCHW and NHWC layout would look like in memory:
nchw (strides [6, 2, 1])
RRGGBB

nhwc (strides [6, 1, 3])
RGBRGB

In the NCHW example, the distance between adjacent channels (e.g. R -> G) is 2 elements. This is why the stride in the C dimension is 2.
In the NHWC example, the distance between adjacent pixels in the width is 3 (because you need to skip past the 3 channels of the first pixel to get to the second pixel). So in this case the stride in the W dimension is 3.

Thanks a lot, your detailed explaination is very helpful for me, I understood deeper in DML strides and layout. So the nhwc strides [6,1,3] calculated by TensorProperties ComputeInterleavedChannel() and nhwc strides [6,1,2] produce by transposing default [6, 2, 1] (my way)  can both perform the same nchw logical dimensions [1,3,2], because the physical memory layout has also been changed into these two concurrently as you said:
nchw (strides [6, 2, 1])
RRGGBB
nhwc (strides [6, 1, 3])
RGBRGB)

Am I right?
So from my side and undersatnding, there are two choices to inference an DML operator:

Keep "nchw" physical memory layout:
transposing default strides and sizes and only  setting them into the description of an DML operator's input tensor will not change the physical memory layout
Use "nhwc" physical memory layout:
if we set a totally new strides calculated by TensorProperties ComputeInterleavedChannel() and set the strides both into the descriptions of an DML operator's input tensor and output tensor, the tensor data in physical memory has been re-ordered into different layout with totally different offsets.
Am I right?

I want to know or make sure two key points based on the understanding above:
1. Will the "nchw" and "nhwc" physical memory layout bring different performance?
Since I saw some comments in DirectMLX sample:
// Determine the best tensor layout based on the GPU vendor.
// This is a fairly coarse-grained method, but recent Nvidia GPUs tend to use NHWC
// layouts, while others use the default NCHW.

It seems that different GPU drivers like different physical memory layout.
2. Which condition will control whether to use or change to "nhwc" physical memory layout?
I guess maybe setting explict or non-default strides for the DML operator's output tensor affects the choice, you know, by default, the output tensor don't need explict strides(or will be implicitly produced).
Sorry for taking  a lot of your time to give so much details, thanks again!  @adtsai

nhwc strides [6,1,2] produced by transposing default [6, 2, 1] (my way)

Note that this will only work for some operators, like ELEMENT_WISE, that are agnostic to interpretation of dimensions. Other operators attach specific meaning to dimensions, so [6, 1, 2] might mean something completely different to [6, 2, 1]. Convolution is one example of this: in DML, convolution always interprets the 0th dimension of the input as Batch and the 1st dimension as Channel, and the remaining dimensions are spatial. So if you supply an input tensor of sizes [6, 1, 2] to Convolution, this means the input channel count is 1, whereas an input tensor size of [6, 2, 1] means the channel count is 2. As you can imagine, the channel count greatly affects the behavior of Convolution.
For this reason we recommend using strides to express layout, rather than transposing the logical dimensions themselves.

Will the "nchw" and "nhwc" physical memory layout bring different performance?

Yes, sometimes. In many cases it won't make a difference and you'll see the same performance, but there are some cases where NHWC layout can be better than NCHW or vice versa. This also depends on hardware - some hardware may have better performance using NCHW, and others may have better performance with NHWC.

Which condition will control whether to use or change to "nhwc" physical memory layout?

If you're using the DirectML graph API (which is the case if you're going through DMLX), then usually you don't need to worry about it. The DirectML graph is an optimizing compiler that can automatically select and rewrite tensor layouts, depending on what's most optimal for the hardware you're running on. DMLX gives you the option (via TensorPolicies) to explicitly set a custom layout anyway, but this is most useful if e.g. you want your graph to produce an output in a specific layout. You should still be able to get good performance just by leaving the layout as default.

Thanks for your detailed explanation and suggestions, very helpful! @adtsai

@mingmingtasd Is this closeable? 🤔

@mingmingtasd Is this closeable? 🤔

Yes, close it, thanks! @fdwr @adtsai


Feature Request: RequireRefreshClientSecret as a client option

reference:
https://tools.ietf.org/html/rfc6749#section-6
spec snippet
require client authentication for confidential clients or for any
      client that was issued client credentials (or with other
      authentication requirements)

Reading the spec on refreshing an access_token one could come to a reasonable assertion that the option of requiring a secret for minting a token is separate from requiring a secret for refreshing the token.  More specifically what one considers a confidential client, or what the term "or with other authentication requirements" actually means lays with the configurator of this service.  That being me...
public class Client
{
...
    /// <summary>
    /// If set to false, no client secret is needed to request tokens at the token endpoint (defaults to <c>true</c>)
    /// </summary>
    public bool RequireClientSecret { get; set; } = true;

    private bool? _requireRefreshClientSecret;

    //
    // Summary:
    //     If set to false, no client secret is needed to refresh tokens at the token endpoint
    //     (defaults to RequireClientSecret)
    public bool RequireRefreshClientSecret
    {
        get
        {
            if (_requireRefreshClientSecret == null || RequireClientSecret == false)
                return RequireClientSecret;
            return (bool)_requireRefreshClientSecret;
        }
        set => _requireRefreshClientSecret = value;
    }
...
}

The good thing about the IdentityServer4 extension points, you can replace the IClientSecretValidator service with one that allows this feature;
using System;
using System.Collections.Generic;
using System.Text;
using System.Threading.Tasks;
using IdentityModel;
using IdentityServer4.Events;
using IdentityServer4.Services;
using IdentityServer4.Stores;
using IdentityServer4.Validation;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Logging;

namespace IdentityServer4Extras.Validators
{
    /// <summary>
    /// Validates a client secret using the registered secret validators and parsers
    /// </summary>
    public class NoSecretRefreshClientSecretValidator : IClientSecretValidator
    {
        private IClientSecretValidator StockClientSecretValidator { get; set; }
        private readonly ILogger _logger;
        private readonly IClientStore _clients;
        private readonly IEventService _events;
        private readonly SecretValidator _validator;
        private readonly SecretParser _parser;

        /// <summary>
        /// Initializes a new instance of the <see cref="IdentityServer4.Validation.ClientSecretValidator"/> class.
        /// </summary>
        /// <param name="clients">The clients.</param>
        /// <param name="parser">The parser.</param>
        /// <param name="validator">The validator.</param>
        /// <param name="events">The events.</param>
        /// <param name="logger">The logger.</param>
        public NoSecretRefreshClientSecretValidator(IClientStore clients, SecretParser parser, SecretValidator validator, IEventService events, ILogger<IdentityServer4.Validation.ClientSecretValidator> logger)
        {
            StockClientSecretValidator = new ClientSecretValidator(clients,parser,validator,events,logger);
            _clients = clients;
            _parser = parser;
            _validator = validator;
            _events = events;
            _logger = logger;
        }

        /// <summary>
        /// Validates the current request.
        /// </summary>
        /// <param name="context">The context.</param>
        /// <returns></returns>
        public async Task<ClientSecretValidationResult> ValidateAsync(HttpContext context)
        {
            _logger.LogDebug("Start client validation");

            var fail = new ClientSecretValidationResult
            {
                IsError = true
            };

            var parsedSecret = await _parser.ParseAsync(context);
            if (parsedSecret == null)
            {
                await RaiseFailureEventAsync("unknown", "No client id found");

                _logger.LogError("No client identifier found");
                return fail;
            }

            // load client
            var client = await _clients.FindEnabledClientByIdAsync(parsedSecret.Id);
            if (client == null)
            {
                await RaiseFailureEventAsync(parsedSecret.Id, "Unknown client");

                _logger.LogError("No client with id '{clientId}' found. aborting", parsedSecret.Id);
                return fail;
            }
            var form = (await context.Request.ReadFormAsync()).AsNameValueCollection();
            var grantType = form.Get(OidcConstants.TokenRequest.GrantType);
            if (grantType == OidcConstants.GrantTypes.RefreshToken)
            {
                // upcast;
                ClientExtra clientExtra = client as ClientExtra;
                if (!clientExtra.RequireRefreshClientSecret)
                {
                    _logger.LogDebug("Public Client - skipping secret validation success");
                    _logger.LogDebug("Client validation success");

                    var success = new ClientSecretValidationResult
                    {
                        IsError = false,
                        Client = client,
                        Secret = parsedSecret
                    };

                    await RaiseSuccessEventAsync(client.ClientId, parsedSecret.Type);
                    return success;
                }
            }
            return await StockClientSecretValidator.ValidateAsync(context);
        }

        private Task RaiseSuccessEventAsync(string clientId, string authMethod)
        {
            return _events.RaiseAsync(new ClientAuthenticationSuccessEvent(clientId, authMethod));
        }

        private Task RaiseFailureEventAsync(string clientId, string message)
        {
            return _events.RaiseAsync(new ClientAuthenticationFailureEvent(clientId, message));
        }
    }
}

Can you explain your use case?

I have a desktop client that wants to bind into my service so that they can make authenticated api calls.
The input to my bind call in one case comes from a previous OIDC login from another system(google).  I am basically exchanging one token for another, and that is where the extension_grants come into play.
  What I pass back to the desktop app is {access_token, refresh_token, client_id}.  I want them to refresh directly against the OAuth2 authority I used to mint those tokens, and I don't want the app to proxy through me simply for me to put a client_secret on the call.  I choose to let them refresh directly against the authority, and for this to happen they don't need a client_secret.  From the spec, it looks like they don't even need a client_id, but I like having a client_id for self protection.
So minting a token requires a clientId/secret, refreshing the token at a minimum requires just the refresh_token.  Those being my rules, and from the spec it looks like that is not in conflict with an OAuth2 service to let that happen.

While we agree with you that the spec is loosey goosey on the topic, so it might seem legit, I don't think we want to add one-off flags for these specific scenarios. We'd rather want a more general feature architecture that allows control per-grant type if credentials are required. This would be work and prolly breaking changes. Are you willing to submit a PR with such an architecture? If so, it would have to wait for 3.0 where breaking changes are acceptable.

I can add a PR for 3.0.


Causes issue with AMD Adrenalin.

When the virtual display driver is installed, AMD Adrenalin will not launch. I uninstalled the display driver and AMD Adrenalin launched successfully. No other changes made to the system.
Hardware:
CPU - Ryzen 5700x
GPU - Radeon 6600
32 GB RAM
Windows 11
This is unfortunate, as using the virtual monitor for sunshine was a nice plus.

Interesting. I have not experienced this. I will test on an AMD GPU ASAP and see if I can recreate the issue. Latest Adrenaline?

Interesting. I have not experienced this. I will test on an AMD GPU ASAP and see if I can recreate the issue. Latest Adrenaline?

Yes, the latest adrenalin. I tried reinstalling adrenalin and uninstalling Sunshine as well.

For me it works with current drivers if not streaming but Adrenaline is really buggy with overlays. If I use a real physical headless plug or vdd I cant enable overlay (or at least it looks like), if I turn on main monitor then I can see overlay is drawn on wrong monitor. Didnt find a way yet to fix it, didnt try disabling other monitor completely in windows yet. Other overlays like steam/reshade/ etc. work so my guess is vdd can not fix that as it seems to be an AMD bug. Like adrenaline overlays wont work for me after energy saving mode too

I was not able to recreate the issue with loading. It seem to work fine in conjunction with the AMD Adrenaline software. HOWEVER, alxtraxxx is correct in that the overlay does not show on the virtual display. I was able to recreate this issue in my testing. Bud and I are looking into this, but unsure of an immediate fix. (Unsure if it's on AMD-side or if there's something we can do to enable the functionality.)


SIGBREAK signal from keyboard not propagated by Windows Terminal

Environment
Windows build number:
Platform ServicePack Version      VersionString
-------- ----------- -------      -------------
 Win32NT             10.0.18363.0 Microsoft Windows NT 10.0.18363.0

Windows Terminal version (if applicable):
1.0.1401.0

PowerShell Core:
7.0.1

PowerShell:
Name                           Value
----                           -----
PSVersion                      5.1.18362.752
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.18362.752
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>

Steps to reproduce

Open PowerShell or PowerShell Core in Windows Terminal
Simulate blocking behavior with Python command:
python -c "import time; import threading; t = threading.Thread(None, lambda: time.sleep(9999)); t.start(); t.join()"
Press keyboard shortcut Ctrl+Break

Expected behavior
Terminated Python process
Actual behavior
Python process is still running
Additional notes:
I tried the same steps in pure PowerShell (powershell.exe)/PowerShell Core (pwsh.exe)/CMD (cmd.exe) - and the result is terminated Python process.

Thanks for the report! This is actually already being tracked by another issue on our repo - please refer to #1119 for more discussion.
/dup #1119


add two themes

I tried adding a few different title styles.

flat (default)

blur (I mentioned this to you a few days ago.)

float (Inspired by #20)

I especially love the float theme! Should we create a Style Settings configurable option for background colour? Because I wonder if a white background, but a coloured border could also look really nice, almost like a pill element.
I have some ideas for the blur theme also, as it's slightly jarring with the hard edges. I will find some time to add some adjustments soon.
Side note: add a setting for enabling/disabling the icons, this will go well alongside the theme choices (what do you think the source edit mode should do, remove the hashes also?). I manually remove them from my local code and set the indent to 0, which mimics the VSCode functionality and allows the hashes to "indent" the items.

Of course, we can add style settings configuration options to modify the theme styles.
I also care about the hard edges of the backdrop: blur effect, but I haven't found a good solution. If you have a way to handle it, that would be fantastic. I even think that if you have other style ideas, you could add new themes as well.
I have already agreed to add a setting for enabling/disabling the icons in #8. However, since we refactored the UI implementation, this part of the logic needs to be re-implemented. (By the way, should we close #8 and re-implement it based on the Svelte approach?)
Other points, such as disabling indentation and displaying # in source mode, can all be controlled through settings options.

For sure close that PR, it's completely outdated now. We can implement a similar solution within the new framework. It's only classes I'm adding and icons I'm hiding. Easy enough. I've set it up to be hooked into a setting later.
Need to setup an option for display # in source mode.

@zhouhua I softened the edges on the blur style. Let me know your thoughts.
We also ought to make the background colour for the float style a style-setting also 👍 After that, it's good to go!


get script

I need a script that recognizes scenes and gets frames from scenes.
I can use these git codes?
Thanl so much

Not out of the box, but you could build such a script using PySceneDetect + OpenCV or PyAV.
See api_test.py for an example of how to get the start/end frames on a video.  Then, you can loop through a video and save the frames yourself.
Hope this helps!

Thanks for your good guidance
You can also run a script to identify the scene
On Mon, May 31, 2021 at 5:44 AM Brandon Castellano @.***>
wrote:

Not out of the box, but you could build such a script using PySceneDetect

OpenCV or PyAV.

See api_test.py
https://github.com/Breakthrough/PySceneDetect/blob/master/tests/api_test.py
for an example of how to get the start/end frames on a video. Then, you can
loop through a video and save the frames yourself.
Hope this helps!
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Breakthrough/PySceneDetect/issues/222#issuecomment-851104737,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AMQJ4I2VKTKSD2WREWQONOTTQLPGXANCNFSM45ZJWCTQ
.


Fix DKAN endpoints, detect proper API and update the registry

DKAN provides several API types including internal REST API https://demo.getdkan.org/api, CKAN compatible API /api/3/action/package_list and e.t.c. and DCAT export data.json files. We need at least one API / data export type working to harvest this data effectively. API should be detected, verified and added to registry data catalog entry.

All endpoints tested and updated for all instances using 'scripts/apidetect.py'


LUNA classification source code

Thanks for your impressive work!
After implment the 3D pretrain on LUNA dataset, would you mind provide the LUNA classification tasks code for evaluation?

@cheliu-computation 你进行评估了吗


eicons not shown in export

I apologise if this is too trivial, I just installed wp2static, exported a simple 1 page site, everything is well except for this detail: Eicons defined as
<i class="eicon-close"> ::before </i> , where in css
.eicon-close:before { content: '\e87f'; }
...don't show up. Any quick workaround?

Super late reply - my apologies!
V7 is ready for testing and solves a lot of rewriting issues like this: https://github.com/WP2Static/wp2static/releases/tag/7.0-alpha-003


One thread from cats IO default thread pool was permanently allocated…

… to Kcl loop.
By default IO thread pool size is driven by number of underling machine CPU cores. Which means if you run your service say on t2.micro, this single core will be allocated to KCL, the rest of the IO will be stalled forever.
Run kcl loop in separate blocking thread pool.

Codecov Report

Merging #216 into master will not change coverage by %.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master     #216   +/-   ##
=======================================
  Coverage   47.68%   47.68%
=======================================
  Files          24       24
  Lines         281      281
  Branches        1        1
=======================================
  Hits          134      134
  Misses        147      147

Impacted Files
Coverage Δ

...modb/src/main/scala/fs2/aws/dynamodb/package.scala
73.07% <100.00%> (ø)

...-aws/src/main/scala/fs2/aws/kinesis/consumer.scala
37.70% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 98ca697...0aae94b. Read the comment docs.


Using indicator variable for constraint checking

Using Indicator Variables
Currently, we use the following loop to check whether a given constraint P is valid, i.e., satisfied by all assignments to the variables:
(push)
(assert (not P))
(check-sat)
(pop)

Instead of doing that, we can attempt to generate "indicator variables" which essentially tell us which constraints can be violated.
For example, we can have a constant t0 along with the formula:
(assert (and t0 (not P)))

This formula is satisfiable if and only if both t0 and P can be assigned to true at the same time. This means that if we allocate a different variable for each assert and the check-sat query says SAT, we can look at the model and look which indicator variables were set to 1 to know which constraints were violated.
Getting all Violated Constraints
Of course, in the above model, we will only get a non-deterministic subset of indicator variables set to 1 when the constraints are violated. In order to get all the violated constraints, we can ask the solver to maximize the value of (sum t0 .. tn) which would force it to assign as many ti to 1 as possible.

Okay, the above encoding doesn't actually work. You need something like:
(assert (or (not P1) (not P2) ...))
(assert (= (= t0 1) (not P1)))
(assert (= (= t1 1) (not P2)))
...

The first assert asks whether any assert P1 ... Pn is a contradiction while the second forces the indicator variables to be set to 1 when there is a contradiction.
This alone is still not enough to get all the contradictions; we need to do a binary search over the sum of t1 ... tn to find all violated constraints.

This doesn't work unfortunately. Let's say that there are n contradictions in the original set. To get the solver to find this set, we'd hope we can just ask it to find a model where the sum of the indicator variables is n. However, what we're actually telling it to do is find one assignment which falsifies all n statements at the same time which might not be possible :-(

I'm quite sure that Andrew would try to pitch SHERRLOC to you if he heard you had this problem, since that's exactly what it's for (i.e., assigning blame when a program fails to satisfy a constraint system) and I know Siqiu recently spent a lot of time making SHERRLOC more usable, but I don't know where the code is or how to use it :joy:

Right! I just want "reasonable" error messages but that's gotten pretty annoying even in the absence of type inference. I think it's definitely going to get way worse once I add inference support to Filament sigh.  CC @andrewcmyers in case this discussion seems interesting

#70 attempted to implement this but it didn't work

If you can encode your analysis into SHErrLoc's constraint system then you could just let it do the checking and the error diagnosis.  Otherwise you might be able to adapt the SHErrLoc approach,  which finds a maximum likelihood explanation for static errors.


Update test to aim for 100% coverage

Codecov Report

Merging #1168 into master will decrease coverage by 0.14%.
The diff coverage is 100%.

@@            Coverage Diff            @@
##           master   #1168      +/-   ##
=========================================
- Coverage   99.95%   99.8%   -0.15%
=========================================
  Files          29      29
  Lines        2057    2063       +6
  Branches       85      85
=========================================
+ Hits         2056    2059       +3
- Misses          0       4       +4
+ Partials        1       0       -1

Impacted Files
Coverage Δ

tests/test_dcos_e2e/test_enterprise.py
100% <100%> (ø)
:arrow_up:

tests/test_dcos_e2e/backends/aws/test_aws.py
95.29% <0%> (-4.71%)
:arrow_down:

src/dcos_e2e/cluster.py
100% <0%> (+0.97%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0438d27...045fd23. Read the comment docs.


Bug report : Backup failure

Describe the bug
Since updating to 3.5, I've been getting this warning that the backup has failed. Logs appear to indicate it's getting stuck on something from Biomancy
To Reproduce

Seemingly just allowing a backup to trigger with Biomancy installed.

Expected behavior
Backup to complete as before
Screenshots

Versioning (please complete the following information):

OS: Windows 11
Minecraft Version(s): 1.20.1
Modloader(s): Forge
Mod Version: 3.5

Additional context
https://gist.github.com/MuteTiefling/cf1e4c6c660c242b27f8ebd1e88ace6d

Hmm, and was 3.4 able to make the backup fine with Biomancy installed?

Okay. I've been trying to reproduce this, to no avail.
With AB and Biomancy installed only, both scheduled backups and manual backups correctly hash the file and back it up.
Could I have a full modlist please?

Hrm, alright. Wonder if something has gotten corrupt in my world then...
Forced a crash to get a full mod list for you: https://gist.github.com/MuteTiefling/a8af94ab98785a979f06124115582361

I'd be surprised if corruption could do it... but perhaps, could you send over your AB config and the biomancy.spatial.db from the data directory?

Sure thing. Hope this link works : https://drive.google.com/file/d/1Auz-zvooDFnBi1QdFqEbkmS27S-MibU9/view?usp=drive_link

ab issue.zip
Take two!

This is being fixed in multiple ways:

Add a blacklist for files to not backup
Skip a file if it cannot be backed up.
If any files were skipped, log them in the console / send to a command runner.
In an upcoming version, toasts will also show the skipped files.


Render with max-length 255 charaters for each line

Hello,
I'm beginner with MJML and I tried to render an example template and I see in html file (rendered) lines with more than 255 characters.
This limit is still true for emails ?
And how to make config to limit render at 255 characters max each line ?
Thanks a lot.

Hi @smatoox,
There is no such limitation in HTML email. This 255 characters limitation can be found on certain clients or proxies, impacting GET requests, but nothing to do with HTML emails.
Therefore I'm closing!


Update adaptors.rs

fixes lower bound error
Fixes #118

Thanks for reading & fixing this code that I thought nobody would care about. This can indeed be better. Quickcheck tests should use an exact test for interleave_shortest. Can you please add the testcase from your bug report though?

Sure, can I just use #[test]?

Yes

Thanks, merged with test.


URI schema

In conjunction with #30 for accessing our resources, we also need a consistent URI schema. For instance:

Type
URI example

Digital representation of an object (with content negotiation)
https://participatory-archives.ch/entity/<ID>

PIA central API
https://data.participatory-archives.ch/entity/<ID>

Linked Art API
https://linkedart.participatory-archives.ch/entity/<ID>

IIIF Image API
https://iiif.participatory-archives.ch/<ID>/info.json

IIIF Presentation API (Manifest/Collection)
https://iiif.participatory-archives.ch/<ID>/manifest.json

Controlled Vocabularies
https://vocab.participatory-archives.ch/vocabulary/<ID>

Project Website
https://project.participatory-archives.ch/

Cf. #35

Cf. https://github.com/Participatory-Image-Archives/ark


Make remix-express agnostic of express

Remix depends on express without using most of its features and mostly relying on node http server inside of it.
Express is quite big package which may not be necessary in most apps. https://packagephobia.com/result?p=express
Here I replaced express types and covered missing features.
In the future this package could be renamed into remix-http or something like that.

Remix doesn't depend on Express - Remix runs on the Web Fetch API - this is just an adapter for Express to convert Express requests into Fetch Requests that can be handled by Remix.  So it doesn't make sense to remove the express dependency.
You are free to use whatever adapter you choose, and even write your own if there's not one available for the HTTP server of your choosing.  There's even a raw node adapter already linked to from the docs: https://remix.run/docs/en/main/other-api/adapter#community-adapters

Remix doesn't depend on Express

Not entirely true
https://github.com/remix-run/remix/blob/main/packages/remix-serve/index.ts#L1
https://github.com/remix-run/remix/blob/main/packages/remix-dev/devServer_unstable/index.ts#L8

@brophdawg11 the idea of this pr is to reduce install size of remix.
So user do no need to install unnecessary dependencies with bad internet. Im travelling a lot and have this problem all the time.
https://packagephobia.com/result?p=%40remix-run%2Fdev
Express in most cases is a fancy api around native http server. So if this adapter remove dependency on express users could use it with many other frameworks similar to express. Or just use http.createServer.

Not entirely true

Sorry, I should have clarified - Remix at runtime does not rely on Express.  You are free to run with whatever HTTP server you want.

the idea of this pr is to reduce install size of remix

That's not what this PR is doing?  You've only removed express from the depedencies of @remix-run/express - which doesn't make any sense since that is an adapter specifically designed for folks who want to use express.  This is not a required dependency and therefore will not be included for installs of Remix unless the dev has opted-in.
Similarly, @remix-run/serve is also an opt-in dependency, and relies on express by design: "It's a production-ready, but basic Node.js server built with Express"
@remix-run/dev does currently use express an an implementation detail - and I left https://github.com/remix-run/remix/pull/7087 open because that could be worth considering (deferring to @pcattori there) to reduce the install size.  Please note that as that PR stands though, you've not removed express as a dependency so it won't reduce the install size of Remix.

Similarly, @remix-run/serve is also an opt-in dependency, and relies on express by design: "It's a production-ready, but basic Node.js server built with Express"

Could be built without it

Remix App Server uses Express middleware morgan and compression, so you can't just use the http module directly. There are alternatives to Express that are probably smaller and still compatible with Express middleware, like Fastify or hapi.
RAS is simple and basic, which is what is intended. The nice thing about Remix is that you can create your own server and adapter with whatever framework you want.


Tone down inlay hint background color in lapce dark theme

Before:

After:

I didn't touch the light theme as I think it's less obnoxious there. Disclaimer: I'm not a designer, so let's put this up to a vote.

Also, I've just randomly changed a few digits until it looked like this. If you have an actual color we should use, feel free to let me know :)

Thanks all, looks like nobody (so far) hates this change. Let's put it up for review :)

I personally prefer it even more distinct from code

@pwouik I hear you. We can tune the text color, too, but my goal with this PR was to make inlay hints at least usable on the default dark theme. I would leave further modifications to follow up PRs


Update README/Contributing to install openssl

Seeing this with a fresh install:
matthew@Matthews-MacBook-Pro:~/Documents/rusoto$ cargo build
   Compiling openssl-sys-extras v0.7.1
   Compiling num v0.1.28
   Compiling rust-crypto v0.2.34
   Compiling uuid v0.1.18
Build failed, waiting for other jobs to finish...
failed to run custom build command for `openssl-sys-extras v0.7.1`
Process didn't exit successfully: `/Users/matthew/Documents/rusoto/target/debug/build/openssl-sys-extras-e38fd8aa704c6185/build-script-build` (exit code: 101)
--- stdout
TARGET = Some("x86_64-apple-darwin")
OPT_LEVEL = Some("0")
PROFILE = Some("debug")
TARGET = Some("x86_64-apple-darwin")
debug=true opt-level=0
HOST = Some("x86_64-apple-darwin")
TARGET = Some("x86_64-apple-darwin")
TARGET = Some("x86_64-apple-darwin")
HOST = Some("x86_64-apple-darwin")
CC_x86_64-apple-darwin = None
CC_x86_64_apple_darwin = None
HOST_CC = None
CC = None
HOST = Some("x86_64-apple-darwin")
TARGET = Some("x86_64-apple-darwin")
HOST = Some("x86_64-apple-darwin")
CFLAGS_x86_64-apple-darwin = None
CFLAGS_x86_64_apple_darwin = None
HOST_CFLAGS = None
CFLAGS = None
running: "cc" "-O0" "-ffunction-sections" "-fdata-sections" "-g" "-m64" "-fPIC" "-o" "/Users/matthew/Documents/rusoto/target/debug/build/openssl-sys-extras-e38fd8aa704c6185/out/src/openssl_shim.o" "-c" "src/openssl_shim.c"
ExitStatus(Code(1))

command did not execute successfully, got: exit code: 1

--- stderr
src/openssl_shim.c:1:10: fatal error: 'openssl/hmac.h' file not found
#include <openssl/hmac.h>
         ^
1 error generated.
thread '<main>' panicked at 'explicit panic', /Users/matthew/.cargo/registry/src/github.com-0a35038f75765ae4/gcc-0.3.20/src/lib.rs:772

Did a brew install openssl then re-ran cargo build but the build failed again.  Ended up having to run brew link --force openssl then cargo build.

The above changes were insufficient to get things to work with actually using Rusoto and integration tests.  Getting this error on running integration tests for sqs::
     Running `/Users/matthew/Documents/rusoto/target/debug/sqs-541ba884893eed25`

running 1 test
test main ... FAILED

failures:

---- main stdout ----
	thread 'main' panicked at 'assertion failed: `(left == right)` (left: `987357026`, right: `1`)', /Users/matthew/.cargo/registry/src/github.com-0a35038f75765ae4/openssl-0.7.1/src/crypto/hmac.rs:96

https://github.com/sfackler/rust-openssl/pull/285/files appears to be the ticket for El Capitan:
brew install openssl
export OPENSSL_INCLUDE_DIR=`brew --prefix openssl`/include
export OPENSSL_LIB_DIR=`brew --prefix openssl`/lib

The above export commands need to be present whenever building the openssl crate, including building Rusoto after a cargo clean.


Add test scaffolding for generated SDK

This PR was copied from sweepai-dev#3 and generated by Sweep.
Resolves #490.

Description
This PR adds test scaffolding for the generated SDK in the fern-typescript repository. The goal is to provide a basic set of tests for each generated module, making it easier for developers to write tests for their specific use cases.
Summary

Added a Jest configuration file (jest.config.js) at the root of the project to define the testing environment for the SDK.
Modified the SdkGeneratorCli.ts file to automatically generate a test file for each generated module. The test file is placed in a corresponding __tests__ directory within the module directory.
Created a base test file (sdk-basic-auth.test.ts) in the __tests__ directory of the fern/basic-auth module. This file serves as a template for all generated tests and contains a basic test that checks if the module exports are defined.
Added a test case in the generate.test.ts file to verify that a test file is created for each generated module and that the test file has the correct structure.

Please review and merge this PR to enable test scaffolding for the generated SDK in fern-typescript.
Fixes #490.
To checkout this PR branch, run the following command in your terminal:
git checkout sweep/feature/test-scaffolding

Hi @amckinney, I'm working on this here: https://github.com/sweepai-dev/fern-typescript/pull/3
Apologies in advance if the PR has some issues, I'm spending some time to fix it. If you install Sweep it can directly leverage the failing GHA.

Apologies for the bad PR, we'll try again next time with a better description.


[Feature ⭐] Delete forbidden functions, formatting norminette in engine, graphics source files.

Feature type
Misc
Implementing process / Goal
forbidden function reported so far :
1.
include <assert.h> _in raw_data.c_

2.
for loops _in raw_data.c_

3.
for loops _in renderer/init.c_

4.
too many functions _in raycast.c_

5.
for loops in _minimap.c_

Related code
No response
Note
Huge-scale construction is required

raw_data.c done.

renderer/init.c done.

--raycast.c
divided to
++raycast.c
++raycast_tool.c
++raycast_floor.c
++raycast_wall.c

minimap.c done

world::deinit method added.


fix: Additional args

:tada: This PR is included in version 1.0.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Support manual signingStyle

Checklist

[x] I've run bundle exec rspec from the root directory to see all new and existing tests pass
[x] I've followed the fastlane code style and run bundle exec rubocop -a to ensure the code style is valid
[x] I've read the Contribution Guidelines
[x] I've updated the documentation if necessary.

Motivation and Context

Fix for #10470
Description

in this comment I explained how export works in Xcode 9. I implemented it in this pull-request

@googlebot check again pls

Hey @threat70, thanks for the PR, we already have https://github.com/fastlane/fastlane/pull/10345, however we haven't decided yet how to go ahead with it yet. What do you think?

@KrauseFx Hello! :)
in #10345 you allow to update provisioning profiles automatically (xcodebuild will create xcode managed provisioning profiles and use them for build). But in implementation you should consider that user doesn't want to use xcode managed provisioning profiles in some cases. e.g. I specify provisioning profiles mapping and gym will not use them without signingStyle=manual. in the same time flag, that allows to update xcode managed provisioning profiles, have no any sense with manual signing.
Sry, that some tests left broken, I will fix as soon as I can :)

Yeah, totally, but if you look at the discussion, you'll see that this might cause issues if the user isn't logged in in Xcode at all, or with the wrong Apple ID.

@KrauseFx
oh, I missed the main point in discussion, sorry. So maybe I will remove -allowProvisioningUpdates, and I'll leave manual signingStyle in case where provisioning profiles map(PPM) is given, it should be correct, because without manual signingStyle PPM will be unused. What you think about it?

Does it even use the PPM for signing? Gym still uses my project defined PPM and certs even if I defined the signingStyle to manual

@Kalvin126 , Hi! Could you explain how you define the signingStyle?

@googlebot check cla pls

  gym(
    workspace: '.xcworkspace',
    scheme: '<Scheme>',
    configuration: 'Release',
    export_options: {
      method: "enterprise",
      signingStyle: "manual",
      team: "<Team ID>",
      provisioningProfiles: {
        "<Identifer>": "<Profile>"
      },
      signingCertificate: "iPhone Distribution: <Company>"
    }
    )```

Is there something I have to enable to make it generate the output plist? It not in Library/Logs/gym/

This +1000
Deploying asap on our local gems repo 👍
Reduced our xcodeproj file messing and PROVISIONING_PROFILE_SPECIFIER juggle.
to just relying on Match and Appfile for Team and Provisioning swapping

@KrauseFx tbh this is the way to going at least when working with match

app_identifier is read from Appfile so we can support different values for each lane!
Then added as env
ENV['APP_IDENTIFIER'] = CredentialsManager::AppfileConfig.try_fetch_value(:app_identifier)

Build lane looks like this now
Once the next xcodeproj comes out, I should be able to drop xcargs
https://github.com/CocoaPods/Xcodeproj/pull/510
  lane :build do |options|
    scheme = options[:scheme] || ENV['scheme']
    configuration = options[:configuration] || 'Debug'
    type = options[:type] || 'development'
    app_identifier = options[:app_identifier] || ENV['APP_IDENTIFIER']
    git_branch = options[:git_branch] || 'widex'
    if is_jenkins
      setup_jenkins
      FileUtils.rm_rf('../output')
      ENV['FL_BUILDLOG_PATH'] = 'fastlane/logs'
      bump_build_number
      app_identifiers = [
        app_identifier,
        "#{app_identifier}.watchkitapp",
        "#{app_identifier}.watchkitapp.watchkitextension",
      ]
      match(
        type: type,
        readonly: true,
        git_branch: git_branch,
        app_identifier: app_identifiers
      )
    end
    pods
    gym(
      scheme: scheme,
      configuration: configuration,
      xcargs: "APP_IDENTIFIER=#{app_identifier}"
    )
  end

What are the chances for this to be the next release?

#10670 why u no include? 😐

I think this PR introduced a bug in that signingStyle is always explicitly set to manual now and there's no way to get it to be automatic. Calling gym with export_options: { provisioningProfiles: nil, signingStyle: "automatic" } still results in signingStyle manual in the generated plist.
New issue is: https://github.com/fastlane/fastlane/issues/10778

Another regression - #10685


BRDF Explorer GUI

We can discuss and update progress on GUI development of BRDF Explorer extension here.
Developer: @Florastamine
Adviser: @manhnt9 @devshgraphicsprogramming
Screenshots: to be updated when there's new progress from @Florastamine
Notes after discussion on Discord:

Enable sRGB textures in CEGUI
Color Picker: CEGUI's class, add color wheel if possible (low priority)
File dialog: nativefiledialog library
Theme: TaharezLook or AlfiskoSkin

Task list made by @Florastamine:

[x] Remove the loaded resource paths list box
[x] Change light & material parameters into sliders
[x] Review slider ranges
[x] Fix roughness sliders & add a lock checkbox
[x] Make the AO & bumpmapping thumbnail click-able
[x] Add an option to select between tweakable shader constants or from texture
[x] Improve the texture preview window
[x] Transparent panels & make the GUI optionally toggle-able
[x] Replace WindowsLook theme with a better fit one (preferably dark, blue-ish)

How to integrate CEGUI into IrrBAW:

Study Example 33 to understand extension architecture
Write a CEGUI manager class as extension
Fork IrrBAW, checkout branch stable-ish
Create another IrrBAW example for BRDR Explorer

How to render CEGUI with IrrBAW:
Use this code snippet for GUI rendering
auto renderState = irr::video::COpenGLState::collectGLState();
irr::video::COpenGLState guiState;
executeGLDiff(guiState.getStateDiff(renderState));
CEGUI::System::getSingleton().renderAllGUIContexts();
executeGLDiff(renderState.getStateDiff(irr::video::COpenGLState::collectGLState()));

See more about how to use IrrBAW's OpenGL state manager at this file.
Notes:

Only use stable-ish branch, will try to merge in to unstable master later (need DevSH's advices)
Commit with CEGUI datafiles (filter out what is used)

This is my provisional and not yet complete list of CEGUI themes I find attractive:

https://sourceforge.net/projects/glossyserpent/ (last resort)
Whats the theme that "CEGUIGameMenuDemo" uses?
Taharez Look (its dark)

Btw, what does CEGUI use for font rendering?

I think you're missing a drop-down list for the BRDF models, which are outlined in the PDF spec of the BRDF Explorer (I think there are about 5 or so).

Change roughness/metallic/refraction spinners into sliders

I personally think that sliders would be good for colors as well ( in addition to color picker), but if that is impossible for some reason, let me know.

I would make it so that when the image slot preview (in the resource dock window) is clicked on, the file dialog is opened instead of the current set-up.
Remember that the BRDF explorer states the following:

Give the User a choice whether values should come from textures or be ”constants”
tweakable with GUI sliders

(can use a dropdown list with 5 options, "Constant", "Texture 0", "Texture 1", ...)
(you can show the texture preview when its a texture, and 4 sliders when its a constant)

Textures are bound to channels, channels can be bound to shader parameter inputs. No textures directly to inputs, shader input can only choose texture channel.

Light position can stay as a spinner.
The roughness value will need two sliders, and its a bit complicated:

a ”lock” checkbox that makes two roughness sliders locked together so both produce
the same value (grey it out when isotropic [note: only anisotropic BRDF is Ashkimin Shirley] BRDFs are used).

The bumpmap height/scale parameter is the only one that does not have a 0-1 range, it needs a bit extra range (like 1-20 or something like that).
P.S. I know I'm being picky, so forgive me for this comment

I think the texture buttons should be arranged into a 2x2 quad like in Shader Maker

The CEGUI GameGUI demo uses something else (it's own theme), we can try that.
This is TaharezLook .
CEGUI uses FreeType for font rendering.

Looks like a perfectly acceptable skin for the final product.

You mean TaharezLook or GameMenu theme?

@manhnt9 asked for a list of BRDF/BSDFs (I presume for the drop-down lists).
Specular:

GGX Diffuse&Conductor Mix
Ashikhmin-Shirley Diffuse&Conductor Mix (that one needs the coupled roughness sliders to be "freed" to be independent)
Perfect Dielectric
GGX Rough Dielectric
Thin Dielectric

Diffuse:

Orren-Nayar
Lambert
Disabled (nice to display when greyed out)

Note: Who's to implement those is yet to be decided.

Should it be a check boxes instead? So viewer can see combined models, maybe?

No, you pick one BRDF for diffuse and another for specular, there are no other combinations that make sense.

OK, so there'll be 2 dropdown lists. I was thinking about the break-down of the components for a fixed BRDF algorithm. So we'll have multiple ways of calculate the effects and visualize the final BRDR render, so the dropdown is like graphics setting. Interesting! I should learn these.

Diffuse and Specular are orthogonal to each other, one is short range scattering the other is reflection.

I've found papers of Lambert, Oren-Nayar, GGX, Ashikhmin-Shirley. Where do I learn about these below?

GGX Plastic&Conductor Mix (how to apply GGX to have it)
GGX Rough Dielectric
Perfect Dielectric
Thin Dielectric

GGX Metallic workflow:
https://seblagarde.files.wordpress.com/2015/07/course_notes_moving_frostbite_to_pbr_v32.pdf
The Dielectrics I'll do myself.

Update about theme choice:

TaharezLook
Or AlfiskoSkin

Current progress. The color picker is now available, along with a complete layout overhaul.

The currently used theme is a variant of AlfiskoSkin with custom modifications & additional widgets ported from the vanilla CEGUI skin. I tried sticking to imgui's default color scheme

For reference, here's the original AlfiskoSkin:

Checking the checkbox (heh!) will now lock one slider & enforce two sliders to be equal:

A drop-down list is now available and will disable GUI sliders if "Constant" is selected:

I've found papers of Lambert, Oren-Nayar, GGX, Ashikhmin-Shirley.

Can you post the links here?

Send to the one assigned to my GitHub account (viewable in profile page), thanks :)

@manhnt9 Can you post the links here?

Guys I made the bibliography in the Overleaf PDF for a reason, you can retrieve most references as links anyway.

@Florastamine can you give me the GUI prototype to play with?
Beautiful work.
I don't currently understand how you've done the whole "source" drop down.

@Florastamine its the "Roughness" that can be anisotropic, not the Refraction Index

I think @Florastamine can send a binary version for Matt to play with it. It's time to tune to GUI features. Visual is good now I think. Do you prefer Windows or Linux version?

Whatever is easiest for you.

@devshgraphicsprogramming Yes, I have just corrected that now

Light position can stay as a spinner.

Should be a numerical value per each XYZ coordinate that can be typed

@devshgraphicsprogramming Could you make an issue specifying what do you need for CEGUI sRGB texture and color picker?

open one and write down your questions, also give me links to CEGUI source files as reference ;)

I have updated the GUI to reflect necessary changes, please download included binaries & test them well.
BRDFExplorerGUI.zip
GUI

I'm missing libgcc.dll and libfreetype.dll etc., could you possibly make a statically linked build?

I hope I've got all possible dependencies sorted out... it's so monstrosity that making a statically build is a huge feat.
BRDFExplorerGUI.zip
It's bloated, but should run on anything. Just Windows things, I guess.

I'm missing: libCEGUIExpatParser.dll

Weird... I got the executable to run fine without these, but anyway, here are the files.
DLL.zip

Nice, it even runs on Linux with Wine.

A bug which made certain UI elements impossible to click through has been fixed, new files were issued. Please download it here.
BRDFExplorerGUI.zip

A bug which made certain UI elements impossible to click through has been fixed, new files were issued. Please download it here.

I think at this point a public git repo is in order ;)

We will publish the source and merge into irrbaw when the GUI part is marked as done. Then we concentrate on the BRDF part and GUI should be touched as little as possible. It helps us to assure the overall integrity.

btw @Florastamine I can no longer download the zip

@devshgraphicsprogramming I've updated the expiration date to a maximum of 1 week. Please try to download the file again.
BRDFExplorerGUI.zip

I still get a 404

BRDFExplorerGUI.zip

Ok my comments from testing:

The example app doesn't like my dual monitor set-up, everything is scaled down badly and not very legible (usually not a problem, as integration will be into irrBAW)
The 2x2 grid of textures is not clickable (no file load dialog from click)
not all GUI gets hidden by F1
its possible to crash the color picker by clicking on its window titlebar repeately
when isotropic roughness is set, the background of the fader fails to update
MapHeight is an integer slider, should allow for fractions between 0 and 1 as well
The Color Picker Preview and "Choose Color ..." GUI elements should not blend against the background (the viewed colours are wrong when blending the color previous against the background)

If you address the above then I believe we're done on the GUI design front.

The example app doesn't like my dual monitor set-up, everything is scaled down badly and not very legible (usually not a problem, as integration will be into irrBAW)

It'd be nice if you could provide a screenshot & your resolution. I personally don't have a lot of resources to test against every single resolution & monitor setup, but I'll see what I can do. Could also be the GUI that is faulty.
not all GUI gets hidden by F1

Do you mean this particular sliders window? It's something I put in just for the demo to easily set background colors, i. e. for testing purposes.

The rest should be trivial to fix.

A new build is now live.

Apart from the fixes, the text fields in both AO & bump-mapping window now accept manual input & would pop out an error message if the file couldn't be found.

The texture preview window was also improved to display its child textures' name.
There are two known issues remain, both of which is CEGUI's: Crash on double-clicking the ColourPicker's titlebar, CEGUI::ColourPickerControls doesn't respect CEGUI::ColourPicker::setVisible().
BRDFExplorerGUI.zip

lovely

Minimal issues:

Native file dialog starts off with *.bmp mask so when I browse my folders I can't see jpegs and pngs all at once, maybe start with .*
The FPS is rather low (only 30 FPS) , I don't need it fixed I just need to know why
The background behind sliders (the slider gradient has issues updating)

P.S. It's sad that the rounded corners have had to be sacrificed, but those buttons could be made a bit smaller (now they feel feally big compared to everything else.

The FPS is rather low (only 30 FPS)
Weird. I got 60 with V-Sync and 5,000 uncapped.

The background behind sliders (the slider gradient has issues updating)
There's nothing I could do about that I'm afraid. For reference, the original Alfisko also exhibits the same inconsistent behavior when it comes to updating the sliders:

It pretty much happens at random and the only way to get the sliders updated correctly is to drag another slider, but then the dragged one is also affected. I really have no clue if it's CEGUI being iffy or the theme itself. Might as well investigate later.
The file dialog issue has been fixed. Additionally, tooltips have been implemented for certain widgets (the texture buttons, for example, which can show additional information when hovering on them):

BRDFExplorerGUI.zip

I have 50-60 FPS and 5000 max on Linux with Wine.

There's nothing I could do about that I'm afraid. For reference, the original Alfisko also exhibits the same inconsistent behavior when it comes to updating the sliders:
image
It pretty much happens at random and the only way to get the sliders updated correctly is to drag another slider, but then the dragged one is also affected. I really have no clue if it's CEGUI being iffy or the theme itself.

File an issue.
ok, we're done here then, lets dump the source to this somewhere and begin making example 13.BRDFExplorer

Okay, I'll do a quick source code review and integrate into the example in this repo.

Three known (non-blocking) bugs remaining:

Crash on double-clicking the ColourPicker's titlebar
CEGUI::ColourPickerControls doesn't respect CEGUI::ColourPicker::setVisible()
AlfiskoSkin does not correctly update sliders' background

Although given the authors' stance on various issues over the years (lack of manpower and such), we'd most likely have to fix them ourshelves somewhere down the line.

Yes, my plans also include CEGUI development so there'll be at least 1 person who helps fighting CEGUI issues in near future.


Improve coverage of failure modes for performSelection.

Issue Number
ADP-1037
Summary
This PR:

adjusts genOutputsToCover so that it occasionally generates outputs with token quantities that exceed the maximum allowable token quantity (maxBound :: Word64).
adjusts genUTxOAvailableForInputs so that it occasionally generates UTxO sets that are completely empty.
rebalances the minimum coverage requirements for failure modes of performSelection.

bors r+


Add github workflow to verify docs

@javiereguiluz and @wouterj may have noticed that I found a few invalid use statement recently. I was trying out my script.
This PR will make sure that Carson is opening an issue when there are invalid use statements.
TODO:

[x] Write logic to open/update an issue. (#106)

I'll wait for other PRs to be merged to avoid conflicts. But I just wanted to show my intent.

Hi @Nyholm. I like your excitement for improving Carsonbot!
However, I think it's better to add this rule to https://github.com/OskarStark/doctor-rst . It's the CI tool we're using the docs repository to check for all sorts of things (including PHP stuff, like alphabetically sorted use-statements). This way, we're immediately reminded of the error when it's introduced.
What do you think?

That is an excellent question. I’ve considered that too. But I decided not to do that.
Most of the invalid use statements in the docs is because the symfony source changed. Ie, the docs were valid at the time of merge.
If we added the check to the docs CI that would lead to the next PR made to the docs had red CI because unrelated changes.
This test is also relatively slow and most of the time it will not report any issues.

I honestly am still not very sure about including this in carsonbot. I think the scope of carsonbot is management of PRs and issues (labelling, milestoning, commenting, etc). The scope of PhpUnit and DOCtor-rst is to validate the repository sources.
I agree that it might not be nice to break someones pull requests because of a change not made by them. We could make this job branch-only (so pull requests don't break). Also, classes in Symfony aren't suddenly moved. That only occurs on major releases and have a deprecation before. That means that there has been a doc issue about the move already, before the build actually breaks.

I would like to extend that scope. I would like Carson bot to also open issues like: https://github.com/symfony/symfony/issues/38771 Currently it is another bot/script that opens that issue.
I also think this is not going to happen very often. It is only when we introduce new big features and refactor them. Ie, the RateLimit component was refactored a bit after the docs got merged.
At the very most, you will have one issue open on symfony/symfony-docs. And I dont really expect the bot to open more than 1 or 2 issues in 2020.
The reason why I think this is important is because it is really really hard to verify this manually.

@Nyholm I think this would be better handled by a workflow running in the symfony-docs repo (based on a cron trigger) that would use https://github.com/symfony-tools/code-block-checker

Im closing this in favour of https://github.com/symfony-tools/code-block-checker/pull/38
That would make this feature part of the CI instead of Carson.


Separate danmaku from subtitles

Checklist

[X] I'm requesting a feature unrelated to a specific site
[X] I've looked through the README
[X] I've verified that I'm running yt-dlp version 2022.07.18 (update instructions) or later (specify commit)
[X] I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue

Provide a description that is worded well enough to be understood
reason:

danmaku is not subtitles, and do not have lang-attr, danmaku do not distinguish users who speak different languages, danmaku data may have all languages.
danmaku often has too many to show as subtitles, it needs other display mode.
danmaku data do not follow subtitle schema, which is hard to convert to srt or else.
danmaku is often changed or is emptied periodically by site, do not like subtitles.
treat danmaku as a subtitle make current subtitle download confusing, Now I need to add --sub-langs en*,zh* for all site has danmaku (like bilibili) to skip danmaku but download subtitles, I think the default behavier should be no-danmaku when download subtitles, and use --download-danmaku to explicitly specify.

danmaku is not subtitles, and do not have lang-attr, danmaku do not distinguish users who speak different languages, danmaku data may have all languages.

The reason that Danmaku (e.g. so-called comments from NicoNico) is treated as subtitles is that, it has timeline.
We do understand that there can be any language in it, like Chinese and Japanese mixed in a video.
The same goes for live chat from YouTube. It has timeline, too. Comments on videos doesn't.

danmaku often has too many to show as subtitles

Yes :)

it needs other display mode.

yt-dlp itself doesn't render. It's the responsible for the player to render.

danmaku data do not follow subtitle schema, which is hard to convert to srt or else.

Here's a go: https://github.com/m13253/danmaku2ass
Yes. It doesn't follow any schema so you'll have to convert by yourself.

danmaku is often changed or is emptied periodically by site, do not like subtitles.

Any example that yt-dlp supports?
On NicoNico, comments will expire and replaced by new ones, but did you mean this?

treat danmaku as a subtitle make current subtitle download confusing, Now I need to add --sub-langs en*,zh* for all site has danmaku (like bilibili) to skip danmaku but download subtitles, (snip)

What Bilibili provides is internally treated as "comments" (but it has timeline too right??? confusing) so it doesn't apply for now.
The possibly only case, NicoNico, declares the language as comments (so the "comments" language). So unless you're specifying Corsican (--sub-langs co*), it doesn't download.

danmaku is often changed or is emptied periodically by site, do not like subtitles.

Any example that yt-dlp supports?
On NicoNico, comments will expire and replaced by new ones, but did you mean this?

Bilibili, comment update logic is complex, and I do not know it accurately.

What Bilibili provides is internally treated as "comments" (but it has timeline too right??? confusing) so it doesn't apply for now.
The possibly only case, NicoNico, declares the language as comments (so the "comments" language). So unless you're specifying Corsican (--sub-langs co*), it doesn't download.

I add real subtitle support for bilibili in https://github.com/yt-dlp/yt-dlp/pull/4221
Bilibili now have subtitles and danmaku.
Maybe I should change it to comments?
I think it may still be downloaded when use --write-subs

Maybe I should change it to comments?

No

I think it may still be downloaded when use --write-subs and there is no real subtitles

Doesn't matter. User should control this with --sub-langs as they need

We won't add new options for danmaku, live chat etc. Also, changing them from being subtitle breaks backward compat.
The most that can be done regarding your request is to extend --compat-option no-live-chat to exclude danmaku as well

get it


MRG: More strict check in simulation

Adds a check that labels in simulate_stc do not overlap. If they did, in master the error would be cryptic:
  File "/home/larsoner/custombuilds/mne-python/mne/simulation/source.py", line 219, in simulate_stc
    stc = SourceEstimate(data, vertices=vertno, tmin=tmin, tstep=tstep)
  File "<string>", line 2, in __init__
  File "/home/larsoner/custombuilds/mne-python/mne/utils.py", line 629, in verbose
    return function(*args, **kwargs)
  File "/home/larsoner/custombuilds/mne-python/mne/source_estimate.py", line 975, in __init__
    verbose=verbose)
  File "<string>", line 2, in __init__
  File "/home/larsoner/custombuilds/mne-python/mne/utils.py", line 629, in verbose
    return function(*args, **kwargs)
  File "/home/larsoner/custombuilds/mne-python/mne/source_estimate.py", line 443, in __init__
    raise ValueError('Vertices must be ordered in increasing '

Now it's:
  File "/home/larsoner/custombuilds/mne-python/mne/simulation/source.py", line 208, in simulate_stc
    'they must be non-overlapping' % (d, hemi))
RuntimeError: Labels had 100 overlaps in the right hemisphere, they must be non-overlapping

cc @nfoti

[Current coverage][cc-pull] is 86.23%

Merging [#3315][cc-pull] into [master][cc-base-branch] will increase coverage by 0.01%

@@             master      #3315   diff @@
==========================================
  Files           336        336
  Lines         57252      57261     +9
  Methods           0          0
  Messages          0          0
  Branches       8712       8714     +2
==========================================
+ Hits          49363      49378    +15
+ Misses         5220       5216     -4
+ Partials       2669       2667     -2

[][cc-pull]

Powered by Codecov. Last updated by [c0e73ee...fca86fa][cc-compare]
[cc-base-branch]: https://codecov.io/gh/mne-tools/mne-python/branch/master?src=pr
[cc-compare]: https://codecov.io/gh/mne-tools/mne-python/compare/c0e73eeb959a636fb25a6f4ca550ca3111f05852...fca86fa12586c0d0d5aa6d6be9f6716c8e126991
[cc-pull]: https://codecov.io/gh/mne-tools/mne-python/pull/3315?src=pr

this is working for me now @Eric89GXL . Thanks!  I'll let you know if it breaks :)

@andrewheusser I assume you're talking about the changes in #3312 but errantly replied here?

oops! sorry about that :(

yes :)

You should have said "no", then I would have been impressed with how quickly you went from having trouble importing mne to hitting a corner-case bug with our simulation code :)

haha, someday!

thx @Eric89GXL


Update parse_zone_file.py

Description
See internal ICM 187136418
We are failing to import zones when domain name in $origin contains hypens.
This is because we don't parse it correctly in the regex (only alphanumeric char are evaluated with "\w" and that doesn't include hypen, like it is for outher resurces like CNAME.
Testing Guide

History Notes

[Component Name 1] BREAKING CHANGE: az command a: Make some customer-facing breaking change.
[Component Name 2] az command b: Add some customer-facing feature.

This checklist is used to make sure that common guidelines for a pull request are followed.

[ ] The PR title and description has followed the guideline in Submitting Pull Requests.

[ ] I adhere to the Command Guidelines.

add to S169

/azp run Azure.azure-cli

/azp run

/azp run

/azp run Azure.azure-cli (Check the Format of Pull Request Title)

/azp run Azure.azure-cli

/azp run Azure.azure-cli

/azp run Azure.azure-cli


Migrated upload controller to v2

refs #9866

@kirrg001 didn't add any unit tests here as there's nothing to do in output serializer :man_shrugging:


Error - Python Package Installation

Environment info
Operating System: Windows 10
Compiler: Visual Studio 15 2017 Win64
Package used (python): Anaconda 5.0
xgboost version used: latest
Steps to reproduce
I have used this nice guide to intall xgboost with VS 2017.
Everything worked fine. The build was successful and I have a file xgboost.dll in the lib folder.
The problem arises when I run python C:\Python\xgboost\python-package\setup.py install from the Anaconda Prompt.
Here is the error:
Install libxgboost from: ['..\\lib\\xgboost.dll']
running install
running bdist_egg
running egg_info
writing xgboost.egg-info\PKG-INFO
writing dependency_links to xgboost.egg-info\dependency_links.txt
writing requirements to xgboost.egg-info\requires.txt
writing top-level names to xgboost.egg-info\top_level.txt
reading manifest file 'xgboost.egg-info\SOURCES.txt'
writing manifest file 'xgboost.egg-info\SOURCES.txt'
installing library code to build\bdist.win-amd64\egg
running install_lib
warning: install_lib: 'build\lib' does not exist -- no Python modules to install

installing package data to build\bdist.win-amd64\egg
running install_data
error: can't copy '..\lib\xgboost.dll': doesn't exist or not a regular file

What have I tried?

Copying the dll to python-package folder. Same error.

Solved. I didn't go the the python-package directory. I just made reference to the setup.py file.
cd python-package and then:
python setup.py install
did the job!


Network connection unusable when using kubelogin

Hi, I've been trying to work with kubelogin because the warning message drives me crazy.
Whenever I convert my kubeconfig file and try to log in using device code authentication, I receive timeouts, and my network connection becomes unusable. When connecting without kubelogin, everything works, but I get the warning message with every kubectl command.
I'm running on a Mac with an Apple Silicon processor and macOS Monterey 12.6.
"azure-cli": "2.41.0",
"azure-cli-core": "2.41.0",
"azure-cli-telemetry": "1.0.8",
"extensions": {
"aks-preview": "0.5.112"
}
kubelogin version
git hash: v0.0.20/872ed59b23e06c3a0eb950cb67e7bd2b0e9d48d7
Go version: go1.18.5
Build time: 2022-08-09T18:33:28Z
Platform: darwin/arm64

The issue was related to the connection and DNS.


Eval is like running forever

Hello author,
Thanks for your tutorial.
I am using the dataset hf-codegen-v2 which has 370k rows.
The validation set is about 1850. The batch size is 4. For other params, they are the same as the ones in run_peft.sh.
The training speed is normal but the eval loop is running forever.
Below is the log for evaluation:
{'eval_loss': 0.1817416101694107, 'eval_runtime': 6666.0306, 'eval_samples_per_second': 8.072, 'eval_steps_per_second': 2.018, 'epoch': 0.5}
I am not sure if this is normal.
Any help will be appreciated!
ZD

same here but have no idea why


Grace period issue: doest not work after to first grace period is done

I have set up google-authenticator-libpam with the following configuration:
auth required pam_google_authenticator.so grace_period=86400
First time I connect I see a line with the IP and a timestamp added to .google_authenticator like so:
LAST0 ip-X-X-X-X<PHONE_NUMBER>
In this situation everything works as I expected, the second time I login no verification code is required.
After the grace period is done, in this example after 1 day, there is a new line like so:
" LAST1 ip-X-X-X-X<PHONE_NUMBER>
(with the same IP of course)
what I excepted was that after the new login with a correct timestamp I would be able to login again with no mfa for the next 24 hours, however the result is that every new login there is a new line with the same IP and a new timestamp and I keep getting asked to put the verification code.
when I clear the .google_authenticator again, removing all lines of IPs I get another day that the grace period is working.
I could have a cronjob clearing the ip lines everyday but I find it really ugly.
Is this the expected behavior of grace_period or is this a bug  ?
Thanks

Yeah that sounds wrong. Let's see. Indeed within_grace_period doesn't support multiple lines for the same IP.
But it clearly does create a new entry.

That's strange. When you say "ip-X-X-X-X" what format do you mean?
I see that when it's localhost the host name used is actually "localhost", which didn't work until I fixed it. Could you retain the original format of the lines you see in ~/.google_authenticator, but changing numbers (not into Xs) for your anonymity?
Could you also check /var/log/auth.log or similar to see if you get any error messages there. You should be able to grep for pam_google_authenticator.

This fixes localhost, at least:
$ git diff
diff --git a/src/pam_google_authenticator.c b/src/pam_google_authenticator.c
index 64c9424..82d09da 100644
--- a/src/pam_google_authenticator.c
+++ b/src/pam_google_authenticator.c
@@ -1560,7 +1560,7 @@ update_logindetails(pam_handle_t *pamh, const Params *params, char **buf) {
     char host[40]; /* Max len of ipv6 address is 8*4 digits plus 7 colons.
                     * Plus trailing NUL is 40 */
     unsigned long when = 0; // Timestamp of current entry.
-    const int scanf_rc = sscanf(line, " %39[0-9a-fA-F:.] %lu ", host, &when);
+    const int scanf_rc = sscanf(line, " %39[0-9a-zA-Z:.-] %lu ", host, &when);
     free(line);

     if (scanf_rc != 2) {

This is the full format of what I see
with 172-00-11-22 being my ip.
" LAST1 ip-172-00-11-22.ec2.internal<PHONE_NUMBER>
" LAST0 ip-172-00-11-22.ec2.internal<PHONE_NUMBER>
I do see the following error:
sshd(pam_google_authenticator)[1849]: debug: google_authenticator for host "ip-172-00-11-22.ec2.internal"
sshd(pam_google_authenticator)[1849]: Malformed LAST0 line

Ah. I see. Yes that explains it. Are those addresses in /etc/hosts on your machine? I hope that's not from DNS as that could make attacker be able to choose address. (unless it's looked up both ways before being accepted)
In any case the patch in my previous comment should solve it for you.

I'll submit the patch when I'm at the right laptop to do so. :-)

Actually im not sure why when I login the server takes my IP and turns it into this name. its could be some AWS related settings
anyway it looks like the change that you suggested worked !
Thanks so much for the quick reply !


Terminate a background process launched with sudo?

I get a permissions error when attempting to terminate a background process using sudo, even when I try to terminate the process (using RunningCommand.terminate()) from within a sudo context. For example:
    with sh.contrib.sudo(password='secret', _with=True):
        p = sh.background_task(_bg=True)
    with sh.contrib.sudo(password='secret', _with=True):
        p.terminate()

Gives me "PermissionError: [Errno 1] Operation not permitted". Is there a better way to kill a process that was spawned as root?
Hackish workaround
I can kill the process in a less cosmetic way by using the bash kill command and swallowing the resultant exception from the background process like this:
    with sh.contrib.sudo(password='secret', _with=True):
        p = sh.background_task(_bg=True, _bg_exc=False)
    with sh.contrib.sudo(password='secret', _with=True):
        sh.kill(p.pid)

I can't reproduce this on the master and the develop branch.  I'm using the following code:
import sh
with sh.contrib.sudo(password='secret', _with=True):
    p = sh.sleep(30, _bg=True, _bg_exc=False)

p.kill()
p.wait()

And it works fine. Please re-open the issue if you can reproduce the problem with real commands that I can try.


Docs : broken "nodes" page

Describe the issue
The nodes page, as opened by clicking "Nodes" on this page, is seemingly broken.
Environment
Does not apply / Web
To Reproduce
Steps or code to reproduce the behaviour:

Go to this documentation page
Click on the "nodes" title
the page, is seemingly broken.

Expected behaviour
Readable layout
Screenshots

Thanks for spotting this! Let me fix it

It is fixed now on this page

I can confirm that. Thanks for this great project !


requirement for cytoolz==0.8.2

cytools 0.8.2 is not working with PyThon 3.6 and newer. Can we change the requirement to >=

@brett-schneider We don't work on microraiden any more, but you're welcome to open a PR.

ah ok, i've done my setup using full raiden now, but i'll keep that on the backlog


Update pipeline-service in production

Upgrade OSP from 1.11 to 1.12
Update Grafana dashboard
Update metrics exporter

/lgtm


Replace exclusive locks with rwlocks in hostdb

It was observed that changing to rwlocks significantly improved hostdb performance.

Especially for use cases where few names are used.

[approve ci autest]


Enhancement: Click dynamic GridContextMenu

How about integrating a helper for clicking a dynamic context menu?
I found a way with:
val tree = _get<TreeGrid<ProductTreeItem>>()
val collateralDetail = tree._get(2)
collateralDetail.name() shouldBe "CollateralDetail (Empty)"
val key = tree.dataCommunicator.keyMapper.key(collateralDetail)
val data = JreJsonFactory().parse<JsonObject>("{ \"event.detail\":{\"key\":\"${key}\"}}")
tree._fireDomEvent("vaadin-context-menu-before-open",data)
view.contextMenu._clickItemWithCaption("Add", collateralDetail)

Hi, thank you for letting me know :+1: However I'm not sure what this code will solve - could you please provide a bit more details, e.g. what is the reason behind firing vaadin-context-menu-before-open, and in what kind of use-case this would be helpful?

@mvysny the reasoning behind this is that a dynamic context menu needs to be initialized like
contextMenu= tree.addContextMenu()
contextMenu.setDynamicContentHandler { item ->
        if (item == null) {
            false
        } else {
            contextMenu.removeAll()
            item.fillContextMenu(contextMenu, tree)
        }
    }

firing the event does invoke the dynamicContentHandler, so the the MenuItems get initialized. Otherwise the menu would be empty.
fun <T> GridContextMenu<T>._clickDynamicContextMenuItemWithCaption(
        caption: String,
        tree: Grid<T>,
        item: T) {
    val key = tree.dataCommunicator.keyMapper.key(item)
    val data = JreJsonFactory().parse<JsonObject>("{ \"event.detail\":{\"key\":\"${key}\"}}")
    tree._fireDomEvent("vaadin-context-menu-before-open", data)
    this._clickItemWithCaption(caption, item)
}

Excellent idea, thank you :+1: I'm even thinking of always firing the vaadin-context-menu-before-open DOM event from _clickItemWithCaption, in order to simulate the Vaadin initialization more closely. I'll prototype this.

Actually the easier way to invoke the DynamicContentHandler is to call it directly via contextMenu.getDynamicContentHandler(). I've modified the _clickItemWithCaption() functions to call the DynamicContentHandler directly.

Looks great!

Excellent, thanks for letting me know :) Released as Karibu-Testing 1.1.27