[SPARK-10890][SQL] don't log expected JdbcSQLException during incompatible Insert test

SPARK-10890: "Column count does not match; SQL statement:" error in JDBCWriteSuite
The org.h2.jdbc.JdbcSQLException is logged by org.apache.spark.executor.Executor and again by org.apache.spark.scheduler.TaskSetManager, so we would need to influence their log behavior / log levels.
Absent any properties or API to do that,  the easiest way to suppress the logging of the expected exception is to temporarily increase the log threshold to FATAL while the test case is executed and restore the original log level right after. This could be done globally org.apache.log4j.RootLogger or just for org.apache.spark.executor.Executor and org.apache.spark.scheduler.TaskSetManager
I realize this is a "hack" with the side effect of suppressing any other log message of less than FATAL severity during the test case execution. Discussion invited :-)

Can one of the admins verify this patch?

Hi Christian,
The patch looks good to me. I would recommend putting the restoration of the logging state inside a finally block. I don't know if that will change the behavior of this test, but it seems like good future-proofing in case someone else wants to copy your approach later on.
Thanks,
-Rick

@Rick, I was running the JDBCWriteSuite in IntelliJ and each test case got setup with log Level.DEBUG, even if an unexpected Exception prevented resetting the log level to it's original.
But your's is a very prudent suggestion and I will push an update shortly -- Thanks

+1 @JoshRosen  --  Good suggestion! I pushed a new version.
Could you kick of a test build :mans_shoe:
Thank you!

Jenkins, this is ok to test.

Merged build triggered.

Merged build started.

Test build #44650 has started for PR 9345 at commit a2e236f.

Test build #44650 has finished for PR 9345 at commit a2e236f.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44650/
Test FAILed.

Merged build finished. Test FAILed.

Hi @JoshRosen, the failing unit test "Single command with -e" in CliSuite should not be impacted by my changes. On my machine I get the timeouts with or without my changes. Is this a known problem?
[info] *** 1 TEST FAILED ***
[error] Failed: Total 19, Failed 1, Errors 0, Passed 18, Ignored 1
[error] Failed tests:
[error] 	org.apache.spark.sql.hive.thriftserver.CliSuite
[error] (hive-thriftserver/test:test) sbt.TestsFailedException: Tests unsuccessful
...
[info] CliSuite:
[info] - Simple commands (1 minute, 35 seconds)
[info] - Single command with -e *** FAILED *** (2 minutes)
[info]   =======================
[info]   CliSuite failure output
[info]   =======================
[info]   Spark SQL CLI command line: ../../bin/spark-sql --master local --hiveconf javax.jdo.option.ConnectionURL=jdbc:derby:;databaseName=/home/jenkins/workspace/SparkPullRequestBuilder/target/tmp/spark-467fd8f3-b708-4fbc-8663-1486857bdb0d;create=true --hiveconf hive.metastore.warehouse.dir=/home/jenkins/workspace/SparkPullRequestBuilder/target/tmp/spark-a45eff3e-fc0f-461f-b658-471176d83655 --hiveconf hive.exec.scratchdir=/home/jenkins/workspace/SparkPullRequestBuilder/target/tmp/spark-869dc65d-560d-4df5-b6d4-7cd673974013 -e SHOW DATABASES;
[info]   Exception: java.util.concurrent.TimeoutException: Futures timed out after [2 minutes]
[info]   Executed query 0 "",
[info]   But failed to capture expected output "OK" within 2 minutes.

Jenkins, retest this please.

It's a known flaky test.

Merged build triggered.

Merged build started.

Test build #44663 has started for PR 9345 at commit a2e236f.

Test build #44663 has finished for PR 9345 at commit a2e236f.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/44663/
Test PASSed.

@ckadner Thank you for the PR. Since it does not really affect tests and the solution at here is kind of hacky, how about we close this PR for now and revisit it later if we have a better approach? Thanks!

Ping @ckadner, would you mind closing this PR per Yin's comment? Thanks!


Automatic links?

Is it possible to use this library and have plain URLs automatically become links like they do in GFM?

This is not supported by pulldown directly, but since this is essentially an exercise in text search, it's relatively straightforward to build directly on top of the library. An example of how this could be done below:
extern crate pulldown_cmark;

use std::io::Write as _;

use pulldown_cmark::{html, CowStr, Event, LinkType, Parser, Tag};
use regex::Regex;

static URL_REGEX: &str = r#"((https?|ftp)://|www.)[^\s/$.?#].[^\s]*[^.^\s]"#;

fn main() {
    let markdown_input: &str = "Hi! This is an URL: http://github.com. Another one is www.google.com.";
    println!("Parsing the following markdown string:\n{}", markdown_input);

    let autolinker = AutoLinker {
        iter: Parser::new(markdown_input),
        state: AutoLinkerState::Clear,
        regex: Regex::new(URL_REGEX).unwrap(),
    };

    // Write to anything implementing the `Write` trait. This could also be a file
    // or network socket.
    let stdout = std::io::stdout();
    let mut handle = stdout.lock();
    handle.write_all(b"\nHTML output:\n").unwrap();
    html::write_html(&mut handle, autolinker).unwrap();
}

enum LinkState {
    Open,
    Label,
    Close,
}

enum AutoLinkerState<'a> {
    Clear,
    Link(LinkState, CowStr<'a>, CowStr<'a>),
    TrailingText(CowStr<'a>),
}

struct AutoLinker<'a, I> {
    iter: I,
    state: AutoLinkerState<'a>,
    regex: Regex,
}

impl<'a, I> Iterator for AutoLinker<'a, I>
where
    I: Iterator<Item = Event<'a>>,
{
    type Item = Event<'a>;

    fn next(&mut self) -> Option<Self::Item> {
        let text = match std::mem::replace(&mut self.state, AutoLinkerState::Clear) {
            AutoLinkerState::Clear => match self.iter.next() {
                Some(Event::Text(text)) => text,
                x => return x,
            },
            AutoLinkerState::TrailingText(text) => text,
            AutoLinkerState::Link(link_state, link_text, trailing_text) => match link_state {
                LinkState::Open => {
                    self.state = AutoLinkerState::Link(LinkState::Label, link_text.clone(), trailing_text.clone());
                    return Some(Event::Start(Tag::Link(
                        LinkType::Inline,
                        link_text,
                        "".into(),
                    )));
                }
                LinkState::Label => {
                    self.state = AutoLinkerState::Link(LinkState::Close, link_text.clone(), trailing_text.clone());
                    return Some(Event::Text(link_text));
                }
                LinkState::Close => {
                    self.state = AutoLinkerState::TrailingText(trailing_text);
                    return Some(Event::End(Tag::Link(
                        LinkType::Inline,
                        link_text,
                        "".into(),
                    )));
                }
            },
        };

        match self.regex.find(&text) {
            Some(reg_match) => {
                let link_text = reg_match.as_str();
                let leading_text = &text.as_ref()[..reg_match.start()];
                let trailing_text = &text.as_ref()[reg_match.end()..];

                self.state =
                    AutoLinkerState::Link(LinkState::Open, link_text.to_owned().into(), trailing_text.to_owned().into());

                Some(Event::Text(leading_text.to_owned().into()))
            }
            None => Some(Event::Text(text)),
        }
    }
}

This produces the following output:
Parsing the following markdown string:
Hi! This is an URL: http://github.com. Another one is www.google.com.

HTML output:

Hi! This is an URL: http://github.com. Another one is www.google.com.

I hope that this answers your question. If it does not, please feel free to reopen this issue.

Seems to work, thanks!
I'm going to put this into a crate under CC0 unless you have a problem with that

That's cool with me. I release the code above to the public domain.

I know this issue is closed, but it would be really nice if this was an option that pulldown_cmark provided. Since it isn't, I'll copy the code from this issue (thanks for providing it :) ), but it seems like this is a sort of common thing to want, and it'd be nice if it came built-in, with sufficient warning next to the option that it may be slow/etc.

The code suggested here breaks for links with underscores in them.


[safeint] Update to 3.0.26 version

https://github.com/dcleblanc/SafeInt/releases/tag/3.0.26

thanks @RT222 !


thefuck fails to install

$ fin install https://github.com/oh-my-fish/plugin-thefuck
fin SAYS Installing 1 plugin
         ✔  thefuck                 https://github.com/oh-my-fish/plugin-thefuck
fish: Unknown command '__tf_updt'
~/.config/fish/conf.d/init.init.fish (line 2):   set -q __tf_func; or __tf_updt
                                                                      ^
from sourcing file ~/.config/fish/conf.d/init.init.fish
	called on line 565 of file ~/.config/fish/functions/fin.fish

in function “__fin_plugin_enable”
	called on line 360 of file ~/.config/fish/functions/fin.fish
	with parameter list “~/.config/fin/thefuck”

in function “__fin_install”
	called on line 201 of file ~/.config/fish/functions/fin.fish
	with parameter list “https://github.com/oh-my-fish/plugin-thefuck”

in function “fin”
	called on standard input
	with parameter list “install https://github.com/oh-my-fish/plugin-thefuck”

fin SAYS Done in 1s 338ms
$ fin install https://github.com/oh-my-fish/plugin-thefuck
fin WARN Nothing to install

I checked that fisherman can install thefuck, which is the case.

Thanks, it was a problem with how I was walking the init.fish function to generate a unique name for init.fish files.
Note: You can run fin omf/thefuck too.

@patrickdepinguin We have merged fin into fisherman and released it as 2.0 🎉
Check it out https://github.com/fisherman/fisherman


DSD-1521: Add sizeBasedOn prop to Image

Fixes JIRA ticket DSD-1521
This PR does the following:

Adds sizeBasedOn prop to Image component

How has this been tested?
Locally on Storybook
Accessibility concerns or updates

N/A
Checklist:

[ ] I have updated the Storybook documentation accordingly.
[ ] I have added relevant accessibility documentation for this pull request.
[ ] All new and existing tests passed.

Front End Review:

[ ] Review the Vercel preview deployment once it is ready.

I tested sizeBasedOn height with different aspect ratio on the Storybook With Controls page and it works correctly. Let me know if I am misunderstanding.

If I keep the src the same with the height of 40px but update the aspect ratio, it's no longer 40px. This is what I'm unsure of. If the aspect ratio is updated to "square", I get the following:

This matches the behavior of size based on width. The height of the image will be determined by the size prop.

Hmm yeah that makes more sense. Let's wait on Marty to review this but looks good otherwise.

Overall, this is working well, but I am seeing one issue.  When sizeBasedOn is set to "height" and the height of the actual image is less than the height set by the size value, the image does not scale up to match the size height value.  It doesn't go any larger than the actual image height.  A larger image will scale down to match the size height value, but a smaller image is not being scaled up.  Conversely, when sizeBasedOn is set to "width", a small image will scale to match the size width value.
I think this is what @EdwinGuzman was also seeing.


AttrJson::Model object not storing

I have the following base model:
class GlobalSetting < ApplicationRecord
  include AttrJson::Record
  include AttrJson::Record::QueryScopes

  attr_json :insurance, GlobalSettings::Insurance.to_type
end

with GlobalSettings::Insurance here:
module GlobalSettings
  class Insurance
    include AttrJson::Model
    attr_json_config(bad_cast: :as_nil)

    attr_json :percentage, :integer
    attr_json :minimum, :integer
  end
end

When I attempt to create a new object and save it, it doesn't save the json data:
[1] pry(main)> gs = GlobalSetting.new insurance: GlobalSettings::Insurance.new(percentage: 15, minimum: 8)
=> #<GlobalSetting:0x00007fbb16766190
 id: nil,
 settings: nil,
 created_at: nil,
 updated_at: nil,
 json_attributes: {"insurance"=>#<GlobalSettings::Insurance:0x00007fbb169c8f78 @attributes={"percentage"=>15, "minimum"=>8}>}>
[2] pry(main)> gs.save
  TRANSACTION (0.2ms)  BEGIN
  GlobalSetting Create (1.0ms)  INSERT INTO "global_settings" ("settings", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id"  [["settings", nil], ["created_at", "2022-04-22 13:07:33.766521"], ["updated_at", "2022-04-22 13:07:33.766521"]]
  TRANSACTION (2.2ms)  COMMIT
=> true
[3] pry(main)> gs.insurance
=> #<GlobalSettings::Insurance:0x00007fbb16603fa0 @attributes={"percentage"=>15, "minimum"=>8}>
[4] pry(main)> gs.reload
  GlobalSetting Load (0.3ms)  SELECT "global_settings".* FROM "global_settings" WHERE "global_settings"."id" = $1 LIMIT $2  [["id", 3], ["LIMIT", 1]]
=> #<GlobalSetting:0x00007fbb16766190
 id: 3,
 settings: nil,
 created_at: Fri, 22 Apr 2022 14:07:33.766521000 BST +01:00,
 updated_at: Fri, 22 Apr 2022 14:07:33.766521000 BST +01:00,
 json_attributes: {}>
[5] pry(main)> gs.insurance
=> nil

Thanks for the error report!  I'll try to find time to reproduce and debug.
The only thing that looks at all unusual there to me is the bad_cast: nil, I wonder if it's a bug in that implementation.  Can you take take that out, and see if it it works without that, to give us more information about what's going on?

No that doesn't appear to make any difference unfortunately.

OK thanks for checking!
Some time next week I will try to reproduce and debug. Since this looks to me like a very ordinary example that I can't see any reason it wouldn't work... it's possible I won't be able to reproduce because it may actually be related to something else going on in your app... If you wanted to try to make a stand-alone reproduction of the problem, that could save us all some time!  One way to do that would be to make a failing test case in the rspec of this project.

I'll do that in a bit for sure. It did seem weird that seemingly basic behaviour didn't work.

You know @jrochkind , sometimes I should get more sleep (or more ☕ ). It would help me somewhat if I named my column correctly...
But that does raise a possible issue - if there isn't a column named either json_attributes OR whatever the config defines it to be, then AttrJson doesn't barf at me to tell me.
Still, this one can be closed. Thank you so much for your time on it, despite it being completely unnecessary! 🙈

No problem, thank you!
I agree that I'd expect a raise with a missing/mis-named column, and it seems unfortunate there isn't one.   I'm surprised I haven't run into this before, how can I have never mis-named a column!
I wonder what the built-in solutions like serialize do in similar situation?


Fix multi level sublcass inheritance

Hi,
this is related to issue #41.
As pointed out there getSubClassesOf  is not resolving transitive subclasses. I made a small fix and an additional test that verifies the behaviour and that can be replicated on https://shacl.org/playground/ (I do not know if that is the intended place for such kinds of tests though) .

Codecov Report

Merging #42 (5b1fd42) into master (dad9520) will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master      #42      +/-   ##
==========================================
+ Coverage   96.74%   96.75%   +0.01%
==========================================
  Files          14       14
  Lines         675      679       +4
==========================================
+ Hits          653      657       +4
  Misses         22       22

Impacted Files
Coverage Δ

src/rdflib-graph.js
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dad9520...5b1fd42. Read the comment docs.

On a second thought: my Implementation likely falls for cyclic subClass relationships. AFAIK they are forbidden by RDFS but you never know...
Will update the PR later this week.

Thanks :)

I created a separate issue for the cyclic subClass relationship problem: https://github.com/zazuko/rdf-validate-shacl/issues/43


winget reports non-existent version <IP_ADDRESS>53 of 4K Video Downloader: OpenMedia.4KVideoDownloader

Please confirm these before moving forward

[X] I have searched for my issue and not found a work-in-progress/duplicate/resolved issue.
[X] I have not been informed if the issue is resolved in a preview version of the winget client.

Category of the issue
Other
Brief description of your issue
winget reports the availability of <IP_ADDRESS>53 of 4K Video Downloader. This version doesn't exist, even the in-app "check for updates" reports <IP_ADDRESS>52 is the latest version.
Steps to reproduce
Run winget list 4kvideo on a machine where version <IP_ADDRESS>52 of 4K Video Downloader is installed.
Actual behavior
PS> winget list 4kvideo
Name                Id                          Version     Available   Source
------------------------------------------------------------------------------
4K Video Downloader OpenMedia.4KVideoDownloader <IP_ADDRESS>52 <IP_ADDRESS>53 winget

winget upgrade --all yields:
(1/3) Found 4K Video Downloader [OpenMedia.4KVideoDownloader] Version <IP_ADDRESS>53
This application is licensed to you by its owner.
Microsoft is not responsible for, nor does it grant any licenses to, third-party packages.
Downloading https://dl.4kdownload.com/app/4kvideodownloader_4.24.1_x64.msi
  ██████████████████████████████  93.3 MB / 93.3 MB
Successfully verified installer hash
Starting package install...
Successfully installed

which has no effect.
Expected behavior
winget shouldn't report a new version is available:

Environment
Windows Package Manager (Preview) v1.5.101-preview
Copyright (c) Microsoft Corporation. All rights reserved.

Windows: Windows.Desktop v10.0.22621.1413
System Architecture: X64
Package: Microsoft.DesktopAppInstaller v<IP_ADDRESS>

Logs: %LOCALAPPDATA%\Packages\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\LocalState\DiagOutputDir

User Settings: %LOCALAPPDATA%\Packages\Microsoft.DesktopAppInstaller_8wekyb3d8bbwe\LocalState\settings.json

Links
---------------------------------------------------------------------------
Privacy Statement   https://aka.ms/winget-privacy
License Agreement   https://aka.ms/winget-license
Third Party Notices https://aka.ms/winget-3rdPartyNotice
Homepage            https://aka.ms/winget
Windows Store Terms https://www.microsoft.com/en-us/storedocs/terms-of-sale

Screenshots and Logs
No response

This is related to winget-update bot.  You might seeing me in the close issue. @stephengillie

Reporting the same issue and still has not been fixed.


Feature/civ 1875

Before creating a pull request make sure that:

[X] commit messages are meaningful and follow good commit message guidelines
[X] README and other documentation has been updated / added (if needed)
[X] tests have been updated / new tests has been added (if needed)

Please remove this line and everything above and fill the following sections:
JIRA link (if applicable)
https://tools.hmcts.net/jira/browse/CIV-1875
Change description
Adding WA task configurations for General Application
Does this PR introduce a breaking change? (check one with "x")
[ ] Yes
[X] No

[ ] PLEASE DO NOT MERGE THIS UNTIL FOLLOWING PRs ARE MERGED:

https://github.com/hmcts/civil-service/pull/1218
https://github.com/hmcts/civil-ccd-definition/pull/983
https://github.com/hmcts/civil-general-applications/pull/394
https://github.com/hmcts/civil-general-apps-ccd-definition/pull/231


Need to add two more properties to AttestationVerificationSuccess

Attestation type:
From https://www.w3.org/TR/webauthn/#attestation-type
Based on https://www.w3.org/TR/webauthn/#sctn-attestation-types, seems like string, one of "Basic", "Self", "AttCA", "ECDAA", or "None"
Trust path:
https://www.w3.org/TR/webauthn/#attestation-trust-path
Seems like could be empty, a byte array (ECDAA) or a number of byte arrays (x5c where x5c has multiple values).

Why do we need them in? Are they needed as parameters to the assertion step?

Alright. But there doesn't seem to be very defined values.
I guess we could use a TypedString with values from the paranthesis is https://www.w3.org/TR/webauthn/#sctn-attestation-types

Basic
Self
None
AttCa
etc..

Should the trustpath be an array of byte arrays maybe?

I don't know what the trust path is supposed to look like either.  Definitely something that will need to be researched.

I now have a better understanding of the wording used. We do not need these after all in AttestationVerificationSuccess, but they are needed for step 16 in VerifyAsync. This can be closed out as well.


Refactor page conversion count and unit test

Description & motivation
Responding to #97
Refactored how page conversions are counted. Create page_key in the stg_ga4__events model rather than multiple times downstream. Should help with performance.
Checklist

[x ] I have verified that these changes work locally
[na ] I have updated the README.md (if applicable)
[x ] I have added tests & descriptions to my models (and macros if applicable)
[x ] I have run dbt test and python -m pytest . to validate exists tests

I'm not really clear on the expected semantics of page_key with the hour component of the key, but I notice that the old code had a DISTINCT. Was the idea that the same event on the same page in the same hour (on the clock, not within a rolling 1h window) would only be counted as one conversion?

Are you referring to this?
conversion_{{ce}} as (     select         distinct (concat( cast(event_date_dt as string), cast(hour as string), page_location )) as page_key,         sum(event_count) as conversion_count,     from events     where event_name = '{{ce}}'     group by page_key )
That feels like possibly a mistake? Why distinct + group by? @dgitis can you clarify?
I think we would want a sum of all the conversions within the hour.

Yeah, that. I see, yeah seems redundant.


[SPARK-11564][SQL] Dataset Java API audit

A few changes:

Removed fold, since it can be confusing for distributed collections.
Created specific interfaces for each Dataset function (e.g. MapFunction, ReduceFunction, MapPartitionsFunction)
Added more documentation and test cases.

The other thing I'm considering doing is to have a "collector" interface for FlatMapFunction and MapPartitionsFunction, similar to MapReduce's map function.

Merged build triggered.

Merged build started.

cc @cloud-fan @marmbrus

Test build #45269 has started for PR 9531 at commit c5c694f.

Merged build triggered.

Merged build started.

Test build #45270 has started for PR 9531 at commit 5f7acc8.

Merged build triggered.

Merged build started.

Test build #45275 has started for PR 9531 at commit 41cee21.

Test build #45269 has finished for PR 9531 at commit c5c694f.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):\n  * abstract class Writer extends BaseReadWrite \n  * trait Writable \n  * abstract class Reader[T] extends BaseReadWrite \n  * trait Readable[T] \n  * case class GetInternalRowField(child: Expression, ordinal: Int, dataType: DataType)\n

Merged build finished. Test FAILed.

Test build #45270 has finished for PR 9531 at commit 5f7acc8.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):\n  * final class ShuffleSortDataFormat extends SortDataFormat<PackedRecordPointer, LongArray> \n  * final class UnsafeSortDataFormat extends SortDataFormat<RecordPointerAndKeyPrefix, LongArray> \n  * abstract class Writer extends BaseReadWrite \n  * trait Writable \n  * abstract class Reader[T] extends BaseReadWrite \n  * trait Readable[T] \n  * case class GetInternalRowField(child: Expression, ordinal: Int, dataType: DataType)\n

Merged build finished. Test FAILed.

Test build #2001 has started for PR 9531 at commit 41cee21.

Test build #2001 has finished for PR 9531 at commit 41cee21.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):\n  * final class ShuffleSortDataFormat extends SortDataFormat<PackedRecordPointer, LongArray> \n  * final class UnsafeSortDataFormat extends SortDataFormat<RecordPointerAndKeyPrefix, LongArray> \n  * abstract class Writer extends BaseReadWrite \n  * trait Writable \n  * abstract class Reader[T] extends BaseReadWrite \n  * trait Readable[T] \n  * case class GetInternalRowField(child: Expression, ordinal: Int, dataType: DataType)\n

Test build #45275 has finished for PR 9531 at commit 41cee21.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):\n  * abstract class Writer extends BaseReadWrite \n  * trait Writable \n  * abstract class Reader[T] extends BaseReadWrite \n  * trait Readable[T] \n  * case class GetInternalRowField(child: Expression, ordinal: Int, dataType: DataType)\n

Merged build finished. Test PASSed.

LGTM


Fix completion item sorting in error constructor context

Purpose
$subject and refactor the FunctionCallExpressionTypeFinder to use Types API.
Fixes #33027
Approach

Describe how you are implementing the solutions along with the design details.

Samples

Provide high-level details about the samples related to this feature.

Remarks

List any other known issues, related PRs, TODO items, or any other notes related to the PR.

Check List

[x] Read the Contributing Guide
[ ] Updated Change Log
[ ] Checked Tooling Support (#)
[x] Added necessary tests

[ ] Unit Tests
[ ] Spec Conformance Tests
[ ] Integration Tests
[ ] Ballerina By Example Tests

[ ] Increased Test Coverage
[ ] Added necessary documentation

[ ] API documentation
[ ] Module documentation in Module.md files
[ ] Ballerina By Examples

@malinthar seems like the following test case is failing. Shall we address it?
language-server-core-test-suite > language-server-core-tests > org.ballerinalang.langserver.completion.ExpressionContextTest > test[102](mapping_expr_ctx_config37.json, expression_context) FAILED

@malinthar seems like the following test case is failing. Shall we address it?
language-server-core-test-suite > language-server-core-tests > org.ballerinalang.langserver.completion.ExpressionContextTest > test[102](mapping_expr_ctx_config37.json, expression_context) FAILED

Done


⚠ Quirk Club Argentina (API endpoint) has degraded performance

In 4399fbc, Quirk Club Argentina (API endpoint) (https://us-central1-quirkclub-dev.cloudfunctions.net/api/check/api) experienced degraded performance:

HTTP code: 200
Response time: 6378 ms

Resolved: Quirk Club Argentina (API endpoint) performance has improved in e5a71fb.


Problem executing binaries installed using nvm (Node Version Manager)

Hi, I'm trying to use mini/base to create a minimal-nvm container.  see this Dockerfile:
FROM mini/base

RUN apk-install \
      bash \
      tar \
      curl \
      git \
      c-ares \
      libgcc \
      libstdc++ \
      libuv

RUN rm /bin/sh && ln -s /bin/bash /bin/sh && \
    touch ~/.bashrc

ENV NODE_VERSION iojs

RUN NVM_VERSION="$(curl -s -L http://latest.nvm.sh -o /dev/null -D - | grep -Ei '^Location: .*tag/v[0-9.]+' | grep -oEi 'v[0-9.]+')" && \
    echo $NVM_VERSION && \
    curl https://raw.githubusercontent.com/creationix/nvm/$NVM_VERSION/install.sh | sh && \
    source ~/.nvm/nvm.sh && \
    nvm install $NODE_VERSION

when running:
→ docker run --rm -it mini-nvm bash

bash-4.3# nvm run iojs -p process.versions
Running io.js v1.5.1
bash: /root/.nvm/versions/io.js/v1.5.1/bin/iojs: No such file or directory

bash-4.3# ls -la /root/.nvm/versions/io.js/v1.5.1/bin/iojs
-rwxrwxr-x    1 500<PHONE_NUMBER>1 Mar  9 18:14 /root/.nvm/versions/io.js/v1.5.1/bin/iojs
kriswill 12 hours ago

Any advice is appreciated!

@kriswill have you tried the same using gliderlabs/alpine:3.1 instead?
https://github.com/gliderlabs/docker-alpine
I've zero experience with node and nvm to comment about this or to determine the real issue of the failure, but testing to see if the problem is consistent will be a start.
You can also check how the node package is built for Alpine itself:

http://forum.alpinelinux.org/apk/main/x86_64/nodejs
http://git.alpinelinux.org/cgit/aports/tree/main/nodejs/APKBUILD

@luislavena thanks for your reply!  I did try it with gliderlabs/alpine:3.1 and the result was the same:
bash-4.3# iojs --versoin
bash: /root/.nvm/versions/io.js/v1.5.1/bin/iojs: No such file or directory

I suppose the linux binary that nvm downloads from the iojs.org site is either incompatible with alpine linux, or there is some other configuration issue.  I could probably make a package for iojs, but I'd prefer to use nvm to version select, since that enables the greatest amount of flexibility configuring this particular software, and would lend itself to better to CI.  I'll keep trying...
PS, any further analysis is appreciated. :)

@kriswill

I suppose the linux binary that nvm downloads from the iojs.org site is either incompatible with alpine linux

This is most likely. Any existing binary for Linux might not work since Alpine uses musl as libc instead of glibc like traditional distributions.
https://www.alpinelinux.org/about/

I could probably make a package for iojs, but I'd prefer to use nvm to version select,

Actually will be better if you build packages for the different versions, so you have really small images that only contains that particular version of iojs/nodejs instead of all of them.
Not only on network and bandwidth savings but also on clarity (eg. setup a explicit dockerfile for iojs:1.0 or something vs using environment variables or tricky bash/sh scripts).
The whole idea behind using Alpine Linux and mini/base is to achieve the smallest and function specific container as possible instead of a general purpose one.

Thanks @luislavena, nvm is kind of like a package manager for use in development, allowing one to dynamically select what version of node/io.js they want.  For instance, it's used by Travis CI to setup a test run.  It also has the ability to download the source for node and compile it.  I haven't check to see if it would be able to perform a source compile on Alpine, been busy with other things.
I will:
[ ] - see if nvm can compile node/io.js on Alpine
[ ] - if so, make an Alpine package for io.js 1.5.1
[ ] - create a mini-iojs Docker image
These tasks will get me to the right place.  Thanks for all your help!

I was on #alpine-linux on Freenode a bit ago, and they were kind enough to push a build of iojs v1.6.2 (latest) to the testing repo.

@pikeas thank you for the info.
@kriswill considering there is a APKBUILD for iojs under testing:
http://git.alpinelinux.org/cgit/aports/tree/testing/iojs/APKBUILD?id=c8a18b22ab119ae31ac78340023a8c8ac380c4bb
I'm going to close this out. You might want to look at docker-alpine-abuild which will allow you to package any APKBUILD package using Docker.
Hope that helps.

@kriswill Were you able to get this to work?

@thedug I have concluded that building a base file system with the iojs compiled binary and npm baked in is the best way to do this.  using nvm from Docker would require an interactive shell, which isn't a good choice when using a supervisor to keep a process running for docker.

@kriswill How are you installing iojs on alpine?

This is what I ended up with..
RUN apk-install libstdc++
ENV iojs_version 1.6.3
RUN apk-install -t build-deps wget ca-certificates gcc g++ python make &&
wget -q https://iojs.org/dist/v${iojs_version}/iojs-v${iojs_version}.tar.gz &&
tar -xzf iojs-v${iojs_version}.tar.gz &&
cd iojs-v${iojs_version} &&
python ./configure &&
make &&
make install &&
rm -rf iojs-v${iojs_version}.tar.gz &&
rm -rf iojs-v${iojs_version} &&
apk del --purge build-deps

For prebuilt ioJS:
RUN \
    apk add -U -X http://nl.alpinelinux.org/alpine/edge/main/ \
        git=2.3.5-r0 \
        libstdc++=4.9.2-r4 && \
    apk add -U -X http://nl.alpinelinux.org/alpine/edge/testing/ \
        iojs=1.6.2-r0 && \
    rm -rf /var/cache/apk/*

@pikeas I saw that but I'm at the mercy of the package maintainers. With the above, I can compile any version I want to test with and tag the versions for fast swapping.

@thedug @pikeas @kriswill I think you might find this interesting:

https://registry.hub.docker.com/u/mhart/alpine-node/
https://github.com/mhart/alpine-node

Leaving those links here for others.
Cheers.

FYI, a giveaway that it's a glibc issue when you get a "No such file or directory" error from a binary that clearly exists is to check its dynamic symbol resolution via ldd:
# Upon login, in which /etc/profile.d/nvm.sh gets sourced:
nvm/versions/node/v5.6.0/bin/npm: exec: line 2: nvm/versions/node/v5.6.0/bin/node: not found
nvm is not compatible with the npm config "prefix" option: currently set to ""
Run `nvm use --delete-prefix v5.6.0 --silent` to unset it.

$ ls -l nvm/versions/node/v5.6.0/bin/node
-rwxrwxr-x    1 500<PHONE_NUMBER>6 Feb  9 16:15 nvm/versions/node/v5.6.0/bin/node

$ nvm/versions/node/v5.6.0/bin/node
bash: nvm/versions/node/v5.6.0/bin/node: No such file or directory

$ ldd nvm/versions/node/v5.6.0/bin/node
        /lib64/ld-linux-x86-64.so.2 (0x55994b70b000)
        libdl.so.2 => /lib64/ld-linux-x86-64.so.2 (0x55994b70b000)
        librt.so.1 => /lib64/ld-linux-x86-64.so.2 (0x55994b70b000)
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x7fce4c176000)
        libm.so.6 => /lib64/ld-linux-x86-64.so.2 (0x55994b70b000)
        libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x7fce4bf63000)
        libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x55994b70b000)
        libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x55994b70b000)
Error relocating node: __isinf: symbol not found
Error relocating node: __rawmemchr: symbol not found
Error relocating node: __isnan: symbol not found
Error relocating node: __strtod_internal: symbol not found
Error relocating node: backtrace: symbol not found

Just got that trying to use nvm to install node 5.6.0 on an alpine:3.3 Docker image.


Update overview.md to include "TeamsModern" option

Added the "teamsmodern" value for host.name to include the new Microsoft Teams client option.
Category

[x] Content fix
[ ] New article
[ ] Example checked item (delete this line)

Related issues
The original document was missing the "TeamsModern" option and as such code would fail in the new Microsoft Teams client.
What's in this Pull Request?
I simply added the "teamsmodern" option to the case switch.

thx - good catch to get this updated as well.


Using EF correction.

The last part of this doc talks about adding EF and it uses an example of adding to "netstandard1.0"... it should be 1.5 like this:
    "netstandard1.5": {
      "imports": [ "dnxcore50", "portable-net45+win8" ]
    }

Superseded by #338


Update Metadata of blobs using REST API

Hello Team,
I need your help to update the metadata for all the blob files using REST API.
I have followed the instruction given in the official document and able to list out the blobs with HAMC SHA256.
But i am not able to access or update metadata using REST API. Can you given me an example.
Thanks,
Balachandar-R

Hi @Balachandar-R, please refer to this page.


CoFH mods got an update, the mod refuses to load

As title

My version: Whoosh-1.12.2-0.1.9-universal

Fixed in 0.1.10


Support universal binaries on macOS

Currently, mull-cxx will extract doubled bitcode (e.g. for x96 and arm64) and mull-runner fails to read mutants from universal binary.
Steps to reproduce:
$ echo "int main(int argc) { return argc + argc; }" > test.c
$ clang -arch arm64 -arch x86_64 -g -fembed-bitcode test.c
$ mull-runner-12 a.out
[error] Cannot create SymbolicFile from: /private/tmp/sc-V1LsVrXo3/a.out
[error] Error messages are treated as fatal errors. Exiting now.
$ mull-cxx-12 a.out
[info] Link mutated program (threads: 1)
[error] Cannot link program
status: Failed
time: 100ms
exit: 70
command: clang /var/folders/1s/hps840w156xfn_m__h2m17880000gp/T/mull-d7e587.bc /var/folders/1s/hps840w156xfn_m__h2m17880000gp/T/mull-fdb944.bc -o /var/folders/1s/hps840w156xfn_m__h2m17880000gp/T/mull-168cac.exe
stdout:
stderr: warning: overriding the module target triple with x86_64-apple-macosx11.0.0 [-Woverride-module]
'apple-m1' is not a recognized processor for this target (ignoring processor)
'+crc' is not a recognized feature for this target (ignoring feature)
<truncated>

Is there any news on this?
If not universal binaries, does/will mull ship binaries for Mac arm64? The current release is only x86_64, so trying to pass the plugin to clang on my M1 Mac fails because clang apparently can't open the plugin.
Do I need to build mull from source myself?

Hi @benthevining, thank you for bringing this up.

If not universal binaries, does/will mull ship binaries for Mac arm64?

We don't ship universal binaries yet, but there is no technical limitation in doing so. I'll see if I can cut a new release soon.

Do I need to build mull from source myself?

I think it is the best option to build mull on your own on the new Apple Silicon. Currently, we do no sign binaries on macOS, so even if we ship a universal binary, you'd then have to codesign it locally (perhaps with an ad-hoc certificate).

@benthevining I realized in the end why we don't do universal binaries: we are relying on LLVM from homebrew which is not universal itself, so we can't easily make mull a universal binary as well.
Until https://github.com/mull-project/mull/issues/957 is not resolved, I'll update the docs suggesting/warning about precompiled macOS binaries on Apple Silicon hardware.

I ended up adding support for universal binaries by teaching mull to only consider one slice from the universal binary. The slice is chosen on which version of mull-runner is running (e.g. arm64 or x86_64).
The changes are here.
@benthevining it doesn't address your use case as it's slightly different from the topic of this issue. If you still have issues with Mull on Apple Silicon - please, consider opening a separate issue.

Thanks! I'll try again on my M1 machine and let you know if I have any issues.


Restore compat with GHC 7.4

This is just an update of https://github.com/bos/criterion/pull/99
It resolves a merge conflict.
CC @hvr

Related to this -- if we want to actually test the GHC 7.4 compat on travis that would be another matter.  Currently we've gone with a stack-only setup in the .travis file and there is no LTS that covers 7.4.

Oops, belatedly realized that this only touches the executable, which is now commented.  So it becomes trivial.  It doesn't hurt though -- just in case we reactivate that executable.


Model Output Summary Function

A function to display a summary table of model_output in pdf format. Related to #91.

I'm adding @zkamvar as a reviewer to this, mostly to check renv compatibility stuff...

I like this as a weekly summary. Let's talk by voices about a possible way to track more systematically a tabular way to assemble the data.

@zkamvar since you requested changes, I think your approval is required here. Do you see the change to using here as blocking approval?


voange.errors.AuthorizationError

When I use vonage.Voice(client), I get the above error. I have made sure that I am putting the right secret, application_id and api_key but still I am getting this error.

Hi @kintama48 more information is needed on what you're doing exactly, as it isn't possible to replicate the problem from the information you've given. Please provide more context/a reproducible example, or join the Vonage Community Slack and ask there.

Closing as there isn't enough information to reproduce


MaskedInput example not working

The MaskedInput example on the docs isn't functional, can't type into it https://v2.grommet.io/maskedinput

Looks like the example has value="" and an empty onChange so it's controlled and always forced to be empty.


Add mapped hydrogens

RDKit Version: 2018.09.3
Operating system: linux
Python version (if relevant): 3.7.3
Are you using conda? yes
If you are using conda, which channel did you install the rdkit from?
If you are not using conda: how did you install the RDKit?

I'm trying to process a molecule with a few labeled hydrogens (e.g. [H:20]CH:21CCCCC.[Cl:10]).
After using MolFromSmiles(), I'm losing the hydrogen with label 20. Adding it back using AddHs() leads to a non labeled hydrogens. Is there any way to preserve the labels of hydrogens?
For example, the code below does not print anything as it does not see the label of the added hydrogen atom.
mol1 = Chem.MolFromSmiles("[H:20][CH:21](C)CCCCC.[Cl:10]")
mol2 = Chem.AddHs(mol1)
for atom in mol2.GetAtoms():
    if atom.GetAtomMapNum()==20:
        print(atom.GetIdx())

@amintavakol :  you need to tell the RDKit not to remove Hs when you read the molecule in:
ps = Chem.SmilesParserParams()
ps.removeHs = False
mol1 = Chem.MolFromSmiles("[H:20][CH:21](C)CCCCC.[Cl:10]",ps)
for atom in mol1.GetAtoms():
    if atom.GetAtomMapNum()==20:
        print(atom.GetIdx())

If you have additional, unmapped, Hs in the molecule that you do want to remove, you can remove the Hs in a second step:
rhps = Chem.RemoveHsParameters()
rhps.removeMapped=False
ps = Chem.SmilesParserParams()
ps.sanitize=False    #<- skip sanitization because it will be done when we call RemoveHs()
ps.removeHs = False
mol1 = Chem.MolFromSmiles("[H:20][CH:21](C)CCCCC[H].[Cl:10]",ps)
mol2 = Chem.RemoveHs(mol1,rhps)


default code style

Hi, today I was trying to clean the code-style from all this scripts. e.g. function-declaration "thisIsAFunction()", try to use local variables (nearly everything is global, except you are in a function and use the keyword local), no tabs (2 spaces), lowercase for local variables, ... I get the code-style from here -> https://google-styleguide.googlecode.com/svn/trunk/shell.xml
What do you think, would it make sense to add a code-style-guide?
PS: here are my changes ... and yes I know we can't merge such big changes ;) it was only a idea: https://github.com/voku/oh-my-zsh/compare/robbyrussell:master...master

"Coding style guide" updated, but I don't know if someone really read it. ;) Maybe you ( @mcornella ) can review it, thx?

A couple items:

The text of the coding guide suggests to use the function keyword in function definitions, but the example code doesn't actually use it.
Text says to prefer $() to backticks, but uses backticks in a lot of the "good example" code
It talks about using ::-qualified packages, but I don't think that exists in ZSH; looks like a holdover from a C++ style guide?
It's written in a pretty authoritative, normative style (e.g. "the following is required for new code", "strongly preferred", "is evil"). Should it be worded so strongly unless it has the blessing of robbyrussell or some core development team?
The guide discusses "return values" when it seems to be referring to the output emitted on standard out, not the exit status in $?. I don't know if "return value" is the typical term for that; I think I've seen "function output" more.

If we're going to discuss style, I'd be interested in working out whether and where double-quotes should be used around scalar parameters regularly. They're not required in ZSH since it doesn't do automatic word splitting upon parameter substitution like other shells. Unless some of its special options like SH_WORD_SPLIT or ksh emulation are set. but emulate could be a cleaner way of protecting against that and other unusual shell settings inside functions.


Add course exercise example notebook

Includes

Example Jupyter notebook with some exercises for the accelerator course
Minor modifications to the Beam class (added particle mass, relativistic beta and gamma, as well as a matched particle initialization method required for the exercise)

There is also a small function to slice up all the Drift elements into many smaller drifts. This helps getting nicer results for the beta-functions in the course exercises, for example. Probably this small function could / should be included in the accelerator tools.

Hi Michael,
I had a look at the notebook and it looks really cool, well done !
I'll add the slicing method to the toolbox.
I saw that tracking a distribution of particles is tedious, I originaly wanted to to have a nice way of doing this which is why I added the sampling functions but I never got around to it. I'll think about a nice way of doing it. If you have any suggestions I'm all ears.

Would be nice to implement a function that can plot the lattice and add labels / indices which index in the phase_space array corresponds to which location in the lattice

By this you mean something like the unused plot you left at the end but with the element names or indices in the lattice object ?

Hi Loic!
It was a lot of fun to work with your toolbox, it’s really great for our purposes of the course (and quite possibly also for other stuff!)
Thanks for adding the slicing method!
Indeed the tracking of a distribution would be really nice to have to simplify everything “on the surface”. You saw my solution in the notebook, not very inspiring. So for the tracking itself I didn’t have a better idea yet :( What’s important certainly for the sampling (I thought you had maybe intended to do that already) is to have the distribution matched to the Twiss where you start tracking, ie. where you “inject” beam. It’s maybe not too bad to have that done explicitly and not automatic.
For the last point: yes exactly! But again, didn’t have a good idea to do it in a neat way so didn’t want to mess up your code that’s why the function is left separate. I mean adding the labels would not be difficult, but maybe you have a better idea how to implement the whole thing. I was thinking you need some kind of mapping of magnet type -> linestyle surely, I think that’s what’s left. Just wasn't very happy with it and didn't have more time to extend and improve.

On 28 Aug 2020, at 19:48, Loic Coyle<EMAIL_ADDRESS>wrote:

Hi Michael,
I had a look at the notebook and it looks really cool, well done !
I'll add the slicing method to the toolbox.
I saw that tracking a distribution of particles is tedious, I originaly wanted to to have a nice way of doing this which is why I added the sampling functions but I never got around to it. I'll think about a nice way of doing it. If you have any suggestions I'm all ears.
Would be nice to implement a function that can plot the lattice and add labels / indices which index in the phase_space array corresponds to which location in the lattice
By this you mean something like the unused plot you left at the end but with the element names or indices in the lattice object ?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Ok understood, I'll merge this and have a think about the rest!
BTW I sent you a collaborator invite so you should be able to interact with the repo directly if you want.


Handle null pointer in Shibb Authenticate

This change is designed to work around null pointer exceptions discovered during testing.
Do we have a better mechanism in place to ensure that the service factory has been properly initialized?

Not sure if this pull request will fix this issue in all places on which it occurs, the authenticationService variable is used in multiple places.
The reason it is null is due to the fact that the "AuthenticationServiceImpl" is creating its method stack prior to spring being "done" with everything:
https://github.com/DSpace/DSpace/blob/master/dspace-api/src/main/java/org/dspace/authenticate/AuthenticationServiceImpl.java#L71

I agree that this solution is not comprehensive.  Who would be the best person to investigate this further?

Well I'm hoping that Tim's new configuration will fix this: https://github.com/DSpace/DSpace/pull/1104 so perhaps we should attempt to merge these branches & try it out ?
If this doesn't work we should create a blocker JIRA for this issue so we don't forget to fix it.

https://jira.duraspace.org/browse/DS-2982 created.
The solution proposed in this PR is not optional, but it is useful for testing.

As I just noted in the ticket for this PR, I agree with @KevinVdV that I believe this will be fixed by #1104, so this PR should likely not be merged (unless we find #1104 doesn't fix the full problem)
My suspicion is that the PluginManager is actually at fault (with AuthenticationPlugins rely heavily on), and it's replaced by a Spring-ified PluginService in #1104. I think this change will fix the strange order of operations issues you are seeing, as everything will now be loaded via Spring.

Closing.  See https://github.com/DSpace/DSpace/pull/1271


Fix to make TreeViewItems non tabbable

As a part of this change verified below:

Made sure tree view items are non tabbable. User can tab into tree view and use up/down arrow keys to navigate
Home and end key press events work as expected

Closing this pr. Sent out another pr which includes changes in this pr plus fixes for few other accessibility issues in PackageContentsTreeView control.


Allow customization of the from and to calculation in TablePagination

[X] I have followed (at least) the PR section of the contributing guide.

Currently to customize the from and to numbers in table pagination it is necessary to provide labelDisplayedRows which is sub-optimal because labelDisplayRows benefits from localization by default and that is lost with a simple customization. In addition, labelDisplayedRows is lacking a passed in property for rowsPerPage which is necessary to make the calculations for from and to.
A use-case I had in mind was that I wanted to control which rows correspond to each page. By default it will always calculate the rows per page like this:
1-10 of 25
11-20 of 25
21-25 of 25

but I had an occasion to want this:
1-5 of 25
6-15 of 25
16-25 of 25

I think it makes sense to separate the calculation of the from and to rows from the display logic of labelDisplayedRows into a new calculateRowRange function. The defaultCalculateRowRange function uses the current logic for calculating from and to, it's just possible to customize that like this now:
<TablePagination
  colSpan={4}
  count={count}
  rowsPerPage={rowsPerPage}
  calculateRowRange={calculateRowRange}
  page={page}
  onChangePage={handleChangePage}
  onChangeRowsPerPage={handleChangeRowsPerPage}
/>

@dpwrussell Thanks for spending time on this. Do you have more details on the use case? From what I understand from your example, it can be solved with a page size of 5 instead of 10.

@dpwrussell Thanks for spending time on this. Do you have more details on the use case? From what I understand from your example, it can be solved with a page size of 5 instead of 10.

Hi. The page size is actually still 10, it's just that I want the "remainder" rows (i.e. count % rowsPerPage) to be in the first page, not the last page.
1-5 of 25 (5 remainder rows)
6-15 of 25 (10 rows)
16-25 of 25 (10 rows)

Without this, the pages would be:
1-5 of 25 (10 rows)
6-15 of 25 (10 rows)
16-25 of 25 (5 rows)

I am unsure how to resolve the last of the CI checks. The propTypes check seems to not want the trailing comma, whereas the doc checks want it to be exactly the same as the code (where the trailing comma is required).

@dpwrussell you need to run yarn docs:api, but like @oliviertassinari, I don't understand what problem you're trying to solve, and why it should be handed by the library.

@dpwrussell Oh, I see, right, thanks for the clarification. I wonder if we should handle it with a boolean flag and have it built-in 🤔

Oh, I see, right, thanks for the clarification.

? 😕

@oliviertassinari It was my first inclination to implement it as a boolean flag, but then I thought that there might be other use cases that would benefit from a more generic implementation. E.g.
# Always show 10 items
1-10 of 25
11-20 of 25
16-25 of 25

# Overlapping pages
1-12 of 25
9-22 of 25
19-25 of 25

I will correct the doc as well by running the command suggested.
Let me know if you'd prefer to switch to an inverse pagination as I can adapt the PR to that, but as I've said, I think the more generic solution could offer more flexibility.

Okay, I understand now. (Perhaps I wasn't fully awake this morning!)
Sorry, but this use case seems really obscure. I don't think we should be adding extra props to support it.

Sorry, but this use case seems really obscure. I don't think we should be adding extra props to support it.

@mbrookes Ok, in this case, my proposal for a boolean to inverse the pagination should likely be ignored.

labelDisplayedRows which is sub-optimal because labelDisplayRows benefits from localization by default and that is lost with a simple customization

@dpwrussell this is not completely accurate. You could for en-US redefine the string representation, and for the other locales wrap the translation that is available in the theme.
It's to be noted that we don't correctly number format the pages. 1-100 of 1000 that's not OK. In en-US, it should be 1-100 of 1,000, in fr-FR it should be 1-100 of 1 000. We will need to add this extra transformation step in the display.
@dtassone What do you think about the proposal?

Well, your decision of course, but at least inverse pagination seems like a pretty routine use-case to me and the TablePagination component has proven really flexible, except for this one thing, so it seems a shame.
I agree with you @oliviertassinari that it is possible for me to achieve my goals using labelDisplayedRows, but it just feels wrong to be overriding display logic and having to implement localization for that, when I only want to manipulate the row calculation not how it is displayed. Also, I'd have to use a closure to provide the missing parameter rowsPerPage in my function because it's not supplied to labelDisplayedRows which isn't the end of the world, just not as clean as it could be.

Also I thought you would want to know that it looks like the yarn docs:apiis at odds with yarn proptypes as I said.
Running yarn proptypes makes this change.
-   *     to: count !== -1 ? Math.min(count, (page + 1) * rowsPerPage) : (page + 1) * rowsPerPage,
+   *     to: count !== -1 ? Math.min(count, (page + 1) * rowsPerPage) : (page + 1) * rowsPerPage

Which yarn docs:api then objects to:
Error: with component /material-ui/packages/material-ui/src/TablePagination/TablePagination.js: Expected JSDOC @default annotation for prop 'calculateRowRange' of "function defaultCalculateRowRange({ page, rowsPerPage, count }) {
  return {
    from: count === 0 ? 0 : page * rowsPerPage + 1,
    to: count !== -1 ? Math.min(count, (page + 1) * rowsPerPage) : (page + 1) * rowsPerPage
  };
}" to equal runtime default value of "function defaultCalculateRowRange({ page, rowsPerPage, count }) {
  return {
    from: count === 0 ? 0 : page * rowsPerPage + 1,
    to: count !== -1 ? Math.min(count, (page + 1) * rowsPerPage) : (page + 1) * rowsPerPage,
  };
}"

inverse pagination seems like a pretty routine use-case to me

Sorry, I used the wrong terminology, since this isn't itself a use-case. Forgive my lack of imagination, but what are some of the practical applications for inverse pagination? That will help explain why it's valuable enough fro prop-creep not to be a concern.

On the whole what I am saying is that there is no particular reason to hardlink the number of pages to the number or disposition of the rows that are indicated as being displayed.
Inverse pagination use-case
A good example would be a bank account statement. When you look at your bank account statement, what you probably want to see first is your most recent transactions, not the oldest transactions. That's fine, we just configure the default page to be the last page. However, because of the way that pagination is currently calculated, if there are 21 transactions and the page size is 10, they would be shown 1 transaction. Instead, they always want to see their 10 most recent transactions.
Overlapping rows use-case
Lets say you were building a log viewer using a table, maybe you want your pages to be a sliding window of your logs with context. E.g. you have your log statements and a page size of 10. By this you mean, I would like there to be Math.ceil(transactions.length / rowsPerPage) pages. On each page, you want to show 10 logs in the context of 5 additional (where possible) transactions above, and 5 additional (where possible) transactions below. Then you'd want your status to say it was showing (for example) Rows 6-25 of 123.
You could say, "why not just display 20 logs per page?", but that isn't the same because the page number is being used to ensure that on each page, the rows that are being examined are in the central 10.

When you look at your bank account statement, what you probably want to see first is your most recent transactions

In which case, the transactions would be ordered most recent first, and nothing else needs to change.

Lets say you were building a log viewer using a table

If I was building a log viewer and wanted the data tabulated, I would use the upcoming data-grid component. But that aside... 🙂

Then you'd want your status to say it was showing (for example) Rows 6-25 of 123.

It sounds like you want to be able to decouple the number of rows that that previous and next page buttons increment?

@dpwrussell Thanks for bringing this topic to our attention. At this point, it seems that we should hold on the topic. I'm closing the pull request. Hopefully, with more requests in this direction, we will reconsider.

Sorry for delay in replying, I have not had time to respond.
I created this to show what the use-cases I have in mind would look like for "Reverse" and "Overlap": https://dpwrussell.github.io/mui-table-pagination-demo/

When you look at your bank account statement, what you probably want to see first is your most recent transactions

In which case, the transactions would be ordered most recent first, and nothing else needs to change.

@mbrookes Ordering the transactions most recent first doesn't really fix the issue because for all other purposes, they are already in the correct order. Yes, the page with the newest transactions will now correctly be "full", but now the page numbers are reversed.
Often the page number is semantically important. For example, it might be used in the URL to allow future direct access to this page of data. If transactions have a natural ordering, it makes sense to use that. E.g. if the url is example.com/transactions?page=4 it stands to reason that example.com/transactions?page=5 would be the next (as in newer) page of transactions.
More to the point, this also holds true for the label. If the order is reversed, it would now tell the user for the first page (of the most recent transactions) that they are looking at Rows 1 - 10 of 123, when semantically that doesn't make sense. If they are looking at rows with a natural ordering, then you'd expect Rows 114 - 123 of 123 because those are the most recent 10 rows.
Happy to be corrected if I am somehow wrong about this, but I don't think this result is possible with a simple reordering of the data.

Then you'd want your status to say it was showing (for example) Rows 6-25 of 123.

It sounds like you want to be able to decouple the number of rows that that previous and next page buttons increment?

That's not quite right. See my demo for the exact behaviour. Basically we're paging n rows at a time, with a further n above and below for context. If you simply page n*3 rows each time, then some rows would never appear as the main paged rows, but only ever in the context. In my demo, the primary rows are in white, the context rows are yellow and partially transparent.
I grant you that these are not the bread and butter of TablePagination, but with this simple PR, it's trivial to implement them (and any other less common scenarios) without interfering with display logic (that might otherwise benefit from localization).


GolangCI internal error

Example: https://golangci.com/r/github.com/istio/istio/pulls/12916
GOLANGCI_COM_RUN=1 golangci-lint run --out-format=json --issues-exit-code=0 --deadline=5m --new=false --new-from-rev= --new-from-patch=../changes.patch
    fatal error: runtime: out of memory

@utka
This is not a required gate. In fact, things like the above are a good reason why we might hold on to make it required. We run CI check on circleci machine with beefier CPU/memory.

Seems to be fixed


Support clusterctl upgrade plan

User Story
As a user/operator I would like to determine the current CAPI* versions being used on the management cluster as well as if there is an upgrade available using clusterctl upgrade plan so that I don't have to take as long to manually figure this out probing deployments or other manual means.
Detailed Description
Only support clusterctl upgrade plan to determine current and upgradeable versions for the mangement cluster.  clusterctl upgrade apply out of scope for this feature request.  Currently if you run this against a cluster which has CAPI and the provider installed, you get this:
Checking new release availability...
Error: invalid management cluster: there should a core provider, found 0

/kind feature

@dtzar This can be achieved by writing a plugin for clusterctl that can work with operator API https://github.com/kubernetes-sigs/cluster-api/pull/8957

/triage accepted


Highlighting entities in Safari Fails

In the New UI.
Getting this error in the console:
{
    "statusCode": 400,
    "error": "Bad Request",
    "message": "You must specify the value that this keyword represents in the user saying",
    "validation": {
        "source": "payload",
        "keys": []
    }
}

from this request:
{
  "userSays": "Can I get a cheese pizza?",
  "keywords": [
    {
      "value": "",
      "keyword": "Toppings", "start": 0,
      "end": 0,
      "keywordId": "1"
    }
  ],
  "actions": ["Order Pizza"]}

Safari appears to be removing the highlight when the dropdown appears.

Okay, we made progress, but one more:

I can't highlight the last entity.


AttributeError: Requirements instance has no attribute 'requirements'

I'm using cookiecutter to create a Python package and would like to use requirements.py to handle package dependencies. I'm using Python 2.7.11 (with Anaconda 2.2.0).
I've got requirements.txt and requirements.py in my package's root directory, and my setup.py contains the lines
from requirements import r
...
setup(
    ...
    **r.requirements,
    ...
)

When I try to run python setup.py develop, I get an AttributeError: Requirements instance has no attribute 'requirements'.

Calling **r.dependencies instead seems to work.

Sorry for being late. This is little mistake in README of course.
Maybe both of properties (requirements and dependencies) can be used to avoid this confusion.


Testing with Datadog!

Hey folks! This PR adds instructions on how to test a Serverless application with Datadog APM, Synthetics, and Monitors!
Feel free to make any suggestions.  This PR is ready for review.
Thanks!

Thanks for the PR @astuyve! We have a couple of folks tasked to review as soon as possible. Appreciate the effort and partnership.

Hey @brnkrygs - one problem I just realized is that the .gitignore file is ignoring the lib directory, which causes my CDK stack not to be committed. I can force-push it, but perhaps we could re-visit this configuration?

@dancfox - thanks for the note!
Could you try pulling this with an updated git ignore which removes lib from the ignore path? Right now the entire project's .gitignore will filter out anylib directory, I referenced that here. I didn't want to remove that setting as I'm not sure why it was added. So I suspect either you may have an older version of this branch, or git pull is ignoring the file.
I can push a new commit which drops this check if that helps.
Thanks!

https://github.com/aws-samples/serverless-test-samples/commit/8554fc12c71f209d2060883f228c5aae07d68abc

@dancfox Aha! also files in /bin directories are ignored

Hi @astuyve - we ran into this in a separate PR as well... anything with CDK will hit it.
To get around it, you can directly add the file with git add ...path..to...file...to...add even though .gitignore is ... ignoring it, but looks like @dancfox beat me to it with a .gitignore update.
Given the mix of frameworks in this repo we may have to go back and forth a bit to tune this, or include some explicit instructions.
Note that you'll probably have to explicitly add a file under /bin as well.
Check out the thread here for our earlier troubleshooting:
https://github.com/aws-samples/serverless-test-samples/pull/204/files/d4fefcd15a5a6a783980711a9d024df3a8e07edd#diff-19191bb47680afb01355baf82f2cf99a2a74c3537a31da851b7455b09dc99a56

Hi @brnkrygs - thanks for the note! I did stumble into that and pushed the file manually about 30s ago in this commit: https://github.com/aws-samples/serverless-test-samples/pull/203/commits/ec1d80d8f53ce26366143e1c5db486b54d9c311d
I pulled this into a new directory and was able to build the stack, so we should be set there. Please let me know if there are further issues, thank you!

Hey thanks, hopefully I'll have an update here in the next couple days!

Hy @brnkrygs - sorry for the delay. I believe I've addressed the comments, please feel free to add more if there's something I missed!
Thanks again!


add login.gov provider

Description
This change does three things:

Adds the login.gov provider, which is basically an OIDC provider, but doesn't have renewal stuff in it, and it also requires more parameters for the authentication step.
Adds environment variables to all config options, so that it is easier to configure the proxy in environments like GCP's App Engine.
Updates the README with info about how to use the login.gov provider.

Motivation and Context
We would like to be able to use the proxy with login.gov.
How Has This Been Tested?
I wrote tests into logingov_test.go, and have been using that.
I also have set up a test application using the login.gov integration environment on my local host,
using the instructions in the README.md.
Checklist:

[X] My change requires a change to the documentation or CHANGELOG.
[X] I have updated the documentation/CHANGELOG accordingly.
[X] I have created a feature (non-master) branch for my PR.

Hey, I don't understand those linter errors.  I'm happy to adjust stuff on my branch if you can help me figure them out.  :-)  I have tried doing a gofmt on the files, which results in no differences. 🤷♂

options.go:1::warning: file is not goimported (goimports)
providers/logingov.go:1::warning: file is not goimported (goimports)
providers/logingov_test.go:1::warning: file is not goimported (goimports)

These are not code-style warnings. This seems to happen only in the source files in which you import "github.com/dgrijalva/jwt-go". I'm not sure why ... but my first guess is that it wants you to give this import a name because the package name does not match the last component of the path. Like:
import (
        jwt "github.com/dgrijalva/jwt-go"
)

Notice also that this codebase puts non-standard-library imports in a separate grouping that's an informal code-style thing.

Perfect.  The separate grouping seemed to do the trick!  Thanks!

Huh, I guess it was a code-style thing :) I think "goimport" is like "gofmt" with even more rules for imports, so it was saying that "goimport" must not have been not run on these files. Good to know ...

Is this ready for review?

Yes, I believe it is ready for review.  Please feel free to give it the approval/feedback/etc that it deserves!  :-)

Hey, I know this is a pretty big set of changes, but it would be great if it could get some review/approval love so that it won't get stale and hard to merge.  Just wanted to ping to make sure that this was still on your mind.  :-)  Thanks!

Sorry, didn't mean to harass you!  Life is indeed busy.  :-)  I have updated everything that you requested and did a merge with master, updated CODEOWNERS, etc.  Let me know what else you need.  Thanks!

By a TODO, did you mean in the code, or an issue in the repo, or something in the README?  I would think that an issue would be the best way to do this, but wanted to make sure in case you had something else in mind.  :-)

I would tend to put a TODO(@<gh_id>): ... in the code and raise an issue to match it. Helps people reading the code notice the known issues IMO, either is sufficient on its own though

Hey, anything else you need me to do here?  I actually have a few more changes that I am maintaining on another branch that I am using to be able to deploy this stuff in GCP that I'd love to clean up and PR, but it would be hard without all the rest of this stuff.  :-)  Love to get this merged in!

@timothy-spencer Thanks for your work on this 😄

Thank you for taking the time to review/merge!


Fix Deep Link for Statements page

Details
On mobile devices, links to Statements page (new.expensify.com/statements/202202/) aren't opening in the app, instead, they open in the web browser.
Fixed Issues
$ https://github.com/Expensify/App/issues/8185
Tests

Open web browser
Click on links that start with "news.expensify.com/..."
Click on links to Statements page "news.expensify.com/statements/..."

NOTE: It's necessary to update the file "new.expensify.com/apple-app-site-association" with my version of the file for all Deep Links to work on iOS.
Also, after iOS 14, Apple uses its own CDN to host a copy of the file so it may take a few hours to update. That's why I tested with my own domain.

[x] Verify that no errors appear in the JS console

PR Review Checklist
Contributor (PR Author) Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[ ] I added steps for local testing in the Tests section
[ ] I added steps for Staging and/or Production testing in the QA steps section
[ ] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[ ] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[x] I included screenshots or videos for tests on all platforms
[ ] I ran the tests on all platforms & verified they passed on:

[ ] iOS / native
[x] Android / native
[ ] iOS / Safari
[ ] Android / Chrome
[ ] MacOS / Chrome
[ ] MacOS / Desktop

[x] I verified there are no console errors (if there’s a console error not related to the PR, report it or open an issue for it to be fixed)
[ ] I followed proper code patterns (see Reviewing the code)

[ ] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[ ] I verified that comments were added to code that is not self explanatory
[ ] I verified that any new or modified comments were clear, correct English, and explained “why” the code was doing something instead of only explaining “what” the code was doing.
[ ] I verified any copy / text shown in the product was added in all src/languages/* files
[ ] I verified any copy / text that was added to the app is correct English and approved by marketing by tagging the marketing team on the original GH to get the correct copy.
[ ] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named “index.js”. All platform-specific files are named for the platform the code supports as outlined in the README.
[ ] I verified the JSDocs style guidelines (in STYLE.md) were followed

[ ] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[ ] I followed the guidelines as stated in the Review Guidelines
[ ] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[ ] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[ ] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[ ] If a new component is created I verified that:

[ ] A similar component doesn't exist in the codebase
[ ] All props are defined accurately and each prop has a /** comment above it */
[ ] Any functional components have the displayName property
[ ] The file is named correctly
[ ] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[ ] The only data being stored in the state is data necessary for rendering and nothing else
[ ] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[ ] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[ ] All JSX used for rendering exists in the render method
[ ] The component has the minimum amount of code necessary for its purpose and it is

[ ] If a new CSS style is added I verified that:

[ ] A similar style doesn’t already exist
[ ] The style can’t be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)

[ ] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[ ] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.

PR Reviewer Checklist

[ ] I verified the correct issue is linked in the ### Fixed Issues section above
[ ] I verified testing steps are clear and they cover the changes made in this PR

[ ] I verified the steps for local testing are in the Tests section
[ ] I verified the steps for Staging and/or Production testing are in the QA steps section
[ ] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[ ] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[ ] I checked that screenshots or videos are included for tests on all platforms
[ ] I verified tests pass on all platforms & I tested again on:

[ ] iOS / native
[ ] Android / native
[ ] iOS / Safari
[ ] Android / Chrome
[ ] MacOS / Chrome
[ ] MacOS / Desktop

[ ] I verified there are no console errors (if there’s a console error not related to the PR, report it or open an issue for it to be fixed)
[ ] I verified proper code patterns were followed (see Reviewing the code)

[ ] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[ ] I verified that comments were added to code that is not self explanatory
[ ] I verified that any new or modified comments were clear, correct English, and explained “why” the code was doing something instead of only explaining “what” the code was doing.
[ ] I verified any copy / text shown in the product was added in all src/languages/* files
[ ] I verified any copy / text that was added to the app is correct English and approved by marketing by tagging the marketing team on the original GH to get the correct copy.
[ ] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named “index.js”. All platform-specific files are named for the platform the code supports as outlined in the README.
[ ] I verified the JSDocs style guidelines (in STYLE.md) were followed

[ ] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[ ] I verified that this PR follows the guidelines as stated in the Review Guidelines
[ ] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[ ] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[ ] If a new component is created I verified that:

[ ] A similar component doesn't exist in the codebase
[ ] All props are defined accurately and each prop has a /** comment above it */
[ ] Any functional components have the displayName property
[ ] The file is named correctly
[ ] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[ ] The only data being stored in the state is data necessary for rendering and nothing else
[ ] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[ ] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[ ] All JSX used for rendering exists in the render method
[ ] The component has the minimum amount of code necessary for its purpose and it is broken down into smaller components in order to separate concerns and functions

[ ] If a new CSS style is added I verified that:

[ ] A similar style doesn’t already exist
[ ] The style can’t be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)

[ ] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[ ] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.

QA Steps

Open web browser
Click on links that start with "news.expensify.com/..."
Click on links to Statements page "news.expensify.com/statements/..."

NOTE: It's necessary to update the file "new.expensify.com/apple-app-site-association" with my version of the file for all Deep Links to work on iOS.
Also, after iOS 14, Apple uses its own CDN to host a copy of the file so it may take a few hours to update. That's why I tested with my own domain.

[✔] Verify that no errors appear in the JS console

Screenshots
iOS
https://user-images.githubusercontent.com/6005972/163654731-c29919cb-dba4-4c7f-b100-5ae0e413ddf3.mov
Android
https://user-images.githubusercontent.com/6005972/163653553-c881ccbe-cae2-41be-92da-4df9ff486efc.mp4

I have read the CLA Document and I hereby sign the CLA

recheck

@tomivs here are the steps that I followed to test this PR.

Build the app on a physical device (tried both dev and production)
Go to tomivs.com/linki
Click on new.expensify.com/statements/202202

But the link opens in a webpage.
npx uri-scheme open "https://new.expensify.com/statements/20202" --android doesn't work either.
Am I doing something wrong? Could you please help me here. Thanks!
Platforms: Android 12 (Pixel 6), iOS 15.4 (iPhone 13).
https://user-images.githubusercontent.com/29673073/163881074-fb16ffad-7e2f-43b2-8f4e-663ac89595bf.mp4

Platform: Android 12 (Pixel 6).
@rushatgabhane it works on Android 11 or lower. We would have to change the SDK versions and update AndroidManifest, build.gradle(s), etc. That's possible but might break a few things.

If you check "main/android/build.gradle", the Android app is set up for targetSdkVersion = 30 (Android 11).
I would recommend another pull request such as "Update Android SDK version to 31 (Android 11)" in order to avoid regression bugs.
But if you test with other Android versions, it should work.

Since there were changes for App Links on Android 12, these would be the steps we would have to follow in order to make the App Links work on Android 12:

Update targetSdkVersion to 31 (Android 12).
Update some old npm packages to versions that support Android 12.
Update AndroidManifest.xml following Android 12 guidelines.
Re-generate and update https://www.expensify.com/.well-known/assetlinks.json in order to verify links for the New App (right now it's only set up for the old app).

Without following the steps mentioned above, the App Links won't work on Android 12.
What do you guys think? Should we go like this for now or update for Android 12? If so, will you create a new Upwork post?
Let me know. I can help with that too.

@tomivs thanks for sharing your insights! I think this is what we should do

Merge this PR.
You should log a new issue on #expensify-open-source slack channel, because you found the cause.
Submit a proposal when the new issue is created.

@tomivs I think the tests and QA steps could be simplified to this -
For iOS, Android and mWeb.

Login to NewDot.
Send this link to any user - new.expensify.com/statements/202202
Click on the link new.expensify.com/statements/202202.
Verify that the link is opened in the native app, and not the web browser.

@tomivs I think the tests and QA steps could be simplified to this -
For iOS, Android.
Login to NewDot.
Send this link to any user - new.expensify.com/statements/202202
Click on the link new.expensify.com/statements/202202.
Verify that the link is opened in the native app, and not the web browser.
For web and desktop
Regression tests only

We already have deep linking for reports, so it seems like from this comment https://github.com/Expensify/App/pull/8656#issuecomment-1102486931 those should also not be working on Android 12 or above?
As for next steps, it looks more like this:

Review / Update / Merge this PR
Have new.expensify.com/apple-app-site-association(https://new.expensify.com/apple-app-site-association) updated for this to be working in production
Test it on staging and production
And then as mentioned, You should log a new issue on #expensify-open-source slack channel
Submit a proposal when the new issue is created.

As for this PR, let's get some screenshots on all platforms - and will this work on desktop as well?

We already have deep linking for reports, so it seems like from this comment https://github.com/Expensify/App/pull/8656#issuecomment-1102486931 those should also not be working on Android 12 or above?

That's right, deep linking for reports doesn't work on Android 12 and above.

and will this work on desktop as well?

Yep, desktop works fine!

As for this PR, let's get some screenshots on all platforms - and will this work on desktop as well?

It works - as there are not regression issues -, however, if you mean opening web links in the desktop app - it doesn't work at all. For desktop, all links open in the browser. The reason is because Electron's API only allows handlers for protocols such as expensify:// (not HTTP URLs) and something like that hasn't been implemented for Expensify app. I could help out implementing it, but this PR is only for adding Statements page's Deep Link to the current implementation.

My bad, I confused opening links from the desktop app with deep linking.
https://user-images.githubusercontent.com/29673073/164271337-80f9aeca-085e-4b64-ae0c-be5603a57cb0.mov

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by @thienlnam in version: 1.1.57-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

@tomivs @thienlnam are we able to QA this in staging?
Team tried using https://staging.new.expensify.com/apple-app-site-association and it's not working. Not sure if this is a PROD PR only?

Yeah, I think this can only be tested on production for iOS due to their app association.  This would be a prod QA

@tomivs @thienlnam are we able to QA this in staging? Team tried using https://staging.new.expensify.com/apple-app-site-association and it's not working. Not sure if this is a PROD PR only?

It should be possible to test it both in staging and production, but the file needs to be updated in the website. I just checked the URL you shared and it's not updated. That might be the reason. If other URLs such as https://new.expensify.com/settings work, then the file apple-app-association needs to be updated in the website.
For staging: https://staging.new.expensify.com/apple-app-site-association
For production: https://new.expensify.com/apple-app-site-association

@thienlnam @tomivs QA team unable to test this PR on both  in staging and production.

@thienlnam @tomivs QA team unable to test this PR on both  in staging and production.

@kbecciv Files in the website must be updated...
For staging:
https://staging.new.expensify.com/apple-app-site-association
For production:
https://new.expensify.com/apple-app-site-association

Okay, I have some extra context on this.  The file should be updated automatically once it is deployed since you updated them.  The thing is, we're actually hosting an old file at those links so you won't see any of the new changes.  Apple has updated their specification to .well-known/apple-app-site-association located here
.  We've also found that we're deploying it to the wrong location due to webpack auto inferring the file type which is being fixed in https://github.com/Expensify/App/pull/8803
Next steps: Once that PR is merged and deployed, this PR can be tested properly

@thienlnam Does it mean we should re-test this now?

@mvtglobally Yes, please retest this on staging and on production once it is there as well

@kbecciv @mvtglobally Any update from QA team on this? @thienlnam mentioned a PR that needed to be merged and it has already been merged. So I believe it should be able to get tested on staging now.

@tomivs sorry I should've posted here as well https://github.com/Expensify/App/issues/8777#issuecomment-1111685591
Retest is Pass

🚀 Deployed to production by @chiragsalian in version: 1.1.57-17 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

How is QA on production going? I tested it out and it seems to be working.
cc: @kbecciv @mvtglobally


Add specify region

Hi, Sugawara san
I tried to export route53 record but got following error.
$ roadwork --version
roadwork 0.5.5
$ export AWS_ACCESS_KEY_ID='AKxxxxxxxxxxxxxxxxxxxxxx'
$ export AWS_SECRET_ACCESS_KEY='xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
$ roadwork -e
[ERROR] Aws::Errors::MissingRegionError: missing region; use :region option or export region name to ENV['AWS_REGION']

I think that it needs to specify region using AWS SDK v2. Therefore, I added to specify region.
Please confirm.
Thank you.

Thanks!


Deep links not working - Or the page must load all the needed resources

Load the resources needed for a page to work without the need to go back and click on each link individually.
For example: if I go directly to http://localhost/agent/1/sayings then we should load the agent 1 and then load the sayings for that saying.
Right now you need to go to the agents page, select the agent then click on sayings.
The router should handle loading the resources as they are needed and they correct order during page load or when a link is clicked, that way if you refresh the current page you will get the same results as clicking thru.

addressed with
https://github.com/samtecspg/articulate/commit/7e2f0be9c3cdcf26397db43b5e9ec86983dc2472


Couchbase: Add Certificate Authentication

Fixes https://github.com/mage-ai/mage-ai/issues/2796
Summary
Couchbase's Python API supports credential authentication here. This modifies the Couchbase template in Mage to allow for credential authentication in place of username and passwords.
This should allow organizations managing Couchbase Server using VPNs to use the connector.
Tests
Code review. I'm not sure how to test this.
cc:
@thomaschung408
@wangxiaoyou1993

You can install your branch in Mage and test the change
pip uninstall -y mage_integrations
pip install "git+https://github.com/your_repo.git@your_branch#egg=mage-integrations&subdirectory=mage_integrations"

@jamie-quickplay Let me know if you're able to test this change.

@jamie-quickplay Let me know if you still have problem testing this change.

Hey @wangxiaoyou1993 I can't test the change locally. After talking to my IT team, it looks like the problem was cross project permissions on a VM. So although I no longer need this, it may be useful for someone else.


how can i handle a network needs different image size

I have seen the tensorrt sample about faster-rcnn, it has a fixed image size, so when I want to use different image, I have two questions:
1.tensorrt will do the optimizations according to network's input layer, or the input shape:
layer {
  name: "data"
  type: "Input"
  top: "data"
  input_param { shape: { dim: 64 dim: 1 dim: 28 dim: 28 } }
}

2.when doing inference,  can  I CudaMalloc a memory which has a different size with input layer's shape param
is there a demo about variable size of input image size?

Hi qingzew, the imageNet, detectNet, and segNet classes automatically read the input size dimensions from TensorRT from the input layer, and perform re-scaling on the raw image/video that you feed to the API, so you shouldn't have to worry about this too much.

Hi @dusty-nv do you mean 'cudaPreImageNet()', but sometimes a net have a different size input to get a better precision, when I resize it to the same size,  I will have bad result, so any good solutions?

@qingzew have you found a solution or a snippet to help?

no, maybe a better idea is to set a max size

Hi qingzew,
If you set a max size (which I did), do you know how to retreive the result knowing its dimensions?
I have convolutions so output size is smaller with a smaller input and I have trouble identifying how the result is stored in the output buffer.

Hi @romzie
I don't what you are doing, detection or classification?  In fact, convolution and pooling layer reduce the dimension of the input image, but you can compute the scale of that, for example, a pooling layer usually has a scale 1/2, if you has point (x, y), you can get (x / scale, y / scale) in origin input

Both actually, using convolutions and pooling, so the output size is not an issue, I know it.
The issue is retrieving the result from TensorRT. I dont know how the output buffer is filled with the inference results. What I mean is that my output buffer has a maximum size so I need to truncate it to get results.

my idea is that when  i have max size input, i can place the input image to the center of the input blob, and padding 0, so i know the padding width and height, i can retrieve the result  now

@dusty-nv You mentioned that the detectNet class automatically resizes the input images, but I can't seem to find where that code appears. Is it baked into the cudaPreImageNetMean?

Hi qingzew, the imageNet, detectNet, and segNet classes automatically read the input size dimensions from TensorRT from the input layer, and perform re-scaling on the raw image/video that you feed to the API, so you shouldn't have to worry about this too much. For example, you can push an HD image into the API and it will automatically downscale it to match the network input size.

Hi, sorry to bring this back up after so much time, but I think your explanation begs the question about aspect ratios. I'd assume that the way these classes fit a bigger image into the required network size would be by resizing without keeping aspect ratio (otherwise it would have to crop some part and the decision of what part to crop would have to be left to the class, risking making the wrong call).
If that's the case (that the images are resized by stretching) would that deformation make it more difficult for the network to make a detection? That would depend on how the network has been trained I'd guess. If they were trained on 16:9 images stretched to 1:1 aspect ratios (like 300x300) is one thing, different from a 4:3 image stretched to 1:1, different from a training set of 1:1 images (probably cropped) that would fit perfectly the input aspect ratio.
Is there any information on how the training on those networks have been made? Or at least what would be the ideal shape of the image that should be passed to each of the different networks?
Thank you.
Best regards.
Eduardo


Update climate prereqs

2024 updates to pre-reqs. Big changes are removing the python advisors, the dates, and reframing to a refresher instead of pre-course for python. Also added a table directly into the markdown file instead of linking to the google sheet.

@iamzoltan let me know if there are any problems here!


Enable "useDefineForClassFields" in tsc --init

Search Terms
useDefineForClassFields, tsc --init, tsconfig
Suggestion
tsc --init's generated tsconfig.json should include "useDefineForClassFields": true
Use Cases
All major web and non-web engines now ship Define-style semantics.
So the safest approach is to ensure new TypeScript projects use standard web-compatible behavior.
Examples
It looks like we just need to add one line here.  I am happy to send a PR.
Checklist
My suggestion meets these guidelines:

[x] This wouldn't be a breaking change in existing TypeScript/JavaScript code
[x] This wouldn't change the runtime behavior of existing JavaScript code
[x] This could be implemented without emitting different JS based on the types of the expressions
[x] This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, etc.)
[x] This feature would agree with the rest of TypeScript's Design Goals.

I think this is a good idea. @ahejlsberg @DanielRosenwasser thoughts?

Three concerns we should discuss:

The emit is bulky for all targets <ESNext.
As of 4.0, we have errors around places where [[Set]] and [[Define]] differences are observable.
[[Define]] semantics are at stage 3, so strict standards compliance isn't really a motivator yet.

We have existing features with non-compliant emit, iteration for example, and errors to prevent the non-compliance from being observed, so there's precedence for prioritising good emit over standards compliance.
If we had multiple --init options, this would be the perfect addition to a "latest" init.

I added it to the recommended tsconfig base: https://github.com/tsconfig/bases/pull/15

From discussion in the design meeting: We decided not to do this until [[Define]] semantics ship in some ES20xx spec, and that ES20xx becomes the default target in tsc --init.
Currently the default target is ES5, which we need to rethink -- in my opinion, ES2015 at least should be the default.


No tutorial, No book...

i have search on google but can't find any tutorial or ebook to learn it T-T....

This is consistent, since there's no code, there should be no book and nothing to learn.

I found this tutorial and it worked for me:


Update README for DCM compatibility

This adds a line about the data source with the correct nomenclaute for DCM

How about other data sources?

Don't have access to Flygon yet, so did not know that. DCM has two separate fields for endpoints: Backend & Data Endpoint
DCM also has the ability to send raw data to multiple data endpoints, so it would still be good to include instructions for DCM as a separate line, with the caveat that Flygon can also handle this. I will await Flygon before updating this PR.
Are there any other data sources to keep in mind?

He dares to forget about Flygon... Minus points from the first PR!
I believe @Fabio1988 meant that Atlas doesn't use DCM and you can't split the traffic without external tools. Unless something has changed and I'm mistaken.

How about a "For DCM" http://ip:port and "For Other Data Sources" http://ip:port/raw to simply and be inclusive?


Lightbox Gallery: How do I install? (DNN Newbie)

I just downloaded the Lightbox Gallery Install ZIP latest release from GitHub, and unzipped it. Should I drop this folder into my DNN extensions folder? How do I get up and running?
I checked the documentation, but it's empty.
Thanks!

Hello:
Thanks for using the module and for your issue.  You'll find that installing any extension is generally the same, and really easy (as long as the developer/vendor is good).
http://www.dnnsoftware.com/community/learn/video-library/view-video/video/542/view/details/how-to-install-a-module-in-dotnetnuke-7

Please let me know if you need anything else.  Otherwise, I'll be closing the issue.


Update dnssd backend to resolve IPP-USB printers (or anything local) to "localhost"

One Avahi issue is that IPP-USB printers get resolved to the local hostname instead of localhost. Prior attempts at fixing this (#185) regressed regular HTTP usage.

[master 94eb38f0c] Fix DNS-SD lookups of local services with Avahi (Issue #970)
[2.4.x 62c01bb11] Fix DNS-SD lookups of local services with Avahi (Issue #970)