autoHeight doesn't work on Nested swiper with images loaded

This is a (multiple allowed):

[x] bug

[ ] enhancement

[ ] feature-discussion (RFC)

Swiper Version: Swiper 4.1.6

Platform/Target and Browser Versions:  IOS Safari.

Live Link or JSFiddle/Codepen or website with isssue:
PLEASE try it in mobile browser, thank you!

Without autoHeight:
http://hardpaper.com/demos/180-nested.html
With autoHeight
http://hardpaper.com/demos/180-nested_autoHeight.html

What you did
I tried to revise the nested swiper sample code with image loaded and I want the images fit the width of the mobile browser. So I set the width= 100% for each image loaded in the swiper slides.
Expected Behavior
Images should be loaded fit the screen width and align the top of the screen. The images in the vertical
swiper should flow one by one with the gap I set in the spaceBetween property.
Actual Behavior
I found if I don't set autoHeight in the horizontal swiper, there will be empty space on the above and bottom of images which I don't want to have.
After I set the autoHeight property, there is no space on the above of the images, but the vertical slide disappeared because the height of vertical swiper-wrapper  is set too high.
P.S. Remember, an issue is not the place to ask questions. You can use Stack Overflow
for that.
Before you open an issue, please check if a similar issue already exists or has been closed before.

This is how vertical slider works, it must have a limited height


Update i18n intros

Checklist:

[ ] I have read freeCodeCamp's contribution guidelines.
[ ] My pull request has a descriptive title (not a vague title like Update index.md)
[ ] My pull request targets the main branch of freeCodeCamp.
[ ] All the files I changed are in the same world language, for example: only English changes, or only Chinese changes, etc.

Closes #XXXXX

Re-running the CodeQL CI - it hit a 500 error.


Epic 7.7.1 imp

Error Management
Unit Tests
Code reorganizing

@TekuriSaiAkhil Could you please add some screenshots when you run your unit tests please?

Unit Tests


Issue word emphasis not working properly

"IsInstance type annotation" in the title makes the title disappear
We're not running emphasis on activity messages, e.g PRs etc. So backticks are "written out" vs used to emphasize the word

Example. I had `IsInstance` first in the title, but had to edit it since the title disappeared completely otherwise.


Virtual clusters PoC

Proof of concept

Codecov Report

Merging #21 (6f5e0a5) into master (d8c0cf6) will decrease coverage by 0.06%.
The diff coverage is 3.70%.

:exclamation: Current head 6f5e0a5 differs from pull request most recent head beddc40. Consider uploading reports for the commit beddc40 to get more accurate results

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files

@@            Coverage Diff             @@
##           master      #21      +/-   ##
==========================================
- Coverage   18.05%   17.99%   -0.06%
==========================================
  Files        1055     1050       -5
  Lines       44009    43584     -425
  Branches    10578    10416     -162
==========================================
- Hits         7945     7844     -101
+ Misses      36064    35740     -324

Flag
Coverage Δ

unit
17.99% <3.70%> (-0.06%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files
Coverage Δ

shell/detail/provisioning.cattle.io.cluster.vue
25.80% <100.00%> (ø)

shell/list/provisioning.cattle.io.cluster.vue
0.00% <ø> (ø)

shell/models/provisioning.cattle.io.cluster.js
13.47% <0.00%> (-0.64%)
:arrow_down:

... and 44 files with indirect coverage changes


Enabling settings to override by language

Problem:
{
    "lowlight-patterns.opacity": 0.33,
    "[json]": {
        "lowlight-patterns.rules": [".*\\$schema.*"],
    }
    "[go]": {
        "lowlight-patterns.rules": [".*\\!= err.*"],
    }
}

Currently when language-specific setting are applied, it raises an error such:
This setting cannot be applied because it is not registered as language override setting.
It could be much better to define language specific regexes. Since, they are not context-aware and there could be collisions of wanted and unwanted text between different languages.

Yup, we'd definitely want to have this. Would you be interested in sending a pr to support this ?
If not I'd take a look at this sometime next week or so.

It looks like there is a value called language-overridable for scope prop of extension parameters.
I could not figure out how to access language overriden config. vscode.workspace.getConfig("lowlight-patterns", editor.document.uri)still gives the variables defined in workspace and ignores language specific rules.


Hardcoded default region

dochttxt calls vbi_teletext_set_default_region() with hardcoded 8 (Poland) as region.
This should be configurable.

Fixed in 0.3.


[FEATURE] document loader google drive

Hello good evening, thank you for this project, it's great
I would like to propose the inclusion of the following feature
document loader for google docs in google drive
In our company that is the source of information, they are very useful to be able to modify them in real time.
https://python.langchain.com/docs/integrations/document_loaders/google_drive

is not necessary currently


No scala files analized?

Hello
First, thanks for you help developing this plugin :)
Now, i run the sonarScan sbt task but no saca files are analized.
This is my conf file:
sonar.projectName=my-app
sonar.projectKey=my-team-app
sonar.sources=app
sonar.scala.version=2.11.1
sonar.host.url=http://backoffice.develop.it/sonarqube/
sonar.java.binaries=target
If i dont include the "sonar.java.binaries" property, the execution fails with: "Please provide compiled classes of your project with sonar.java.binaries property"
When i set that property, the scanner only checks java, js, css and html files. But no scala ones.
This is the output:
[info] Loading project definition from /Users/dev-user/Desarrollo/proyectos/my-app/project
[info] Set current project to my-app (in build file:/Users/dev-user/Desarrollo/proyectos/my-app/)
[info] INFO: Scanner configuration file: /private/tmp/sonar-scanner-<IP_ADDRESS>7-macosx/conf/sonar-scanner.properties
[info] INFO: Project root configuration file: /Users/dev-user/Desarrollo/proyectos/my-app/sonar-project.properties
[info] INFO: SonarQube Scanner <IP_ADDRESS>7
[info] INFO: Java 1.8.0_121 Oracle Corporation (64-bit)
[info] INFO: Mac OS X 10.13.6 x86_64
[info] INFO: User cache: /Users/dev-user/.sonar/cache
[info] INFO: SonarQube server 6.7.1
[info] INFO: Default locale: "en_US", source code encoding: "UTF-8"
[info] INFO: Publish mode
[info] INFO: Load global settings
[info] INFO: Load global settings (done) | time=490ms
[info] INFO: Server id: AWHn2LOv4XHqZknEHeot
[info] INFO: User cache: /Users/dev-user/.sonar/cache
[info] INFO: Load plugins index
[info] INFO: Load plugins index (done) | time=325ms
[info] INFO: Process project properties
[info] INFO: Load project repositories
[info] INFO: Load project repositories (done) | time=245ms
[info] INFO: Load quality profiles
[info] INFO: Load quality profiles (done) | time=233ms
[info] INFO: Load active rules
[info] INFO: Load active rules (done) | time=3221ms
[info] INFO: Load metrics repository
[info] INFO: Load metrics repository (done) | time=236ms
[info] INFO: Project key: my-app
[info] INFO: -------------  Scan my-app
[info] INFO: Base dir: /Users/dev-user/Desarrollo/proyectos/my-app
[info] INFO: Working dir: /Users/dev-user/Desarrollo/proyectos/my-app/.scannerwork
[info] INFO: Source paths: app
[info] INFO: Source encoding: UTF-8, default locale: en_US
[info] INFO: Load server rules
[info] INFO: Load server rules (done) | time=1059ms
[info] INFO: Index files
[info] INFO: 188 files indexed
[info] INFO: Quality profile for java: Sonar way
[info] INFO: Quality profile for js: Sonar way
[info] INFO: Quality profile for web: Sonar way
[info] INFO: Sensor JavaSquidSensor [java]
[info] INFO: Configured Java source version (sonar.java.source): none
[info] INFO: JavaClasspath initialization
WARN: Bytecode of dependencies was not provided for analysis of source files, you might end up with less precise results. Bytecode can be provided using sonar.java.libraries property
[info] INFO: JavaClasspath initialization (done) | time=16ms
[info] INFO: JavaTestClasspath initialization
[info] INFO: JavaTestClasspath initialization (done) | time=1ms
[info] INFO: Java Main Files AST scan
[info] INFO: 2 source files to be analyzed
[info] INFO: Java Main Files AST scan (done) | time=288ms
[info] INFO: 2/2 source files have been analyzed
[info] INFO: Java Test Files AST scan
[info] INFO: 0 source files to be analyzed
[info] INFO: 0/0 source files have been analyzed
[info] INFO: Java Test Files AST scan (done) | time=5ms
[info] INFO: Sensor JavaSquidSensor [java] (done) | time=847ms
[info] INFO: Sensor Embedded CSS Analyzer Sensor [css]
[info] INFO: 36 source files to be analyzed
[info] INFO: Sensor Embedded CSS Analyzer Sensor [css] (done) | time=223ms
[info] INFO: 36/36 source files have been analyzed
[info] INFO: Sensor SmellMeasuresSensor [smells]
[info] INFO: Sensor SmellMeasuresSensor [smells] (done) | time=26ms
[info] INFO: Sensor SonarJS [javascript]
[info] INFO: 1 source files to be analyzed
[info] INFO: Sensor SonarJS [javascript] (done) | time=79ms
[info] INFO: 1/1 source files have been analyzed
[info] INFO: Sensor SonarJS Coverage [javascript]
[info] INFO: Sensor SonarJS Coverage [javascript] (done) | time=2ms
[info] INFO: Sensor FindBugs Sensor [findbugs]
[info] INFO: Loading findbugs plugin: /Users/dev-user/Desarrollo/proyectos/my-app/.scannerwork/findbugs/findsecbugs-plugin.jar
[info] INFO: Findbugs output report: /Users/dev-user/Desarrollo/proyectos/my-app/.scannerwork/findbugs-result.xml
The following classes needed for analysis were missing:
scala.runtime.AbstractFunction0
scala.Serializable
scala.runtime.AbstractFunction1
scala.runtime.AbstractFunction2
com.despegar.routing.filter.ComponentProvider
scala.runtime.AbstractFunction0$mcV$sp
play.api.mvc.WithFilters
com.myproject.routing.filter.HeadersFilter
.....
.....
.....
[info] INFO: Sensor FindBugs Sensor [findbugs] (done) | time=56757ms
[info] INFO: Sensor SurefireSensor [java]
[info] INFO: parsing [/Users/dev-user/Desarrollo/proyectos/my-app/target/surefire-reports]
[info] INFO: Sensor SurefireSensor [java] (done) | time=4ms
[info] INFO: Sensor JaCoCoSensor [java]
[info] INFO: Sensor JaCoCoSensor [java] (done) | time=1ms
[info] INFO: Sensor SonarJavaXmlFileSensor [java]
[info] INFO: Sensor SonarJavaXmlFileSensor [java] (done) | time=2ms
[info] INFO: Sensor Web [web]
[info] INFO: Sensor Web [web] (done) | time=337ms
[info] INFO: Sensor Snyk Dependency Check Sensor [despegardepcheck]
[info] INFO: Snyk installation OK. Executing plugin.
[info] INFO: Sensor Snyk Dependency Check Sensor [despegardepcheck] (done) | time=20ms
[info] INFO: Sensor Retirejs Dependency Check Sensor [despegardepcheck]
WARN: Retire command not found
[info] INFO: Sensor Retirejs Dependency Check Sensor [despegardepcheck] (done) | time=14ms
[info] INFO: Sensor Zero Coverage Sensor
[info] INFO: Sensor Zero Coverage Sensor (done) | time=46ms
[info] INFO: Sensor CPD Block Indexer
[info] INFO: Sensor CPD Block Indexer (done) | time=18ms
[info] INFO: SCM provider for this project is: git
[info] INFO: 38 files to be analyzed
[info] INFO: 38/38 files analyzed
[info] INFO: 6 files had no CPD blocks
[info] INFO: Calculating CPD for 33 files
[info] INFO: CPD calculation finished
[info] INFO: Analysis report generated in 221ms, dir size=220 KB
[info] INFO: Analysis reports compressed in 319ms, zip size=118 KB
[info] INFO: Analysis report uploaded in 1660ms
[info] INFO: ANALYSIS SUCCESSFUL, you can browse http://backoffice.develop.it/sonarqube/dashboard/index/my-app
[info] INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
[info] INFO: More about the report processing at http://backoffice.develop.it/sonarqube/api/ce/task?id=AWZeAEaDPe2FC9iyiYIB
[info] INFO: Task total time: 1:08.725 s
[info] INFO: ------------------------------------------------------------------------
[info] INFO: EXECUTION SUCCESS
[info] INFO: ------------------------------------------------------------------------
[info] INFO: Total time: 1:11.792s
[info] INFO: Final Memory: 34M/1593M
[info] INFO: ------------------------------------------------------------------------
[success] Total time: 73 s, completed Oct 10, 2018 9:42:48 AM
The line that call my atention is:
[info] INFO: 188 files indexed
Because I have 188 scala files
In the sonar report none Scla references appears
I missing something?
Thanks in advance for your help!

I suspected that could be that.
Thank you very much for the super fast response ;)
Maybe you could include this clarification in the doc? To prevent another goofies like me ask the same question hahaha.
Tanks!


Review feasible_ddp documentation

In GitLab by @cmastalli on Mar 27, 2019, 17:56
So far I have detected these two problems:

$Q_x$ and $Q_u$ missed the $L_x$ and $L_u$ terms, respectively. (section 1.5.2)
$k$ and $K$ have an error in its formula, it should be $k=Q_{uu}^{-1}Q_u$ and $K=Q_{uu}^{-1}Q_{ux}$ (section 1.5.2)
few extra typos

In GitLab by @cmastalli on Mar 28, 2019, 11:08
mentioned in merge request !150

In GitLab by @cmastalli on Mar 28, 2019, 11:08
Solved in !150

In GitLab by @cmastalli on Mar 28, 2019, 11:08
closed

In GitLab by @cmastalli on Mar 28, 2019, 22:36
mentioned in commit 85f24b0a6e5961c10b9e538680bb48fffa23cf18


update embed package

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 68.49%. Comparing base (7efcd57) to head (e28a7a9).

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

:exclamation:  There is a different number of reports uploaded between BASE (7efcd57) and HEAD (e28a7a9). Click for more details.
HEAD has 1 upload less than BASE

Flag
BASE (7efcd57)
HEAD (e28a7a9)

2
1

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #302      +/-   ##
==========================================
- Coverage   77.69%   68.49%   -9.20%
==========================================
  Files          35       35
  Lines        2511     2511
==========================================
- Hits         1951     1720     -231
- Misses        560      791     +231

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Attributes are not colored correctly

Attributes should be colored as types.
See below:

As we're just mirroring GitHub for the most part, I'm alright with them being uncolored.

I would rather see them coloured, but the colour scheme should be consistent:
If we use Visual Studio colouring, what do we use for Java? InteliJ, Eclipse or GitHub? Perhaps it's the best to stick with GitHub.


"Go to Definition" in SFC is broken in Yarn 2 PnP Project.

demo repo

install ZipFS extension
press F1 and set typescript version to workspace version, both volar and typescript.
open src/App.vue
right click on union function and click "Go to Definition" and error pop up.

It would have no issue if it's in .ts file.

For context:
We (Yarn) create a local TypeScript server file that adds the zip: protocol to the file paths that passes through tsserver.server.Session which allows the ZipFS extension to provide the content of the files and works perfectly fine with the default TypeScript server.
https://github.com/yarnpkg/berry/blob/2a421a616281cacfb4280e5c94fd643934148372/.yarn/sdks/typescript/lib/tsserverlibrary.js#L133-L160
It seems Volar doesn't use that patched Session but, like I mentioned in https://github.com/johnsoncodehk/volar/issues/248#issuecomment-864375594, I couldn't figure out where we would need to make changes for go to definition to work with Volar, @johnsoncodehk do you have any pointers?

@merceyz volar don't use TS server, it call TS api direct, so I think this patch don't working for volar.
Does Angular support this?

Close because this is not volar side problem.
If Angular support it please let me know, volar may have some way to support too by reference to Angular.


LateInitUsage -> LateinitUsage

Should fix #513.

I'd rather prefer we fail on a mismatch.

When we have a generating tool for the config this will be automated anyway?

That would be ideal.


Support for declaring title in frontmatter?

I have an existing set of notes where I have my page titles defined in frontmatter (the notes are used in gatsby-theme-brain), like this:
---
title: Drafts
aliases: ["drafts"]
---

Would you be open to a PR that checks the frontmatter for a title, but falls back to the first h1 if not found?

all my notes have titles in the frontmatter. would be happy to work on this as well. maybe make it an optional configuration?

Hi!
I think what you describe should be ok for the majority of users. I'll be happy to accept a PR with the change you mentioned, even better with a setting to turn it on/off (not sure which one should be the default though).
I will also try to do it myself as soon as I find time but unfortunately it might take several days.
Thanks again!

I have created a common issue for all feature requests regarding title discovery and linking systems. See #28 for the progress.


Partition skew when TensorizeIn computes 'max' of its integer columns

There appears to be an intermittent performance issue, where the "head at TensorMetadataGeneration.scala:113" job can take almost an hour in a single executor, while all other executors take less than 1-2 minutes.
Perhaps agg(Map[String, String]) is less efficient (or less Catalyst-optimizable) than agg(Column, Column*).

Hi @ddicato, thanks for raising this up.
Some follow-up items:

Could we confirm if it is an intermittent performance issue? It might be caused by cluster instability.
I'm okay to check in this change. But we might need your help to experiment and monitor its performance with some more experiments.

Please let me know what you think.


Provide / improve documentation on plugins

Hi!

It's not clear how plugins should be loaded - as a script tag, after loading picturefill? Same for all plugins?
Should docs be merged from respimage? E.g.: https://github.com/aFarkas/respimage/tree/gh-pages/plugins/mutation
Picturefill socs say "To use saveData, create a picturefillCFG array that runs before the main plugin". What is "main plugin", picturefill itself?

:+1:

@rayshan @capaj Does this PR answer your questions? If not, could you add what else you would like to see in that PR? Thanks! https://github.com/scottjehl/picturefill/pull/603

Reviewed, thanks!


Cannot save list attributes when it is a size-1 list

What happened?
I am trying to save an xr.Dataset object that has an attribute that is a list. When the list has size > 1, it works fine. But when it is a size-1 list of a string (like ['foo']) then when I open the saved dataset the list was converted into a "bare" string (i.e. just "foo" in the example above).
What did you expect to happen?
I would expect that saving the file would work the same way regardless of the list length
Minimal Complete Verifiable Example
import xarray as xr
import numpy as np

data = xr.Dataset(
    dict(
        foo=xr.DataArray(
            data=np.arange(10),
            dims=('i',),
            coords=dict(i=np.arange(10) * 10)
        )
    ),
    attrs=dict(size_1=['test'], size_2=['one', 'two'])
)
data.to_netcdf('test.nc')
roundtrip = xr.open_dataset('test.nc')

print(data.attrs['size_1'])  # gives ['test']
print(roundtrip.attrs['size_1'])  # gives 'test

MVCE confirmation

[X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
[X] Complete example — the example is self-contained, including all data and the text of any traceback.
[X] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
[X] New issue — a search of GitHub Issues suggests this is not a duplicate.
[X] Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Relevant log output
No response
Anything else we need to know?
 This is an upstream issue.


Issue for App preview

https://github.com/SAP-samples/teched2021-IIS160/blob/main/exercises/ex1/README.md
Exercise 1.3 Starting the App
Getting issue for preview application. Not have any blocks message for sight . Try to update port 4005 as suggested, still getting same issue. please help.
issue.docx
issue.docx

Hi @MinalGithub ,
sorry for not responding more early, I must have missed the notification.
In the attached screenshot, in the terminal I can see  error message:
'cds watch' needs '@sap/cds-dk' to be installed.'
This node dependency needs to be installed globally wrt your dev space, and is required for the CDS compiler command line tools. If not installed, the service can't be launched by running cds watch.
Package @sap/cds-dk should be available per default for dev spaces that were created with profile 'Full Stack Cloud Application', as described in Exercise 0 step 'Creating the development space'.
Could you please recreate the dev space and make sure that the right profile was selected?
Alternatively, you can install the package in your existing dev space by running terminal command
npm i -g @sap/cds-dk
Please confirm whether this helps.
Best regards
Christian


Read default ENV variables for unknown CI

Use the values of the recommended ENV variables for non-supported CI's.
See https://docs.coveralls.io/supported-ci-services#insert-your-ci-here
Solves #47

Pull Request Test Coverage Report for Build #120

6 of 6 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build #119:
0.0%

Covered Lines:
309

Relevant Lines:
309

💛  - Coveralls

Thanks for fixing this :+1:


System.Web.Util not included in adapters?

I'm trying to migrate a solution to .NET Standard 2.0 using the System.Web adapaters, and it's been working well until I came across a project where we use the System.Web.Util namespace, specifically the RequestValidator.
using System.Web.Util results in "The type or namespace name "Util" does not exist [...]"
Does this mean that the Util namespace isn't supported by this project? If so, are there any plans to add support for it later?

@g2petter Can you share more about what you're using the RequestValidator for?
@blowdart @GrabYourPitchforks Do you have any guidance on whether/how to approach replacing RequestValidator usage with something from the Core world?

It would be replaced with a custom middleware in which you'd inspect and validate the properties you want to validate. Validating body elements would be difficult though, as you'd have to buffer, read the body, and then get asp.net to parse it for things like form elements. I'm not sure if you can plug into the pipeline at a point after the request has been parsed but before routing happens, @Tratcher may have some ideas.

Middleware can easily read the request path, query, headers, and even small-medium forms. Validating request body content in any other format would require buffering and duplicate parsing.

@adityamandaleeka, we're using the RequestValidator.Current.InvokeIsValidRequestString method to check if certain strings contain potential XSS attacks.

Inspecting the content is not a robust safety measure. A better approach is to ensure that all output values are properly encoded.

We are also doing that. This is intended as an additional safety measure and to provide the submitting user with immediate feedback that the data isn't valid.
In the process of updating another solution to .NET Standard 2.0 I have also found that the HttpContext.Current.ApplicationInstance isn't available in the adapters. Will this be added? Is it preferred that I open a separate issue asking about that?

@g2petter Did you see the new issue created #273 about Applicationinstance? We'd like to understand your use case there.


Matrix tests: Initial tests for the matrix bridge, from #413 & beyond

The test files are in the zulip tree, but are not discovered by the unittest discover function at present.
The test runner in #431 with pytest will discover and run these tests, or simply using pytest as a command (or pytest <dir>) will work too.

Pushed updated commits:

Added comments
Amended code in last commit slightly
Use unittest asserts more consistently

@timabbott I separated out the tests to make them easier to apply independently, given the slowness in applying the #413.

Merged, thanks @neiljp!


Remove a transtion on the snippet component to avoid layout shifts

Description
Just a quick fix to some UI issues in snippet component
Best Way to Test

Type of Change

[x] Bug fix (non-breaking change that fixes an issue)
[ ] New Example
[ ] New feature in existing example

New Example Checklist

[ ] 🚀 Link to deployment URL on Vercel
[ ] 📱 Is it responsive? Are mobile and tablets considered?
[ ] 📚 README.md preview link in PR description (branch)
[ ] ⚙ Secrets have instructions for how to set up in the readme
[ ] 🛫 npm run new-example was run to create the example

will reopen on new branch with clean git history


impl Display and Error for FromStr::Err

This allows using .parse()? in functions returning anyhow::Result.

Nice, thanks for the contribution! Have pushed v1.5.0 with your changes included.


flyspell-prog-mode spell checking in string are missing

# adsf

asdf = 5
"asdf"

using flyspell-prog-mode on above file, the first asdf is underlined, the second not (as expected), but the third also not (not expected).
Note this works in normal julia-mode

Hi @mauro3 !
I think this must be something with the flyspell configuration. The first asdf has font-lock-comment-face whereas the second has font-lock-string-face. Hence, you must have font-lock-string-face in flyspell-prog-text-faces to make it work. I am not sure if we should modify this configuration inside julia-ts-mode.

Ok, thanks.  In other modes (julia-mode and python-mode) it works though.


[BUG] Backup - S3 Timeout

Describe the bug
In the backup UI, I get an error and the backups are not listed.
error listing backups: error listing backup volumes: Timeout executing: /var/lib/longhorn/engine-binaries/longhornio-longhorn-engine-v1.0.0/longhorn [backup ls --volume-only s3://PATH/], output , stderr, , error <nil>
Number of Volumes: 39
Number of Volumes with backup enabled: 32
S3 Bucket Size: 2TB
Expected behavior
Backups of Volumes are showed
Log
time="2020-11-05T12:55:15Z" level=warning msg="backup store monitor: failed to list backup volumes in s3://S3PATH/: error listing backup volumes: Timeout executing: /var/lib/longhorn/engine-binaries/longhornio-longhorn-engine-v1.0.0/longhorn [backup ls --volume-only s3://longhorn-production@ch-dk-2/], output , stderr, , error <nil>"
Environment:

Longhorn version: 1.0.0
Kubernetes version: 1.18.3
Node OS type and version: Centos 7.7

Additional context
On an empty bucket the backups are listed normally.
S3 Provider https://exoscale.com

32 volumes is not a big number. And this command only checks the volume itself, so 2TB of data should be out of concern. We need to check how we can reproduce this issue in our lab.

From the AWS SDK, the default time is 30 seconds https://github.com/longhorn/backupstore/blob/18aa87a/vendor/github.com/aws/aws-sdk-go/aws/session/cabundle_transport.go#L17
From the issue reported, the longhorn version is 1.0.0 which means that the list operation can't be fulfilled with 30 seconds.
At longhorn release 1.1.0, we add retry count to 3 https://github.com/longhorn/longhorn-engine/blob/v1.1.0/vendor/github.com/longhorn/backupstore/s3/s3_service.go#L28 which means the list operation tolerance time is 90 seconds.
@timmy59100 Have you tried to upgrade your longhorn cluster to version 1.1.0 to see the error is still happening?

From the AWS SDK, the default time is 30 seconds https://github.com/longhorn/backupstore/blob/18aa87a/vendor/github.com/aws/aws-sdk-go/aws/session/cabundle_transport.go#L17
From the issue reported, the longhorn version is 1.0.0 which means that the list operation can't be fulfilled with 30 seconds.
At longhorn release 1.1.0, we add retry count to 3 https://github.com/longhorn/longhorn-engine/blob/v1.1.0/vendor/github.com/longhorn/backupstore/s3/s3_service.go#L28 which means the list operation tolerance time is 90 seconds.
@timmy59100 Have you tried to upgrade your longhorn cluster to version 1.1.0 to see the error is still happening?

@jenting Still the same after upgrade to 1.1.0.
It's critical if we can't restore any volumes.

@jenting Still the same after upgrade to 1.1.0.
It's critical if we can't restore any volumes.

@timmy59100 could you please provide the ping latency between your LH cluster to the S3 provider?

@timmy59100 could you please provide the ping latency between your LH cluster to the S3 provider?

In my testing

3 control plane nodes & 3 worker nodes. Each node with 50G SSD and 4 CPUs and 8G memory
create 50 volumes and run a recurring backup every 5 mins at all volumes.

If the ping latency from LH cluster to the minio server more than 315ms, click "Backup" on GUI displays timeout error.
From the LH manager log, I found that all volumes get at once without pagination.
Therefore, one thing we could improve is when the user clicks "Backup" at GUI, we do not get all volumes at the same time, but with pagination.

In my testing

3 control plane nodes & 3 worker nodes. Each node with 50G SSD and 4 CPUs and 8G memory
create 50 volumes and run a recurring backup every 5 mins at all volumes.

If the ping latency from LH cluster to the minio server more than 315ms, click "Backup" on GUI displays timeout error.
From the LH manager log, I found that all volumes get at once without pagination.
Therefore, one thing we could improve is when the user clicks "Backup" at GUI, we do not get all volumes at the same time, but with pagination.

In my testing

3 control plane nodes & 3 worker nodes. Each node with 50G SSD and 4 CPUs and 8G memory
create 50 volumes and run a recurring backup every 5 mins at all volumes.
If the ping latency from LH cluster to the minio server more than 315ms, click "Backup" on GUI displays timeout error.

In my testing

3 control plane nodes & 3 worker nodes. Each node with 50G SSD and 4 CPUs and 8G memory
create 50 volumes and run a recurring backup every 5 mins at all volumes.
If the ping latency from LH cluster to the minio server more than 315ms, click "Backup" on GUI displays timeout error.

@jenting If I just ping the S3 interface it's under 1ms.
Can I measure this s3 specific?

@jenting If I just ping the S3 interface it's under 1ms.
Can I measure this s3 specific?

@timmy59100 it would be good if you could measure the s3 by aws s3 CLI

Install aws s3 in one of the LH manager pod https://docs.aws.amazon.com/cli/latest/userguide/install-linux.html
Configure the s3 credential credentialaws configure

List s3 objectstime aws s3 ls --endpoint-url <your-s3-endpoint-url> # add --no-verify-ssl if the endpoint is the self-signed certificate

From my testing, if I add a 1-second delay, the list s3 objects operation increase 3 seconds because 3 HTTP operations performed.

real    0m0.739s
user    0m0.528s
sys     0m0.077s

@timmy59100 could you please log in to one of the longhorn-manager pod and configure the related environment variable to the shell, the environment variable could get from the backup target credential secret (for example, aws-secret).
kubectl get secret aws-secret -o yaml

export AWS_ACCESS_KEY_ID=<your-aws-access-key-id-in-base64-encoded>
export AWS_SECRET_ACCESS_KEY=<your-aws-secret-access-key-in-base64-encoded>

Then, provide the time it consumes to us? thank you.
time /var/lib/longhorn/engine-binaries/rancher-longhornio-longhorn-engine-v1.1.0/longhorn backup ls --volume-only s3://longhorn-production@ch-dk-2/

I had a lot of trouble with this on backblaze as well, it would frequently timeout before bringing up the restore dialogue

Whilst I really don't recommend this (or at least copy to another bucket or snapshot it first) I did some hackery on my backup repository where I went in and Deleted a load of the .cfg files for the volume.
Obviously that's going to leave you with orphan blocks so you'll probably have to write off the backup repo after but it allowed me to actually restore the latest snapshot.

@dragon2611we cleanup orphaned blocks when you request a backup delete.
So you can request a backup delete operation from the longhorn ui and we will evaluate all blocks and cleanup orphaned ones.
This will also update the backup volume size to be correct.

Verified with Longhorn-master - 03/23/2021
Validation - Pass
With v1.1.0
Tried listing 50 volumes backups as per https://github.com/longhorn/longhorn/issues/1955#issuecomment-768128322 and it got timed out.
[longhorn-manager-rzvxz] time="2021-03-23T17:11:40Z" level=warning msg="HTTP handling error error listing backups: error listing backup volumes: Timeout executing: /var/lib/longhorn/engine-binaries/longhornio-longhorn-engine-v1.1.0/longhorn [backup ls --volume-only s3://backupbucket@us-east-1/backupstore], output , stderr, , error <nil>"
[longhorn-manager-rzvxz] time="2021-03-23T17:11:40Z" level=error msg="Error in request: error listing backups: error listing backup volumes: Timeout executing: /var/lib/longhorn/engine-binaries/longhornio-longhorn-engine-v1.1.0/longhorn [backup ls --volume-only s3://backupbucket@us-east-1/backupstore], output , stderr, , error <nil>"

With master
Longhorn listed the backups.

@jenting about 30 volumes with 8TB bucket (should not be that big.)
I created a new backup to see if the timeout comes again after some time, like last time.

@jenting about 30 volumes with 8TB bucket (should not be that big.)
I created a new backup to see if the timeout comes again after some time, like last time.

Thanks. And how many CPUs in the longhorn manager Pods?

@jenting Do we know what's taking the time? It's the listing, or parsing the config files, or anything else? If we can profile it, we might able to fix it from the root, by e.g. using another way to store the cfg files.

I think the problem is single S3 operation needs about 700-800ms in the user's environment plus we have a 2-hashed folder structure.
For example, total 32 volume backups, the worst case is 1 + 32 + 32 = 65 s3 list command queries.
It approximately need 65 * 800ms = 52 secs for list backups.
I think we could choose to remove the 2-hashed folder structure, to make the list of all volume backups in 1 list command query. The problem is the backward-compatible that we should help the user moves the folder structure. So, the folder structure would from
0c/5a/pvc-xxx
83/08/pvc-yyy

to
pvc-xxx
pvc-yyy

@jenting What do you mean by CPUs in Pods?
here is the CPU usage for a longhorn-manager pod:

I mean the number of CPUs, cat /proc/cpuinfo

8 CPUs.

@jenting I have timeouts for listing individual backups on a nfs share. Is the mechanism the same as with s3?
error listing backups for volume 'pvc-489cd2ed-6094-42c3-824c-14b0dd9b0c92': error listing backups: Failed to execute: /var/lib/longhorn/engine-binaries/rancher-mirrored-longhornio-longhorn-engine-v1.1.1/longhorn [backup ls --volume pvc-489cd2ed-6094-42c3-824c-14b0dd9b0c92 nfs://PATH], output context deadline exceeded , stderr, time="2021-06-02T08:15:31Z" level=error msg="context deadline exceeded" , error exit status 1

Hi, we also ran into some timeout issues while trying to recover the backups from s3. Is there any possibility to configure this timeout, or is it hard coded?

Hi, we also ran into some timeout issues while trying to recover the backups from s3. Is there any possibility to configure this timeout, or is it hard coded?

It's hard coded :(

Would it be possible to let the user configure this? This is preventing us from recovering backups for large files :(

Would it be possible to let the user configure this? This is preventing us from recovering backups for large files :(

We choose not to let the user configure this is because the browser also has its own timeout setting and some browser like Chrome did not allow the user to change it.
Which page do you encounter the timeout issue on? Click the Backup or clicks Backup -> BackupVolume?

error listing backups for volume 'nexus-restore': error listing backups: Failed to execute: /var/lib/longhorn/engine-binaries/longhornio-longhorn-engine-v1.1.1/longhorn [backup ls --volume nexus-restore s3://nov000005-km-prod@eu-central-1/production/longhorn/], output context deadline exceeded , stderr, time="2021-07-13T07:45:35Z" level=error msg="context deadline exceeded" , error exit status 1

This is the error, and it is shown when I select the backup option in the volume itself:

It is also shown if I go to backups directly und try to select the volume itself in the backups view

I have to admit, I missed that you've already released 1.1.2 :D
Will upgrade later and report back. Thank you for the Info!
We've also cleaned our s3 store of too many *.cfg files and reduced the retention of the backups.
However on 1.1.1 we still run into the timeout. (not always, but like 90% of the time)

It'd be good if you could choose the S3 bucket region near your Longhorn cluster, it'll decrease the network latency.
Like you use S3 region eu-central-1, it's better to put your Longhorn cluster in Europe.
Also, clean up old backups (back_backup-xxx.cfg) in the S3 bucket manually would be helpful.

Yea, our cluster is located in Europe, so we've already selected the nearest S3 location :)
Shouldn't the retain setting for the backups cleanup old backups?

@jenting Upgrade to 1.1.2 worked. Backups are available all the time now. Thank you! Keep up the good work!

Resolved in 1.1.2.


Configuring time format

I notice that the time format in _layouts/home.html and _layouts/post.html is hardcoded. Is it possible/would it make sense to make this configurable in _config.yml?
Am I correct that right now my main option to change the format is to make a copy of those files and edit those copies?

@ward you're correct about the copy part.
Are you interested in submitting a PR to handle date format in configuration file?

Sure, I can take a look at it. I should have some time for it over the weekend.

Well I looked at it on the tram home and it seemed simple enough. https://github.com/jekyll/minima/pull/70


Switch to window except some handles. Switch to last frame.

In some case need to switch to window with same title, but with some another handle. For this situation new method may be helpful.
In some case need to switch to frame which is included in another "parent" frames. For this purpose you can use new switchToLastFrame method

@dimand58 Method switchToLastFrame() actually switches to every frame, not only to the last one.

@dimand58 What is the real reason for using switchToWindowExceptHandles()?  Is it really useful to use both title and handles in the same method?
My initial intention was free user from all these handles that are absolutely unusable. Title is much more convenient and human-readable.

@asolntsev
Af course! Switching to the "child" frame impossible without switching to it "parent". (Frame not found exception)
switchToWindowExceptHandles is needed for situations when target window has same title as current window (in this case we can use handle for exception)

Can you add an automated test for the new function? Please look at FramesTest for example .

OK.
What about switchToWindowExceptHandles  ?

Yes, please this test too.
Sent from my HTC
----- Reply message -----
From: "Demin Dmitrii"<EMAIL_ADDRESS>To: "codeborne/selenide"<EMAIL_ADDRESS>Cc: "Andrei Solntsev"<EMAIL_ADDRESS>Subject: [selenide] Switch to window except some handles. Switch to last frame. (#175)
Date: Sat, Apr 11, 2015 11:50
OK.
What about switchToWindowExceptHandles  ?
—
Reply to this email directly or view it on GitHub.

@asolntsev , hi !
Please check new commit with tests implementation.
P.S. Can you release minor version of Selenide with selenium 2.46 ?

@dimand58 Yes, I am going to release Selenide 2.19 with Selenium 2.46 soon. Probably tonight.

Thanks! I look forward to =)

This comment in Selenium has led to performance degradation when we switch frames often (switching by name/id method in 2.46):
https://github.com/SeleniumHQ/selenium/commit/98161bc5c743918318e81be04f62cf37e49cd971
I created another solution for switchToLastFrame method, because of issue fixing in Selenium is very long process.

Method was enhanced, please review last comment. This version still fast, even on IE8/IE11.

dimand58
Well, now I've got the point. Thank you for the tests!
I have merged your pull request. Now i suggest the following improvements that seem reasonable for me:

Method switchToLastFrame() without parameters should switch to default content (now it just does nothing that may be confusing).
The name switchToLastFrame()is confusing. At least for me. It's not obvious that it actually switches to all frames one-by-one. And intention of this method (switching into inner frame) is not obvious. Let's rename it: Selenide.switchTo().innerFrame("parentFrame", "childFrame_2", "childFrame_2_1");
Use better API for switchToWindowExceptHandles():
switchTo().windowExceptHandles().
I am still in doubt. This method switchTo().windowExceptHandles() still seems quite inconvenient and unclear for me. It's just not convenient to user to store some previous handle in variable and use it later. Trying to invent some better api... How will you actually use this method? Can you bring a real example?

@dimand58 I have released Selenide 2.19 (with Selenium 2.46.0) and your pull request.


PERF OPTIMIZATIONS AND CROSS-PLATFORM SUPPORT

Pythonically optimized get path performance by reducing LOC 33%.
Added cross platform set support.

Awesome. But the import statement is no longer needed.

We will keep the import in case someone was depending on it for backwards compatibility.

Unfortunately this change increased the LOC in the shittydb.get function twofold, therefore increasing the slowness twofold.  Is there a workaround?


Update README.md

@loran-steinberger  while you are at it please update the Installing section of the readme i.e.
libraryDependencies ++= Seq(
  "com.lendup.fs2-blobstore" %% "core" % "0.1.+",
  "com.lendup.fs2-blobstore" %% "sftp" % "0.1.+",
  "com.lendup.fs2-blobstore" %% "s3" % "0.1.+"
)

that should have the latest version, since people might just be blindly copying this


[ClangImporter] Don't add duplicate search paths

Besides saving some calls to stat(), this also causes problems when the user specifies a search path that Clang already adds by default, like $SDKROOT/Library/Frameworks/. Why? Because Swift adds its search paths after Clang has already configured its defaults, but Clang reconfigures its search paths from scratch when compiling a module to a PCM file to cache. This led to system search paths being found sooner in the primary Clang instance than in the PCM files, which in turn resulted in the PCM files being considered out of date.
This isn't likely to affect people much in practice, but it's better to get right. (We ran into this during Doug's experiments in making /System/Library/PrivateFrameworks a default search path in Clang r313317; turns out that's problematic for other reasons as well.)
While here, tweak the filtering for Clang diagnostics so that next time something like this breaks we'll actually see it in the build log.
rdar://problem/34664596

@swift-ci Please test

Build failed
Swift Test Linux Platform
Git Sha - 308f2c6425306e2282978d040260e8bc71fc3f9f

@swift-ci Please test

Build failed
Swift Test OS X Platform
Git Sha - 308f2c6425306e2282978d040260e8bc71fc3f9f

@swift-ci Please test source compatibility

@swift-ci Please test source compatibility

@swift-ci Please test source compatibility

This is not at all urgent but I'd like to merge it before it gets stale. Review ping?
@swift-ci Please smoke test

Jenkins issue
@swift-ci Please smoke test Linux

/home/buildnode/jenkins/workspace/swift-PR-Linux-smoke-test/branch-master/swiftpm/Tests/SourceControlTests/RepositoryManagerTests.swift:199: error: RepositoryManagerTests.testBasics : Asynchronous wait failed - Exceeded timeout of 1.0 seconds, with unfulfilled expectations: Repository lookup expectation

@aciidb0mb3r, interesting or no?
@swift-ci Please smoke test Linux

@jrose-apple Looks random, have you seen this before?

Nope, but anything that does network access seems suspicious to be in the test suite.

It is mocked, nothing in the SwiftPM test suite requires network access :)

@swift-ci Please smoke test Linux

@swift-ci Please smoke test Linux

@swift-ci Please smoke test Linux

Thanks, Doug. Let's double-check that nothing changed…
@swift-ci Please smoke test

@swift-ci Please smoke test Linux


Add enum StrSlice under enum Evaluated

Description
after #966
If a function that handles a string is chained and executed on the query builder, the string is copied every time by the Clone trait. This overhead is solved by always having a sliced enum with a value.
Tasks

[x] add enum StrSlice
[ ] modify string function structure for StrSlice
[ ] tests

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

1 of 77 (1.3%)  changed or added relevant lines in 4 files are covered.
27 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-0.2%) to 97.734%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

core/src/executor/update.rs
0
1
0.0%

core/src/data/key.rs
0
3
0.0%

core/src/data/value/string.rs
0
21
0.0%

core/src/executor/evaluate/evaluated.rs
1
52
1.92%

Files with Coverage Reduction
New Missed Lines
%

core/src/executor/aggregate/mod.rs
27
85.56%

Totals

Change from base Build<PHONE_NUMBER>:
-0.2%

Covered Lines:
39296

Relevant Lines:
40207

💛  - Coveralls

If this structure is to be used in another function, it is a structure that borrows and returns an owned string, so the implementation of StrSlice {&str} fails and closes.


(typescript) optional property not highlighted correctly

Describe the issue
optional property (prop?: number) not highlighted correctly.
Which language seems to have the issue?
TypeScript
Are you using highlight or highlightAuto?
highlight? Using highlightjs through revealjs VSCode extension.
Sample Code to Reproduce
type Options = {
  srcdir: string,
  outdir: string,
  minify?: boolean
}

Result (using Github Dark):

Expected behavior
As seen above, rendered by Github

In TS can you write: prop ?: number?

Yes. That is a valid syntax.

I have the same problem even almost a year later. It might work to insert the space, but it is inconvenient apart from being an incorrect parsing of the properties.
Any prospects of this being corrected?

Bumping it up.

In TS can you write: prop ?: number ?

@joshgoebel yes, it is used all over the place in typescript. More often without space in between key name and a question mark.
Example from the typescript repo.

@joshgoebel  i believe i've managed to fix this please review #4114


Update donations filters header

Resolves #1871
Description
Update the donations page to have a proper filter title.

Type of change

Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?
This update was tested on the local environment.

Screenshots

@rodolfoasantos hey there -- looks like this failed earlier by a flaky/brittle test that had nothing to do with this PR. I'll take a look into it :).
Otherwise I think this is good to go! :)

@edwinthinks Thank you, looks like it's all set.


fix: Add the global header chunk

I think this chunk is a bit too much? We don't exactly want apps to pull it all, right?

Yea, I'm going back to the drawing board with a new idea.


util: add 0x16 example lines to unit tests

Closes #158

@valarnin Since you reviewed the broader refactoring on the example log lines, would you mind giving this a review too?


vmware

Can you explain some of these changes, please.
What labels need the rename app?
and this line:
if [[ $appversion == *"${appNewVersion}"* ]]; then
What do you expect to get by this change?
In general we would like label changes to be separate from other changes, if you can separate those, it would be great.
Thanks!

Seems like this PR has grown a lot. I don't think it will be vice for us to add a script-running mechanism to this project. And if we did something like this, we would not name it anything with bash, as we use zsh, so more like "shell" (after all it's also a .sh file it is looking for). But the scripts it is executing does not have any signatures, so it would be introduction of a feature that would circumvent previous effort.
The label haxm is not even downloading haxm, that is only part of the comments in the label. And haxm is actually a pkgInDmgInZip, and we do not have a mechanism for that, but not sure we need to run any script from the zip-file to run that installer anyway.
And why was all this put in a VMware PR.
I would think that all of this looks as so specialised features, it should be handled manually, but I will ask the others…

Did you read my last comment?
This PR has too many changes, and we are not prepared to integrate all of this.
Please make separate PRs for new labels, and separate PRs for feature changes. This is too much in one PR.
I have copied the insomnia label out for a future release.
I will close this PR.


Fix Rosetta Code "" parsing

A lot of the results are showing code as "type: commentary".

Let's fix this.

Works, thanks!


Update to use utility container and collection-ized playbook locations

Shall we create a v0.8.x branch before we merge this?

I would be fine with that

v0.8.x created, will merge this now


fix: respect rejectUnauthorized and ca opts when proxying https

This PR fixes https://github.com/usebruno/bruno/issues/362
Problem
When Bruno would make SSL/TLS connections through a proxy, it would not respect connection options like rejectUnauthorized or ca. This was making it difficult to use when going through a self-signing MiTM proxy, or using a custom certificate authority.
Cause
The issue was caused by an upstream bug in https-proxy-agent, where it doesn't permit options  to be passed down when upgrading sockets to TLS:

https://github.com/TooTallNate/proxy-agents/issues/194

The same underlying issue has been reported against multiple versions of that library over the years. As you can see in the comments, people have either switched to other libraries, or patched it to pass down such options:

https://github.com/TooTallNate/proxy-agents/issues/89
https://github.com/TooTallNate/proxy-agents/issues/121

Solution
The solution employed here is to patch the class, similar to what others have done.
When/if the issue is fixed in proxy-agents, that should be used instead.
Note to Reviewers:

I put this in the existing proxy-util.js file, which is currently duplicated for CLI and Electron. I didn't see a good way to just have one copy of the code. Maybe the JS package would make sense eventually -- there seems to be lots of opportunity for consolidating common code there.
I was able to test successfully using the UI, but not the CLI -- I think there may be a separate bug in CLI currently where the Collection-configured proxy is not actually attempted to be used -- the code seems to look for proxy.enabled: true but it's saved as proxy.use: true in my bruno.json file in the collection dir. I didn't try to address that as part of the PR, as it seems a separate topic.

Contribution Checklist:

[x] The pull request only addresses one issue or adds one feature.
[x] The pull request does not introduce any breaking changes
[x] I have read the contribution guidelines.
[x] Linked issue to the pull request.

@cwilper You're awesome !!!

there seems to be lots of opportunity for consolidating common code there.

Yup, post out v1 launch, we will consolidate this.

I was able to test successfully using the UI, but not the CLI -- I think there may be a separate bug in CLI currently where the Collection-configured proxy is not actually attempted to be used -- the code seems to look for proxy.enabled: true

Yup, this discrepancy needs to be fixed. We will be using enabled in both places.
Thanks again @cwilper !
Merged!


Inspection launches interactive pyplot windows

Appears that file inspection e.g., new raw .fif file type causes camera to launch pyplot window in non-interactive mode.
TODO
try a plt.isinteractive() in camera.py
check DB for MRI files

Another todo:

check why camera is called for fif files in the first place, i.e. what is the stacktrace.


URL not valid: https://github.com/microsoft/shell-intune-samples/tree/master/Misc/MDATP

[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: ae1c59b1-e4aa-6a18-7c34-720d0eb83c5d
Version Independent ID: f5a3fc3c-bec2-3d7a-1d61-e23a0c1f8d4f
Content: How to schedule scans with Microsoft Defender for Endpoint on macOS
Content Source: microsoft-365/security/defender-endpoint/mac-schedule-scan.md
Service: microsoft-365-security
Sub-service: mde
GitHub Login: @Dansimp
Microsoft Alias: dansimp

@tomrys Thank you for your feedback. I do not see the URL in this Document and also notice that the URL does not exist anymore. Are you referring to a different document which points this out?
Thanks
Sri

@yogkumgit  This can be closed.


OAuth2 Implicit grant type

Not sure where to ask this question, I'm about to implement OAuth2 for my application and the suitable grant type for my application is implicit. I've noticed that this library is not supporting it (or am I missing it), but the scala-oauth library does have it.
Is there a particular reason why it's not present and can I give it a shot to add it to this project?

Thanks @mauropalsgraaf!
Yeah, finagle-oauth2 was inspired (forked) from scala-oauth2 several years ago. At that time, scala-oauth2 was purely synchronous, which was a deal-breaker for adoption within a Finagle services (that are async by nature). As far as I know, scala-oauth2 now works with Scala futures, which makes it quite usable within Finagle. That said, I suggest you try using it directly (given the conversions between futures).
In the meantime, I should find some time to restructure finagle-oauth2 just to be a simple wrapper around scala-oauth2 instead.

Hey Vladimir,
Thanks for your response, I'll try to use scala-oauth2 directly. If there is anything I can help with, feel free to ask. I'm not an expert in Scala, but would love to help.
I'm closing this issue since my questions are answered


Removed unnecessary conversions

No need to convert to the same type.

Codecov Report

Merging #7557 into master will decrease coverage by <.01%.
The diff coverage is 70%.

@@            Coverage Diff             @@
##           master    #7557      +/-   ##
==========================================
- Coverage   41.25%   41.24%   -0.01%
==========================================
  Files         469      469
  Lines       63620    63620
==========================================
- Hits        26244    26242       -2
  Misses      33955    33955
- Partials     3421     3423       +2

Impacted Files
Coverage Δ

modules/lfs/locks.go
49.79% <0%> (ø)
:arrow_up:

modules/git/tree.go
46.05% <0%> (ø)
:arrow_up:

modules/repofiles/update.go
39.49% <0%> (ø)
:arrow_up:

models/repo_permission.go
73.77% <0%> (ø)
:arrow_up:

routers/org/setting.go
0% <0%> (ø)
:arrow_up:

modules/context/panic.go
50% <0%> (ø)
:arrow_up:

modules/git/repo_blame.go
0% <0%> (ø)
:arrow_up:

modules/git/notes.go
40% <100%> (ø)
:arrow_up:

modules/log/writer.go
97.35% <100%> (ø)
:arrow_up:

modules/log/colors.go
66.47% <100%> (ø)
:arrow_up:

... and 14 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cc384d7...82b6235. Read the comment docs.


pkg/list: list.Max sees empty list when unified with JSON on command line

commit 6f37a71
The following testscript example fails:
stdin foo.json
exec cue vet x.cue json: -
-- foo.json --
{
    "ports": [
        2,
        5,
        7,
        2,
        76,
        98,
        3
    ],
    "newPort": 99
}
-- x.cue --
import "list"

ports: [... int]
newPort: list.Max(ports) + 1

The error is:
newPort: error in call to list.Max: empty list:
    ./x.cue:4:10

The list isn't empty, so this should not fail.

This can be fixed by making JSON (and YAML) a first-class citizen in cue/load, which they should be.

This issue has been migrated to https://github.com/cue-lang/cue/issues/345.
For more details about CUE's migration to a new home, please see https://github.com/cue-lang/cue/issues/1078.


CSS Clean Up

Your idea:
Some styling in the Blank and 5ePHB theme stylesheets is duplicated. We should work through the theme stylesheets and eliminate any duplicate styling from the 5ePHB styling (unless we deliberately want to keep it for clarity).
I expect that this clean up will also be required on all other theme styling.

[ ] 5ePHB
[ ] 5eDMG
[ ] Journal

I'm not sure that duplication is a dire problem, per se. I'd be wanting to avoid any unnecessary dependencies, and to also honor the possibility that some "duplication" is actually simply contemporaneous/ephemeral coincidence.

I think it comes down to a question of "Exactly HOW blank is Blank?" Either the styling exists for every page (and thus exists on the Blank page) or it's theme specific and thus is declared only in the theme's own CSS.

I would posit that the "blank" styling would be a minimalist black & white form. Sufficient that headings can be distinguished from each other and paragraph text, that tables can be recognised as being tables and not some accidentally arranged blocks of text, etc.
There are various "normalize" or "reset" stylesheets floating about — some of them go too far though, rendering all text (i.e.even headings) to be the same size/weight. Anything that has body, p, h1, h2, h3 {font-size: 100%}. Hmm .. a bit of googling suggests I'm recommending a "normalize" stylesheet, and not a "reset" stylesheet
At best, it might be useful as the theme/style for https://homebrewery.naturalcrit.com/account and other (very) basic UI.
It shouldn't have any rules set up for book-specific constructs, such as descriptive notes, stat blocks, etc.


Context.params isn't setup in the MultiCommand.list_commands method

Expected Behavior
I can refer parsed options in MultiCommand.list_commands method.
Actual Behavior
I can't. Options aren't initialized in the list_commands.
I'm creating a CLI tool that dynamically loads subcommands from a file. And the location of the file is specified by an option of the CLI tool. I can refer the parsed option in MultiCommand.get_command but I can't refer it in MultiCommand.list_commands method.
I think the list_command method is called before finishing the parse_args method so we can't use parsed options in the list_commands.
https://github.com/pallets/click/blob/fef921a4d0c807673d7800af4dd4ca604f6489e4/src/click/core.py#L1242
https://github.com/pallets/click/blob/fef921a4d0c807673d7800af4dd4ca604f6489e4/src/click/core.py#L1217
I created the following file and I got an empty dictionary object by the logger.debug when I execute the following command.
https://github.com/polikeiji/bkm/blob/master/bkm.py#L94
python -m bkm -f AAA --help
Is this behavior the expected one?
Environment

Python version: 3.8.0
Click version: 7.1.2

Could you provide a shorter example that shows where you are calling MultiCommand.list_commands? I have created a example where it is being called late enough for Context.params to be populated:
import click

class NosyGroup(click.Group):
    def list_commands(self, ctx):
        print(f'Group ctx params: {ctx.params}')
        return super().list_commands(ctx)

@click.command(cls=NosyGroup)
@click.option('-a')
@click.option('-b')
@click.pass_context
def run(ctx, **kwargs):
    click.echo(f'run: {kwargs}')
    click.echo(f'params: {ctx.params}')

    click.echo('calling MultiCommand.list_commands:')
    commands = ctx.command.list_commands(ctx)

    click.echo(f'commands in Group: {commands}')

@run.command()
@click.option('-x')
@click.pass_context
def comm1(ctx, **kwargs):
    click.echo(f'comm1: {kwargs}')
    click.echo(f'params: {ctx.params}')

@run.command()
@click.option('-y')
@click.pass_context
def comm2(ctx, **kwargs):
    click.echo(f'comm2: {kwargs}')
    click.echo(f'params: {ctx.params}')

Output:
% multi -a foo -b bar comm1 -x bazz
run: {'a': 'foo', 'b': 'bar'}
params: {'a': 'foo', 'b': 'bar'}
calling MultiCommand.list_commands:
Group ctx params: {'a': 'foo', 'b': 'bar'}
commands in Group: ['comm1', 'comm2']
comm1: {'x': 'bazz'}
params: {'x': 'bazz'}


Implemented OnBoarding Wizard

Issue ga4gh/dockstore#743 has been updated.
Updated the setup page, which now includes the on-boarding wizard.

Updated pull request:

merged duplicated functions into one service.
changed  to a  for the navigation buttons of the OnBording Wizard page (i.e: previous, next, and finish)
changed component names to upper case (StarredEntriesComponent and AdvancedSearchComponent).


NullPointerException in BetterNpcHighlight causing client to freeze

I am using the Better NPC Highlight plugin for Vardorvis's axes. Every so often, the client completely freezes for a few seconds mid-fight. When I look at Runelite's logs, I see the following crash:
2023-08-24 09:39:09 EDT [Client] WARN  n.runelite.client.eventbus.EventBus - Uncaught exception in event subscriber
java.lang.NullPointerException: null
	at com.betternpchighlight.BetterNpcHighlightPlugin.onGameTick(BetterNpcHighlightPlugin.java:854)
	at net.runelite.client.eventbus.EventBus$Subscriber.invoke(EventBus.java:70)
	at net.runelite.client.eventbus.EventBus.post(EventBus.java:223)
	at net.runelite.client.callback.Hooks.tick(Hooks.java:211)
	at client.zf(client.java:23677)
	at client.bs(client.java)
	at bm.ay(bm.java:401)
	at bm.run(bm.java:380)
	at java.base/java.lang.Thread.run(Unknown Source)

In Better Npc Highlight, the only settings I changed were to highlight the true tiles for IDs 12225 and 12227.

Do you have the slayer plugin enabled?


Support for Autoscale, using VNF in L2 Networks.

ISSUE TYPE

Enhancement Request

COMPONENT NAME

L2 Networks, Autoscaling

CLOUDSTACK VERSION

4.19.0

CONFIGURATION

OS / ENVIRONMENT

SUMMARY

Autoscaling was introduce few months ago and so far it works well.
However in more enterprise environments, Autoscale is usually required for web servers, which needs a layer of security applicance in the front to prevent the service from being exposed to vulnerabilities.
VNF Appliances have been introduced in Cloudstack which allows users to bring in their own Virtual Network Appliances (eg. PFsense, FortigateVM) to help with this. But these work effectively only in a L2 Environment, because if it was used in a normal VPC, the default gateway for all VMs would be the Virtual Router, hence bypassing the VNF altogether.
As a cloud provider, looking to service enterprise customers, id like to be able to use both autoscaling and VNF appliances together. Hence, am wondering if it is possible to enhance the virtual router to forward specified traffic (based on specific rules eg. CIDR, Private IPs) to a specified VNF hosted within the VPC. Then, the VNF would be configured to handle the 'filtering/firewalling' and other sort of functions that the VNF is responsible to handle. And once done, it will route the traffic back to the Virtual Router.
This method would not only solve the autoscaling issue, but would solve the issue in general where customers cant filter their traffic to the internet/virtual router if using a VPC. Also would solve the pain of using ACL Lists right now, where if a VPC has a large number of networks, ACL rules are too tedious to manage. This would be made much easier with a VNF.
Just ideas, may sound crazy, open for discussion.
STEPS TO REPRODUCE

EXPECTED RESULTS

To be able to use VNFs together with Autoscaling Function.

ACTUAL RESULTS

VNF is only available for L2 Networks. If VNF is used for VPC, every single VM would need to be configure to use the VNF as the default gateway, which is a tedious approach for cloud. (Feels like going back to on prem practices)

Currently the autoscale feature has dependency on loadbalancing service. For now it only works with isolated networks or vpc with lb.
Do you want VNF to be working as the LB provider in the isolated network with VR( as source nat) ?

Hi @weizhouapache ,
Allowing VNF to be the LB Provider would be great, but come to think of it, i think it will pose a challenge because Cloudstack will need to recognise the different metrics that the VNF outputs for Scale Up, Scale Down etc.
If Autoscale relies on Virtual Router for now, then would this suggestion below be easier to implement instead?

In this diagram:

No change to the way Autoscaling works is required.
Virtual Router just needs to be allowed to route incoming traffic from the internet, to the VNF.
VNF will then return the traffic back to Virtual Router.
Any VNF can be used as its just routing of traffic.

I believe a UI is needed to specify the conditions in which the Virtual Router will route the traffic to the VNF. I think basic conditions will do, as the more advanced stuff will be done by the VNF.

Hi @weizhouapache ,
Allowing VNF to be the LB Provider would be great, but come to think of it, i think it will pose a challenge because Cloudstack will need to recognise the different metrics that the VNF outputs for Scale Up, Scale Down etc.
If Autoscale relies on Virtual Router for now, then would this suggestion below be easier to implement instead?

In this diagram:

No change to the way Autoscaling works is required.
Virtual Router just needs to be allowed to route incoming traffic from the internet, to the VNF.
VNF will then return the traffic back to Virtual Router.
Any VNF can be used as its just routing of traffic.

I believe a UI is needed to specify the conditions in which the Virtual Router will route the traffic to the VNF. I think basic conditions will do, as the more advanced stuff will be done by the VNF.

currently Autoscaling supports 5 counters, see https://docs.cloudstack.apache.org/en/latest/adminguide/autoscale_with_virtual_router.html#id2
the cpu/memory usage are fetched from hypervisor
the network/lb metrics are fetched from virtual routers.
for now, cloudstack does not get metrics for Autoscaling from other sources.
it looks like you need a feature like traffic mirroring.
@btzq

@weizhouapache actually in my diagram above, im suggesting to not use VNF to provide Autoscaling Feature. Let the current logic work as is.
The end result we are trying to achieve is to implement some sort of security implementation from the internet, just before it gets distributed to the VMs in the VPC.
So in my diagram, i was suggesting another way to achieve that is by having the Virtual Router, route traffic to the VNF.
VNF will then return the traffic after processing. Then, the Router can continue to distribute the traffic to whichever services need it via the existing logic (Eg. Autoscaling)

@weizhouapache actually in my diagram above, im suggesting to not use VNF to provide Autoscaling Feature. Let the current logic work as is.
The end result we are trying to achieve is to implement some sort of security implementation from the internet, just before it gets distributed to the VMs in the VPC.

Internet -> VNF -> Virtual Router -> Autoscale Grousp (Ideal Scenario)

But this is not possible because the default gateway must be the Virtual Router.
So in my diagram, i was suggesting another way to achieve that is by having the Virtual Router, route traffic to the VNF. VNF will then return the traffic after processing. Then, the Router can continue to distribute the traffic to whichever services need it via the existing logic (Eg. Autoscaling)
The implementation would look like this.

Internet -> Virtual Router -> VNF -> Virtual Router -> Autoscale Groups

the second option looks like
https://docs.aws.amazon.com/vpc/latest/mirroring/what-is-traffic-mirroring.html
not exactly the same
If you have full control on the virtual router and VNF, you can do PoC by some manual changes.
I have no idea if it is possible.
option 1 make more sense to me. the public Ip needs to be associated to the VNF.

@weizhouapache traffic mirroring is a great tool but it doesnt allow for taking action against malicious traffic.
Would Option 1 be more straightforward to implement? And in this option, i suppose assigning a public ip to the vnf should not be an issue.
But how would the traffic then get send back to the virtual router?
And how would the virtual router recognise the vnf as the new default gateway?

@weizhouapache , ive updated the diagram to reflect Option 1 better.
Is my understanding right?

@weizhouapache , ive updated the diagram to reflect Option 1 better.
Is my understanding right?

this seems like a discussion that ccan lead to several implementation ideas/issues. I'm converting it to a discussion.


How to enable logging or instrumentation?

Hi, I'm addding support for MongoDB but I get a browser notification saying:
Invalid `prisma.user.create()` invocation:
Prisma needs to perform transactions, which requires your MongoDB server to be run as a replica set. https://pris.ly/d/mongodb-replica-set

while my terminal only shows the error POST /api/auth/join 500 and that's it.
How do I enable logging / instrumentation to understand what's going on?
Edit: please, besides changing log: ["error"] in lib/prisma.ts

Hi @nopmop, unfortunately there is nothing more than the logging and tooling prisma provides in this case. We can't do much when the database is down.


Added 2 custom metrics for used/limit bytes

These metrics will be exposed via the quota plugin tool using the Yammer API

@scholzj @tombentley - Is there another file that would need to be added for the build tests to pass? I have tested the changes on my end and I'm able to run the plugin with the custom metrics in. Maybe the kafka-quotas-plugin.iml file?

@scholzj @tombentley - Is there another file that would need to be added for the build tests to pass? I have tested the changes on my end and I'm able to run the plugin with the custom metrics in. Maybe the kafka-quotas-plugin.iml file?

The build is failing because of this:
[WARNING] Used undeclared dependencies found:
[WARNING]    com.yammer.metrics:metrics-core:jar:2.2.0:compile

You need to add the Yammer metrics as dependency (provided again I guess)

How do the used bytes differ from what Kafka itself reports already or what Kubernetes reports? It would be good to understand the differences since we might need to explain it to the users as well (who are confused by this already even without these metrics :-o).

The plan for the used bytes metric was that it would be in one location (1 source of truth so to speak). It replicates what is done by using kubelet_volume_stats_used_bytes. @k-wall mentioned it might be good to have the metric served by the plugin if we were going to use this for metrics in the future.

Also, how is the new metric named when it is consumed? Is it sufficiently clear form it where is it coming from? LimitBytes and UsedBytes alone is quite generic and might be confused with some Kafka native metrics.

I can add more info to the names to indicate that it comes from the quota plugin

Oh, sorry, it looks a comment I had yesterday was still pending. But you're resolved it anyway @npecka.


fix for *_TMY3.epw reading issue #782

This PR is for solving epw file reading problem.
A note is added in docstring of epw.py and a validity check is added in atmosphere.first_solar_spectral_correction. This validity check is done according to kwargs min_pw and max_pw. If the value is higher, np.nan is returned.
pvlib python pull request guidelines
Thank you for your contribution to pvlib python! You may delete all of these instructions except for the list below.
You may submit a pull request with your code at any stage of completion.
The following items must be addressed before the code can be merged. Please don't hesitate to ask for help if you're unsure of how to accomplish any of the items below:

[x] Closes issue #782
[x] I am familiar with the contributing guidelines.
[x] Fully tested. Added and/or modified tests to ensure correct behavior for all reasonable inputs. Tests (usually) must pass on the TravisCI and Appveyor testing services.
[ ] Adds description and name entries in the appropriate docs/sphinx/source/whatsnew file for all changes.
[ ] Code quality and style is sufficient. Passes LGTM and SticklerCI checks.
[x] New code is fully documented. Includes sphinx/numpydoc compliant docstrings and comments in the code where necessary.
[ ] Pull request is nearly complete and ready for detailed review.

Brief description of the problem and proposed solution (if not already fully described in the issue linked to above):

Hi,
I need some help to understand why stickler-ci finds lints only for the lines I have edited in epw.read_epw docstring and not for the others previously existing lines (but which have the same lint however: E501 line too long).
And for the rest, any other feedback is welcomed as well.
Thanks in advance!

I need some help to understand why stickler-ci finds lints only for the lines I have edited in epw.read_epw docstring and not for the others previously existing lines (but which have the same lint however: E501 line too long).

I believe this is to assign lint responsibility to the most recent editor. It would  interfere with usage of git-blame to troubleshoot code since it would insert a format change in that line that might obscure a bug. The alternative would be the CI making you responsible for more lines than you edited. You can imagine this being dangerous if done on running code as opposed to the documentation in your case.
I'm sure it'll stop barking at you if you shorten those lines.

Thanks for your feedbacks!
New version is pushed. I have made the suggested change, except for editing the table in docstring of read_epw() (sorry for that, I took the easy way to get around the problem ^^')


QueryException when running 'twill:install'

Installed the package to an existing Laravel 7 application from Composer. When running 'php artisan twill:install' command, I saw migrations and config files are copied successfully. Then it asked for an email for Superadmin, after entering a valid email, it threw a QueryException as following:
SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that corresponds to your
MySQL server version for the right syntax to use near 'where email = ?' at line 1 (SQL: select count(*) as aggregate where email =<EMAIL_ADDRESS>MySQL version is  5.7.30. Searched open and closed issues where some mentioned about JSON type was not supported on some MySQL version. But my application also uses JSON type in a couple of tables. so it is not my case. Any idea?

Hello, please feel free to reopen this issue and provide a reproducible setup of more information about your environment. Thanks!


'type' in Input config may be plugin name

Support for input plugin name in 'type' input configuration key for Zend\InputFilter\Factory::createInput().

Ping @weierophinney. Is there anything else I should add to this PR? Thank you.


Floorplan - convert non-progress, non-actionable output to debug info

Description
This output doesn't really tell me anything about how far along the process is, what is going on and I can't use the information for anything, so probably should be debug output:
[INFO MPL-0023] Calling Partitioner.
[INFO PAR-0016] Partitioning using multilevel methodology.
[INFO MPL-0023] Calling Partitioner.
[INFO PAR-0016] Partitioning using multilevel methodology.
[INFO MPL-0023] Calling Partitioner.
[INFO PAR-0016] Partitioning using multilevel methodology.
[INFO MPL-0023] Calling Partitioner.
[INFO PAR-0016] Partitioning using multilevel methodology.
[INFO MPL-0023] Calling Partitioner.
[INFO PAR-0016] Partitioning using multilevel methodology.
[INFO MPL-0023] Calling Partitioner.
[INFO PAR-0016] Partitioning using multilevel methodology.
[INFO MPL-0023] Calling Partitioner.
[INFO PAR-0016] Partitioning using multilevel methodology.
[INFO MPL-0023] Calling Partitioner.
[INFO PAR-0016] Partitioning using multilevel methodology.
[INFO MPL-0023] Calling Partitioner.
[INFO PAR-0016] Partitioning using multilevel methodology.
[INFO MPL-0023] Calling Partitioner.
[INFO PAR-0016] Partitioning using multilevel methodology.

Suggested Solution
Convert to debug output
Additional Context
No response

@oharboe This will be addressed in #4452.


Enhancement:  Protect parsing from errors

The call to serializer.parse() should be inside a try..catch.  Users may directly tamper with stored values which might cause errors while parsing.

My preference is to only catch when there is something that can be done about it.
For situations where errors are expected and custom serializer can be passed.

Things can be done.  The ideal scenario would be to run an onError handler and let the consumer of the store to determine what to do.
onError, (just as validate in my other issue) would be passed as options.
This way the consumer of the value will be assured that any malformations will be notified through this mechanism.
Why leaving it as-is is bad
Because your application may have 1000 entry points that could trigger reading the malformed.  Are we going to write try..catch everywhere?  No.  We would most likely have to introduce a proxy of some kind.

I understand, but I want to be careful about how many options this repo provides. The more that is added, the more docs are needed and it brings more complexity for an super edge case.
Feel free to fork and add any edge cases you need.


Rc 1

Includes:

updated readme
had to restructure all of the button and inputs mixin so they can be easily composable within quiche, thus decreasing the size of css there.

Todo:

include usage write up within docs, now that individual mixins are removed
update git tagging. All merges should be tagged. When package.json version changes a release should be created.

:ship:


Read progress indicator for a more responsive user experience

As reading of packages now includes many computation-heavy validation steps, we should add a progress indicator for this task. This is only a minor feature, but certainly important for the user experience.
Intuitively I would go for something like System.ProgressBar. But maybe it's better to craft something from scratch with System.Console.ANSI which is already in our long list of dependencies?

An interesting issue occurs if there is a broken package among the ones currently loading. Then the progress indicator will be interrupted to print the relevant error message. It's not trivial to do so in a visually pleasing way.

Added a simple counter in 4d6222af107271bdd9d83e93f51d4d79602d4126


what should i do if i wanna evaluate this model on my own dataset ?

Hi kenshohara !
I want to evaluate this model on my own dataset without modifying too much the code. I have skimmed through the code and then i think i should modify the file dataset.py. However i dont know how to begin to modify it. Would you please give me some suggestions ? Thanks

Also, I noticed that the class annotation is loaded in json files and the file seems to be generated by utils/hmdb51_json.py and so on. However there is no introduction  about those files. Would you please introduce those files or tell me how to generate the json files of my new dataset. Thanks

Yes, you should implement your own yourdataset.py.
For details, please read some documents for dataset class of pytorch.
This tutorial may be useful for you.
http://pytorch.org/tutorials/beginner/data_loading_tutorial.html
The json file is not needed.
You can use any file format if you implement your own dataset class.

@FesianXu, hi
Could you implement your custom dataset?
If so, how did you solve issue with json file?
thank you.

@Fazlik995 Hi
The problem was a long time before and I can't find the project files now. But I think kenshohara already gives a good explanation about the method about custom dataset implementation.
First the json files are only a way of label annotation so if your own datasets are simply some action label or something like that, just ignore the json files, maybe name your sample files like:
A01_P01_V01.avi would help you in later dataset customing. (Axx for action label, Pxx for person id and Vxx for view id for example.)
Second, to make a custom dataset, you just need to follow the intructions in:
http://pytorch.org/tutorials/beginner/data_loading_tutorial.html
make sure that the custom class method __getiem__ would return the items you need (e.g. sample matrix and labels), always you need to sparse the sample's name here to get the corresponding sample's label, And that is how the json files work here - to get the corresponding label in a formatting way.
So the conclusion is that, you don't have to use the json files.

@FesianXu @Fazlik995
Hi, I got the accuracy of val.log, but I want to calculate the accuracy of video level, where can I get the val.json file and how to use eval_ucf101.py?

@FesianXu thank you for.

@slighting666
If you got val.log then run command below:
python main.py --resume_path ''result_path/save200.pth" --no_train --no_val --test --test_subset val
in result_path you will receive val.json file.
Then add below code to eval_ucf101.py:
if name == 'main':
ucf = UCFclassification('/home/fazlik/Desktop/3D-ResNets- PyTorch/ucfTrainTestlist/ucf101_01.json', '/home/fazlik/Desktop/3D-ResNets-PyTorch/result/MUNet_result/val.json', subset ='validation', top_k=1)
ucf.evaluate()
print(ucf.hit_at_k)
Last,  run utils/eval_ucf101.py

@Fazlik995  ok，thanks!
Clip level is a video that randomly selects the beginning in time and then 16 consecutive frames. I want to know how many clips are extracted from a video？


Object Graph Handlers:  Blacklist of own properties, with local setters for blocked properties

Same as issue 11, but with a blacklist instead of a whitelist.

API:  membrane.modifyRules.filterOwnKeys(fieldName, proxy, filterCallback, filterThisArg);
For the filterCallback and filterThisArg, refer to:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/filter

Duplicate of #11 given the API mentioned there.


Fix DQMHarvest input data placement in MSTransferor; print exception error

Fixes #9948
Status
In development
Description
DQMHarvesting input data placement with Rucio needs to be performed at the container level. Also print the exception error in case it fails to create a Rucio replication rule. Last but not least, print the number of DIDs getting pinned under a given container name (note that it could be a single DID which is actually the container name itself...)
Is it backward compatible (if not, which system it affects?)
yes
Related PRs
Complement (read, bug-fix) to the DQMHarvest bit provided by https://github.com/dmwm/WMCore/pull/9889
External dependencies / deployment changes
none

tests went fine. I still want to run a final test once it gets cleanly in a new release though.


ttl.tensor.rsqrt operation breaks with low PCC on some ranges [Wormhole]

ttl.tensor.rsqrt operation breaks with  low PCC on some ranges.
To Reproduce
Steps to reproduce the behavior:

Checkout main branch
Run unit test test_eltwise_rsqrt_in_depth.py using the command:
pytest tests/tt_eager/python_api_testing/non_working_unit_tests/grayskull/test_eltwise_rsqrt_in_depth.py
[PS. Unit test is in grayskull subfolder since its the same issue on gray skull card]

Expected behavior
There are test cases presented in test_eltwise_rsqrt_in_depth.py  unit test which go through various ranges of input values and 126 of them  are expected to fail with low PCC error.
Ranges swept:

1.4 - 1.5
1.5 - 1.6
...
4.9 - 5.0
5.0 - 6.0
6.0 - 7.0
...
99.0 - 100.0

Test cases with most of those ranges fail with low PCC (<0.99)

Hi @nemanjagrujic ,
I don't see the file name test_eltwise_rsqrt_in_depth.py from the path tests/tt_eager/python_api_testing/non_working_unit_tests/wormhole/test_eltwise_rsqrt_in_depth.py on the latest main.
Is the issue is still persistent?

Hi @nemanjagrujic ,
I don't see the file name test_eltwise_rsqrt_in_depth.py from the path tests/tt_eager/python_api_testing/non_working_unit_tests/wormhole/test_eltwise_rsqrt_in_depth.py on the latest main.
Is the issue is still persistent?

@ruthreshx Was deleted in the meantime (although it fails). Look in branch ngrujic/test_elt_rsqrt_in_depth. I recreated file from tag and checked (it still fails). Ill try to merge branch to main soon.

@ruthreshx Unit test is now merged to main.

Hi @nemanjagrujic ,
Rsqrt has been implemented using the Newton's method iteration. Hence the accuracy is too low was an expected one.
Between I could see the supported range is -10 to 10. I would suggest to fix inc rate as 0.5. Generating the values between the low ranges will lies into low PCC.
detail::bind_unary_op_with_param(
            m_tensor, "rsqrt", &rsqrt,
            py::arg("fast_and_approx") = true,
            R"doc(Returns a new tensor with the reciprocal of the square-root of each of the elements of the input tensor ``{0}`` for the input range -10 to 10.)doc",
            R"doc("Indicate true for approx and fast mode; false for accurate and slow mode", "bool", "default of true")doc"
        );

Tested, works now.


runtime: Windows DLL preloading attack possible for winmm.dll

Go 1.11 seems vulnerable to dll preloading on windows with winmm.dll. It looks like #14959 mostly fixed this, kernel32.dll etc are protected, but winmm.dll still seems to be affected. It seems to be loaded implicitly by the go runtime — https://github.com/golang/go/blob/6174b5e21e73714c63061e66efdbe180e1c5491d/src/pkg/runtime/thread_windows.c#L31 — but I notice is not listed with the other safely loaded DLLs — https://github.com/golang/go/blob/6174b5e21e73714c63061e66efdbe180e1c5491d/src/pkg/syscall/zsyscall_windows_amd64.go#L9-L19
What version of Go are you using (go version)?

$ docker run --rm golang:1.11 go version
go version go1.11.2 linux/amd64

Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
go env Output
$ docker run --rm golang:1.11 go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build782466443=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Create a main.go with:
package main

import "fmt"

func main() {
	fmt.Println("Hello world")
}

Cross-compile for windows:
docker run -v $PWD:/go -e GOARCH=amd64 -e GOOS=windows --rm golang:1.11 go build -o test.exe main.go

Copy test.exe to a windows vm

Add a winmm.dll beside test.exe with contents not a dll

Double click test.exe

What did you expect to see?
"Hello world"
What did you see instead?

Thank you for filing this bug @sj26 !
I'll page some folks who can help both with Windows and security /cc @alexbrainman @bradfitz @jordanrh1 @FiloSottile
In the future @sj26, please report security bugs to<EMAIL_ADDRESS>as recommended at https://golang.org/security

It looks like #14959 mostly fixed this, kernel32.dll etc are protected, but winmm.dll still seems to be affected. It seems to be loaded implicitly by the go runtime —

kernel32.dll also loaded by runtime - see many lines ending with  "kernel32.dll" in os_windows.go file. Going by your logic, kernel32.dll must be as broken as winmm.dll. I suspect, you can do your trick with kernel32.dll, if you make your program more complicated.
I tried your trick with notepad.exe and corrupted kernel32.dll, and notepad.exe does not start. And I do not see much difference between notepad.exe and Go built executable. They both have import table in the executable that specifies which DLL and functions to use. And Windows program loader searches for DLLs as per standard rules. How else do you propose Windows executable access OS?
Leaving this to real security experts.
Alex

I confess I don't use Windows and don't understand its dynamic library loading in detail. I'm reporting an issue we were alerted to which seemed odd. It sounds like a bug that some libraries would be protected and some would not. I didn't consider it a high severity security issue, it just seemed like a library search order problem.
kernel32.dll does not seem affected, nor any other dll I could find referenced, only winmm.dll.
The only difference I could find is that unaffected dlls are referenced in that generated syscall source file, and then again referenced in runtime, but winmm is only referenced in runtime. Perhaps this changes when and how they are loaded?
I notice that back in #14959 the effort was to switch these well-known libraries to use LoadLibraryEx with a flag to make sure they were loaded only from the system directory, not the regular search path. I don't understand the codebase deeply enough, but I couldn't find evidence of this any more, I can only find LoadLibrary[A|W]. So I'm not sure how only winmm.dll seems affected.
Perhaps this is because of Windows' underlying "known dlls" functionality?
I'll leave the sleuthing to folks who understand the functionality. Maybe the answer is that this is not a problem, it's just how Windows functions?

Hi guys, I've been a Windows dev for some time and I do stuff in security. I'm not affiliated with the project, I just stumbled upon this bug looking for another issue.
So, the default DLL search path starts with the application directory itself by default. This is because Windows considers application directories to be bundles, much like OSX .app folders: https://blogs.msdn.microsoft.com/oldnewthing/20110620-00/?p=10393 and https://blogs.msdn.microsoft.com/oldnewthing/20161013-00/?p=94505
KnownDLLs was a Longhorn (Vista) optimization, which loads the listed DLLs into memory at boot and then every time an application asks for one the DLL is mapped in automatically, without ever searching the disk for the file and running the PE loader again. You can probably see why this is a performance optimization, avoiding these disk hits for every process. This has some security implications, as it means windows ignores the usual search paths, so you won't be able to inject kernel32.dll even if you try really hard. It is not, however, a security boundary or defensive measure (it just has useful consequences).
There's also another method that is rarely discussed, WinSxS. Most PE binaries (especially those that implement GUIs) have an application manifest xml file embedded as a resource. dot net developers will be more familiar with this, I suspect. This XML resource can specify alternative versions of DLLs (assemblies) to be loaded from the WinSxS directory. In fact, kernel32.dll is just hardlinked into system32. This is an aside, but I want to mention it for completeness.
So, now we come to how DLLs actually get loaded. There are two ways: statically, from the import directory table. In this case, the load order is the default supplied by Windows and you can't change it. The other case is dynamically, your LoadLibrary* or dlopen. This you can alter the order at runtime.
The pragma dynimport above is creating a static, embedded in the PE IDT reference to winmm.dll; here's the output from dumpbin (windows objdump):
V:\go\src\test\testapp>dumpbin /imports testapp.exe
Microsoft (R) COFF/PE Dumper Version 14.16.27023.1
Copyright (C) Microsoft Corporation.  All rights reserved.

Dump of file testapp.exe

File Type: EXECUTABLE IMAGE

  Section contains the following imports:

    winmm.dll
                548000 Import Address Table
                5DF2F6 Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                           0 timeEndPeriod
                           0 timeBeginPeriod

    ws2_32.dll
                548018 Import Address Table
                5DF30E Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                           0 WSAGetOverlappedResult

    kernel32.dll
                548028 Import Address Table
                5DF31E Import Name Table
                     0 time date stamp
                     0 Index of first forwarder reference

                           0 WriteFile
                           0 WriteConsoleW
                           0 WaitForSingleObject
<snip>

As a consequence, there's nothing the go runtime code can do to alter the search path, because no go runtime code can possibly be called before that DLL is resolved and loaded by Windows. The only alternative is to load the library dynamically.
A security issue arises with DLL loading when an application is executed either with higher privileges, or with different privileges, and the application directory is writable. In this case, where such static DLLs are loaded and they're not KnownDLLs, DLL Injection is trivial and you have a local privesg bug (or lateral movement). As I said above, Windows considers directories to be application bundles and for them to be safe locations to load libraries from (installers in the NSIS case are a bit of a special case, because they're not run from trusted locations). The privileges of such a directory should be set appropriately by code installing the binary in question. I suggest using WIX and generating an MSI; Windows Installer will then do this for you.
There are other ways a developer may shoot themselves in the foot in addition to this DLL. I am assuming any cgo C code that did some kind of pragma import would have that honoured and the DLL added to the IDT as well (I have not looked into this). Same problem occurs. Go plugins also produce loadable DLL components out of go code and developers have plenty of opportunity to do something inadvisable with those.
I don't really have strong opinions as to what to do about this. It really depends on if you want to support writing application installers in golang. However, as things stand this is totally expected behaviour and would apply to any non-KnownDLL that is referenced in the import table.

@diagprov, thanks for the information!

So just quickly, on linux systems, the "API" linux applications can expect to remain consistent and mostly working is directly at the syscall level. The syscall numbers are stable and if your application calls them directly and statically links (like go does), fine. This is not true of Windows, since NT supports multiple subsystems - the syscall interface is an internal API, as is NTDLL.DLL. The only subsystem of this kind (WSL works differently) still used is "win32"; public API for that is kernel32.dll, user32.dll, advapi32.dll, etc etc. You more or less have to link against kernel32.dll and possibly others from KnownDLLs. This is OK, it's how Windows works.
I don't think there's anything wrong with LoadLibrary/handle check/GetProcAddress/null check/use function as a way to load winmm.  You can control the search path this way and allow the static imports by the runtime to be only "standard" DLLs.
One possible question is why Golang is raising timer resolution in the first place. I would guess this is to make the GC as performant as possible, but this has global impact, so, for the life of any golang process the timer resolution goes from the default 15.6ms to 1ms for the whole machine. This would be of particular concern from windows services ("daemons") written in golang. Chrome have discussed it: https://bugs.chromium.org/p/chromium/issues/detail?id=153139 and MSDN also discusses the performance considerations, to quote:

This function affects a global Windows setting. Windows uses the lowest value (that is, highest resolution) requested by any process. Setting a higher resolution can improve the accuracy of time-out intervals in wait functions. However, it can also reduce overall system performance, because the thread scheduler switches tasks more often. High resolutions can also prevent the CPU power management system from entering power-saving modes.

For background on why the Go runtime fiddles with the timer resolution frequency see #8687 and #20937.

So it seems the calls were removed, but it slows down go's scheduling code in some cases, so the removal was reverted. #7876 seems to be an open issue for removing these calls. More complicated than I thought and unfortunately I don't know go nearly well enough to help on that issue.

Wow, I was looking at a very old version of the code. Most of the code I posted is very out of date.
I had a go at making winmm load dynamically and safely using LoadLibraryEx with LOAD_LIBRARY_SEARCH_SYSTEM32. It works!
https://github.com/golang/go/compare/master...sj26:load-winmm-dynamically
This approach could be applied to the other LoadLibraryA calls here, too.

In the meantime, could this library be imported dynamically with the system-only flags right before the timeBeginPeriod call? That would solve this particular issue with hopefully minimal performance impact?

Personally I don't feel there is a problem here.
Alex


fix codeowners

Description
Fixes CODEOWNERS with the right set of members on this GitHub org (conduitio-labs).

@raulb how about we merge this PR despite the linter failing, since the CODEOWNERS file obviously doesn't have anything to do with it?