Create a Publication - RMO property name is incorrect

Under this section:
to-create-a-snapshot-or-transactional-publication
It specifies PublicationType but that fails, and looking at the object it is named just Type

That also makes me think this page is wrong: https://learn.microsoft.com/en-us/dotnet/api/microsoft.sqlserver.replication.publicationtype?view=sqlserver-2016

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 1f4babe3-56f1-a6d9-e725-39abe290a54e
Version Independent ID: a4440d47-1ae5-7295-6f2f-fae7ee741886
Content: Create a Publication - SQL Server
Content Source: docs/relational-databases/replication/publish/create-a-publication.md
Product: sql
Technology: replication
GitHub Login: @MashaMSFT
Microsoft Alias: mathoma

@jpomfret -- Jess, thank you for your feedback.
@MashaMSFT -- Masha, please look into this issue.

@jpomfret which DLL are you looking at? I see PublicationType in my SSMS v19 Microsoft.SqlServer.Replication.dll, as well as the SMO DLL distributed with dbatools v1.1.143.

@jpomfret Well will you look at that... I went into TransPublication class and had a look there: the property is called Type, but it's a PublicationType type.
@MashaMSFT I think we'll have to edit that article. I'll do a PR.

Thanks @rwestMSFT!


commit messages not properly escaped for linux: ping www.google.de -c 2

... is often used for code fragments in markdown
If you use this style inside a commit message on linux, it will be interpreted by the shell. While this is to be expected by git running on the command line, it is a bit surprising for a GUI wrapper like atom-git-control. Entering commit message

ping www.google.de -c 2

results in this commit message
    PING www.google.de (<IP_ADDRESS>) 56(84) bytes of data.
    64 bytes from fra02s18-in-f23.1e100.net (<IP_ADDRESS>): icmp_seq=1 ttl=56 time=24.3 ms
    64 bytes from fra02s18-in-f23.1e100.net (<IP_ADDRESS>): icmp_seq=2 ttl=56 time=24.3 ms

    --- www.google.de ping statistics ---
    2 packets transmitted, 2 received, 0% packet loss, time 999ms
    rtt min/avg/max/mdev = 24.319/24.335/24.351/0.016 ms

The behaviour will probably differ for other OS. I think a GUI-wrapper should try to escape this or not execute git in the usual shell context.

I just hit this issue here: https://github.com/commercialhaskell/stack/pull/2305/commits/675f834b2ea46963206d48cd84ec19efa833040a

I tried to make a commit message saying
Save Stripe `acct_` keys in database

And, git-control, in the console showed: /bin/sh: acct_: command not found.

Just consider
sanitize input like `rm *` <-- don't try this or say goodbye to your repository! ;-)


[Doc Improvement][Add links to relevant docs][2535658]

Added the description with relevant doc links for app cycle.

Acrolinx Scorecards
We currently enforce a minimum score of 80.
Click the scorecard links for each article to review the Acrolinx feedback on grammar, spelling, punctuation, writing style, and terminology:

Article
Score
Issues
Scorecard
Processed

msteams-platform/concepts/design/planning-checklist.md
93
9
link
:white_check_mark:

More info about Acrolinx
Use the Acrolinx extension, or sidebar, in Visual Studio Code to check spelling, grammar, style, tone, clarity, and key terminology when you're creating or updating content. For more information, see Use the Visual Studio Code extension to run Acrolinx locally.

Docs Build status updates of commit 6382328:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/concepts/design/planning-checklist.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Acrolinx Scorecards
We currently enforce a minimum score of 80.
Click the scorecard links for each article to review the Acrolinx feedback on grammar, spelling, punctuation, writing style, and terminology:

Article
Score
Issues
Scorecard
Processed

msteams-platform/concepts/design/planning-checklist.md
95
8
link
:white_check_mark:

More info about Acrolinx
Use the Acrolinx extension, or sidebar, in Visual Studio Code to check spelling, grammar, style, tone, clarity, and key terminology when you're creating or updating content. For more information, see Use the Visual Studio Code extension to run Acrolinx locally.

Docs Build status updates of commit 2275169:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/concepts/design/planning-checklist.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


Indexes

What should be set up in indexes (cloud Firestore) to get the filter running

There are no indexes needed to set up.

Ok


Update subscribe prop len to account for property type byte

Details about the issue opened here: https://github.com/ntex-rs/ntex-mqtt/issues/103

could you add unit test? your change is encoder only, decoder should fail then.
could you also add entry to changes.txt

Fixed a unit test in encode.rs to account for the increased length. So 0x14 should be 0x15.
Added a unit test for decode that includes subscription id, and it seems like decode still works. Which part of decode do you expect should fail?
I was looking at this block of code, but not sure how prop_len is accounted for in decode.

does new test fails with old code?

thanks!


RA issue

on Discord from Parmesan#1113
there are 2 games that i can't seem to get successstory to work with. i can't seem to figure out why that's the case since it works for every other game just fine. the games are:

Pop'n Twinbee: Rainbow Bell Adventures (https://retroachievements.org/game/2236)
Super Mario 64: Splitscreen Multiplayer (https://retroachievements.org/game/14897)
The games' titles match their respective titles in their retroachievements pages but it still doesn't work. logs attached.
it could be a bug but i'd like confirmation if possible. any help would be greatly appreciated

extensions.log

Must be fixed with the next release


Markdown: color of numbers in code blocks is not suitable with light theme at Preview

In Markdown preview with light theme, numbers in code blocks are not clearly distinguishable because of their color. It will be good if a different color can be switched in light theme mode

It seems that in /extensions/markdown-language-features/media/highlight.css , the following part may add a .vscode-light switch. Thank you!
.hljs-number,
.hljs-class {
   color: #B8D7A3;
}

Please share the text of the code along with screenshots

Actually, code itself does't matter this issue. It influences on any kind of code with numbers.
test.txt

Hey, I am a new contributor here.

Hey, I am a new contributor here.

Thanks very much for your contribution.
I have learned .css code after opening this issue, and then opened a PR #82453 after yours, which resets the colors of more than numbers. It may be mine to be merged, but greatly appreciate your help.


Quests API suggestion!

Please please please implement this I beg you, Improve the API please don't make devs write JARS for modules for simple things like a boolean ;-; please please please, make a registerRequirement() method i beg i assure you it isn't too hard I mean is probably harder to load the jars than to load the class anyway that is what you currently do you loop thru the jar and search for the "main" class and load it in so why not just make a method to load the things in directly without having to put the jar in the modules ? now if there is already a method for this I do really really apologize for my ignorance but I haven't found please look into this 👍

Not exactly sure what it is you’re asking. You want Quests to load non-jar files, like scripts? This is already possible through Denizen iirc.
Sent with GitHawk

no, not with script ok so i have Quests as a dependency on my plugin and i hooked into it and i am loading before Quests etc... so i want you to add a method to load requirements into Quests etc.

Due to the way Quests manages it's files, all requirements, custom or otherwise, must be ready at load time (when the server starts). This is partially why Quests uses modules, but it's mostly to give the server owner some control over whether they want a requirement or not. Dropping a module in the folder is a form of cheap authentication.
If I were to allow plugins to "inject" a requirement (or reward or objective), there would be no way for a server owner to refuse it. Granted they wouldn't have to use it, but let's say the requirement doesn't work right or includes a bunch of slow or undesirable background tasks that run regardless of its use (after all, Quests can't assume all devs would code responsibly). In order to stop this, the owner can't just go and remove a module - they would have to remove the offender's entire plugin (or disable some config option somewhere in that plugin, assuming it even has one and it works).
Tl;dr current method allows Quests to maintain some degree of control. Not seeing any advantage to class registration beyond ease of use.

While I can see your reasoning - there are still plenty of cases where a user of Quests would have to do additional configuration to use a binding to other plugins either way.  Even as a server owner that does not do any coding, you still have to realistically deal with configuration of plugins, it's practically unavoidable.
One of the things I do not like about the current approach - I have to create a stand alone intermediary between both Quests and my Plugin, and then I have to provide a way to version that separately from my plugin.  Then I have to include it in my build process somehow.  One of the many reasons we use maven or gradle is to greatly simplify the management of libraries (jars) and their versions, it seems strange to not leverage them.
In order for my plugin to work with your plugin, I have to create an entirely separate jar that isn't a library or second plugin, which does not seem like standard practice.
As a plugin creator I would have to create an API for my plugin that provides all of the functionality needed within an intermediary jar and then connect that to the current api that you've created.
If you would like to see how some other plugin creators have created an API the allows for an extendable or configurable system I can provide you a few examples:

Player Analytics - PluginData API
Annotation Command Framework - System for registering all of your plugin commands; includes completions, contexts, replacements, conditions and Locales

I think Player Analytics provides functionality that is more in tune with what you might do.
He provides many bindings for between other plugins and the Player Analytics system - each provides custom data from those plugins and makes them presentable to Player Analytics.  The config file for Plan also allows you to designate which plugins it can use.
My use case is a little different than most, I'm not creating a standalone, drop in plugin that's going to be widely used, and as far as I can tell, my needs are currently met.  Still, I thought I'd offer you some examples and reasoning as to why you might want to try another approach.
One thing you might do is create a skeleton/blue print project that others can use as a starting point to create their own Custom jars.


[Feature] Add annotations to Backup Job template

Is your feature request related to a problem? Please describe.
I am using mariadb-operator in a ServiceMesh environment (Istio), which adds to each pod a envoy proxy sidecar.
Setting some general annotations/labels the mariadb pods are well running.
In case of the Backup a job is created.
In the job pod also a sidecar is injected, which is not deleted after the backup container finished, which leads to the situation, that the job will never be finished. (https://medium.com/redbox-techblog/handling-istio-sidecars-in-kubernetes-jobs-c392661c4af7)
To overcome this problem you have 2 possibilities:

add a sidecar container to kill the envoy proxy after the job container finishes
add a label to the job template to disable the sidecar injection

  jobTemplate:
    spec:
      template:
        metadata:
          labels:
            sidecar.istio.io/inject: "false"

The option 2 would help in case of mariadb-operator.
Describe the solution you'd like
I suggest to add an option in the Backup CRD (https://github.com/mariadb-operator/mariadb-operator/blob/main/docs/API_REFERENCE.md#backupspec) to define additional annotations for job templates
Describe alternatives you've considered
The alternative was to force the istio platform to disable the sidecar injection for the specific jobs.
But this is not working, as there is no fixed jobname:
sidecarInjectorWebhook:
  # You can use the field called alwaysInjectSelector and neverInjectSelector which will always inject the sidecar or
  # always skip the injection on pods that match that label selector, regardless of the global policy.
  # See https://istio.io/docs/setup/kubernetes/additional-setup/sidecar-injection/#more-control-adding-exceptions
  neverInjectSelector:
    - matchExpressions:
        - {key: job-name, operator: In, values: ["..."]}

Environment details:

Kubernetes version: 1.27.5
Kubernetes distribution: Vanilla
mariadb-operator version: 0,24,0
Install method: helm via ArgoCD
Install flavor: recommended

Additional context

Hey there @andreasgeisslerdt ! Thanks for reporting this.
I agree, the option 2 seems better to integrate with mariadb-operator. You can actually achive this by specifying spec.inheritMetadata in the MariaDB resource:
apiVersion: mariadb.mmontes.io/v1alpha1
kind: MariaDB
metadata:
  name: mariadb
spec:
  # metadata that will be inherited by all children objects
  inheritMetadata:
    labels:
      database.myorg.io: mariadb
    annotations:
      database.myorg.io: mariadb

  volumeClaimTemplate:
    annotations:
      database.myorg.io: mariadb
    resources:
      requests:
        storage: 1Gi
    accessModes:
      - ReadWriteOnce

and then, when you create a Backup referring to it:
apiVersion: mariadb.mmontes.io/v1alpha1
kind: Backup
metadata:
  name: backup
spec:
  mariaDbRef:
    name: mariadb
  storage:
    persistentVolumeClaim:
      resources:
        requests:
          storage: 100Mi
      accessModes:
        - ReadWriteOnce
  args:
    - --single-transaction
    - --all-databases

The following Pod will eventually be created:
apiVersion: v1
kind: Pod
metadata:
  annotations:
    database.myorg.io: mariadb
  creationTimestamp: "2024-01-25T16:52:32Z"
  finalizers:
  - batch.kubernetes.io/job-tracking
  generateName: backup-
  labels:
    batch.kubernetes.io/controller-uid: 28688b01-21bd-4cb7-8ce1-dd1fe8599259
    batch.kubernetes.io/job-name: backup
    controller-uid: 28688b01-21bd-4cb7-8ce1-dd1fe8599259
    database.myorg.io: mariadb
    job-name: backup
  name: backup-2sc28
...

Which I believe it should cover this case, could you confirm please? 🙏🏻

Hi Martin,
the suggestion does not cover it completely.
When I set the label to spec.inheritMetadata, it will affect all resources, which means, that the sidecars are disabled for all pods and Istio is disabled for the whole component.
This is actually what I am currently doing, as otherwise the Metrics export is not working.
A better way I actually saw in other operators (get for Postgres) where I struggled with the same problem.
https://access.crunchydata.com/documentation/postgres-operator/latest/references/crd/5.5.x/postgrescluster
You can see, that in each level (PostgresCluster, PostgresCluster.spec,  PostgresCluster.spec.backups,...) you have an "metadata" entry to add annotations and labels to the resources, which works really fine.
I just added for the PostgresCluster.spec.backups my lable to disable the sidecar injection and the created jobs were deployed without sidecars.
I know, that's maybe a big change, but could be an improvement idea for the mariadb-operator.
Thanks a lot for your support...
Actually I would like to help you in implementation, but I am currently too busy with my job
Best regards
Andreas

Hey there @andreasgeisslerdt !
Understood, I see your point, I've created the following issue for this:

https://github.com/mariadb-operator/mariadb-operator/issues/353

Actually I would like to help you in implementation, but I am currently too busy with my job

Contributions are always welcome! I will leave some time till the end of the release to see if someone decides to contribute and otherwise will try to squeeze it in my tasks for this release.

Hey there @addreas !
This has been addressed. You can now define dedicated metadata for Backup, Restore and SqlJob objects via their new inheritMetadata field. The metadata is propagated to the Pod template as well. Please take a look at the updated examples:

https://github.com/mariadb-operator/mariadb-operator/blob/main/examples/manifests/mariadb_v1alpha1_backup.yaml

Closing! This will be released in v0.0.26. Stay tuned!


Set version 4.10.0

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 87.841%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
5960

Relevant Lines:
6785

💛  - Coveralls


Add devices Galaxy Tab/Alcatel

Galaxy Tab/Alcatel

galaxy tab
Mozilla/5.0 (Linux; Android 6.0.1; Galaxy Tab 2)...

900[0-9][A-Z]
Mozilla/5.0 (Linux; Android 8.1.0; 9009G)...

Does not seem to work
https://jsfiddle.net/gzj29b3v/8/

Weird both have been tested with the actual device and they did worked. :confused:

@Ark74 could you provide complete User-Agent values?

Here:
Mozilla/5.0 (Linux; Android 6.0.1; Galaxy Tab 2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.101 Safari/537.36

Mozilla/5.0 (Linux; Android 8.1.0; 9009G) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.101 Safari/537.36

Ohh! I see my error. I moved them on the PR to fit incremental order. Later on I guess I was having a cached browser response.
Let me correct.

@em230418  Removed leading pipe causing that issue.

Anything else missing?

Anything else missing?

No, just dealing with other todos. Sorry for the delay and thanks!

I approve to merge it now


movewindow left won't work when default_split_ratio setting is active.

Hyprland Version
Hyprland, built from branch HEAD at commit 03ebbe18ed8517ee22591eac82cd54322f42cb7d  (props: bump ver to 0.34.0).
Date: Mon Jan 1 12:03:15 2024
Tag: v0.34.0
flags: (if any)
Bug or Regression?
Bug
Description
The configured binding for moving window left does not work when combined with setting: default_split_ratio = 1.1, my keybinding is binde = SUPER, Left, movewindow, l.  The whole related configs are:
dwindle {
    default_split_ratio = 1.1 # 1 means 5/5 split
}

binde = $mainMod, Left, movewindow, l

 System/Version info
Linux 6.6.9-arch1-1 #1 SMP PREEMPT_DYNAMIC Tue, 02 Jan 2024 02:28:28 +0000 x86_64 GNU/Linux

How to reproduce
Set the following three config at once:
binde = SUPER, Left, movewindow, l
general {
    layout = dwindle
}
dwindle {
    default_split_ratio = 1.1 # 1 means 5/5 split
}

Then see that super+left won't move the window left.
Crash reports, logs, images, videos
No response

I also have this issue when I use a number less than 1, that making down and right not work

I also have this issue with any number other than 1


Hosted service LanguagesInCrimeaTiles/MapServer does not have a service definition item

The hosted service "Languages in Crimea (Tile Service)" item exists 93f8ddef0a5a462e8730940234b13b9c but does not have an associated service definition portal item. Without the service definition item the service won't be published to any user conference machines.
NOTE: Only hosted services published from ArcGIS Desktop or ArcGIS Pro, which upload a service definition item to portal, will be published to the UC servers.
Owned by Craig C. but looks like he doesn't have GitHub account or has account but no access to Esri org.

@CraigCleveland

@elinz @jrweakland Ditto to previous comment.  Will re-work the web maps as a workaround for this release.

@CraigCleveland FYI: Re-ran report and the hosted service item still exists. Without service definition portal item, the service won't be published during stand-up of Ops Server. Options are to republish the hosted service so service definition item is uploaded to portal or delete this hosted service. Thanks.

@elinz see my comment on previous thread for full details.  Short version is I plan to utilize only feature services...

@CraigCleveland Please delete the hosted LanguagesInCrimeaTiles/MapServer map service. Thanks.

@elinz - deleted.

@CraigCleveland hosted service LanguagesInCrimeaTiles.MapServer service and portal item 93f8ddef0a5a462e8730940234b13b9c still exist. Let me know and I can delete this for you.

@elinz they should be gone now.  The previous delete failed for some reason...


feat(vector): add basic functions (vector/vector_extract etc)

Context
This PR adds native support for vector functions in the LibSQL.
This PR were extracted from the branch https://github.com/tursodatabase/libsql/pull/1402/ in order to make review more granular and simpler

Vectors are represented as binary blobs with vector components encoded in little-endian format according to IEEE-754 standard

LibSQL now supports f32/f64 vectors of any dimension not more than 65536 (see MAX_VECTOR_SZ)
Zero-length vectors are supported - and they are always represented as zero size BLOB (not NULL)

LibSQL can extended vector encoding on disk with 1 trailing byte which will encode vector type information (for now: 1 = F32, 2 = F64).

[F3 76 86 C4] [BC 9D F1 3F] [01] - this is f32 vector because type equals to 0x01 = 1 (value = [-1075.72,1.88763])
[F3 76 86 C4] [BC 9D F1 3F]      - this is f32 vector because default type is 1 (value = [-1075.72,1.88763])
[F3 76 86 C4   BC 9D F1 3F] [02] - this is f64 vector because type equals to 0x02 = 2 (value = [1.10101])

Changes

5 new builtin functions are added:

vector/vector32/vector64 - convert TEXT or BLOB to binary vector BLOB. If text is provided - then vector/vector32 will produce F32 vector, but vector64 - F64 vector
vector_extract - convert binary vector BLOB to the human readable TEXT string (like [1,2,3])
vector_distance_cos - calculates cosine distance (not similarity) between vectors of same dimension and same type

SQLITE_OMIT_VECTOR preprocessor directive and autoconf parameter --disable-vector which will remove any signs of vector functions from the final build
libsql_vector.test - TCL based test suite

CI is failing:
found uncommited changes in the bundle; please run 'cargo xtask build-bundled' and commit the result


HDFS-15957. The ignored IOException in the RPC response sent by FSEditLogAsync can cause the HDFS client to hang

2878

 Quality gate: Failed
1 Issues

 1 Vulnerabilities

Details


Column Limit ambiguity

"Dataset size limit - there is a 1 GB limit for each dataset in the Power BI service.
Row limit - the maximum number of rows in your dataset (when not using DirectQuery) is 2 billion, with three of those rows reserved (resulting in a usable maximum of 1,999,999,997 rows); the maximum number of rows when using DirectQuery is 1 million rows.
Column limit - the maximum number of columns allowed in a dataset, across all tables in the dataset, is 16,000 columns. This applies to the Power BI service and to datasets used in Power BI Desktop. Power BI uses an internal row number column per table included in the dataset, which means the maximum number of columns is 16,000 minus one for each table used in the dataset."
The first line reads "maximum number of columns allowed in a data, across all tables in the dataset, is 16,000 columns" whereas the last line reads "the maximum number of columns is 16,000 minus one for each table used in the dataset."
So the total limit, of 16,000 columns, applies to dataset or table?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: bb95c243-439b-5f81-51ae-c28efa0d22c3
Version Independent ID: 7e0c5adf-c752-4ebf-5022-e3bd64f3f8d3
Content: Data sources for Power BI - Power BI
Content Source: powerbi-docs/service-get-data.md
Service: powerbi
GitHub Login: @davidiseminger
Microsoft Alias: davidi

Hi @dkikan and thanks for your comment. I have assigned the issues to the author. He is on vacation until after the holidays, FYI. Regards - Kelly

I think the content is correct and clear; if you have multiple tables, you lose one per table in the dataset. So if you had three tables, the max would be 16,000 minus 3.


[Proposal] SOAR-0004: Streaming request and response bodies

Motivation
See the proposal at https://github.com/czechboy0/swift-openapi-generator/blob/hd-soar-0004/Sources/swift-openapi-generator/Documentation.docc/Proposals/SOAR-0004.md.
Modifications
N/A
Result
N/A
Test Plan
N/A

This proposal is now In Review, which will run until 15th September - feel free to leave your feedback either on this PR or on the corresponding Swift Forums post.
Thanks, and thanks @czechboy0 for the write-up!

Overall really like the shape of things here, great job Honza 🥳 I've only got two pretty minor comments:

As a general rule, we tend to favour extending Bar with init(foo: Foo) when Foo can be converted into a Bar, instead of having Foo.asBar() methods. I wonder if we should make the following changes - have you thought about this?

HTTPBody.collectAsString(upTo maxBytes: Int) async throws -> String -> String.init(collecting httpBody: HTTPBody, upTo maxBytes: Int) async throws
HTTPBody.collectAsData(upTo maxBytes: Int) async throws -> Data -> Data.init(collecting httpBody: HTTPBody, upTo maxBytes: Int) async throws
StringProtocol's extension to add asBodyChunk -> HTTPBody.ByteChunk.init(_ string: String)

This one really is about the proposal itself and quite minor, but if collect(upTo: .max) is discouraged (as stated in the text), should we use it in examples in the proposal? Maybe we should steer clear of what we deem bad practices in sample code.

Overall really like the shape of things here, great job Honza 🥳 I've only got two pretty minor comments:

As a general rule, we tend to favour extending Bar with init(foo: Foo) when Foo can be converted into a Bar, instead of having Foo.asBar() methods. I wonder if we should make the following changes - have you thought about this?

HTTPBody.collectAsString(upTo maxBytes: Int) async throws -> String -> String.init(collecting httpBody: HTTPBody, upTo maxBytes: Int) async throws
HTTPBody.collectAsData(upTo maxBytes: Int) async throws -> Data -> Data.init(collecting httpBody: HTTPBody, upTo maxBytes: Int) async throws

I quite like that. I guess it's less common for async initializers, but that shouldn't stop us. While it's a little less discoverable, I think the consistency with general Swift guidelines is more important here. I'll make that change.

StringProtocol's extension to add asBodyChunk -> HTTPBody.ByteChunk.init(_ string: String)

This is an internal, not public, method, it just showed up there as it's inlinable. I'll see later about changing it, but probably not before the review ends.

This one really is about the proposal itself and quite minor, but if collect(upTo: .max) is discouraged (as stated in the text), should we use it in examples in the proposal? Maybe we should steer clear of what we deem bad practices in sample code.

Ha - keeping me honest. Yes, I'll update it 🙂
Thanks, @gjcairo! 🙏

@gjcairo I applied all your feedback, please take a look at v1.3: https://github.com/czechboy0/swift-openapi-generator/blob/hd-soar-0004/Sources/swift-openapi-generator/Documentation.docc/Proposals/SOAR-0004.md

The review period is now over, and the proposal has been accepted. SOAR-0004 is now Ready for Implementation.

Landing the proposals to avoid more conflicts, will update their status once the implementation lands.


mailto links support

For now only https links properly translated to ....
https://github.com/shtanton/gmi2html/blob/2ee16d29c2ffd4057f2e89efb0d323ef79c70010/src/main.zig#L20
But it would be nice to handle mailto links in the same way.

Yep, I'll add that when I get time or accept a pull request if someone beats me to it

Thanks!


Mul

I met this problem,
escriptor 'mul' requires a 'torch._C._TensorBase' object but received a 'builtin_function_or_method.
Is  the operation of torch.mul not supported here?

hi  can you tell me how do you solve it?


feat: The double factorial is positive

and other basic results. Also include a positivity extension to encode that new result.
From LeanAPAP

Thanks! :tada:
bors merge


Ask Users To Use Relative Paths On Add File To Playlist

Closes #338
TODO:

[ ] https://github.com/Zeugma440/atldotnet/issues/232
[ ] https://github.com/Zeugma440/atldotnet/issues/233

Closing in favor for #349


JDK-19 adds String::valueOf calls to string concatenation

With https://github.com/openjdk/jdk/commit/cfee4512f7048e9cf440078e9eb87d37c7ebcdd9 extra String::valueOf calls are inserted for objects passed to makeConcatWithConstants. Quiltflower should remove these for better readability.

Hi, I can probably write this up real quick. Let me know where I should look!

Hey, thanks! I'd recommend that you look here, and just before the return insert a loop that goes over the parameters array and call, and runs the existing removeStringValueOf method over each exprent, to remove the valueOf call.

Understood! Shouldn't be too difficult a fix, definitely will get it done within a few hours once I get a bit of free time. Question: how should we run tests before committing? Is there code style that I'll need to follow/recommended linting? Should I squash commits (probably unnecessary, my guess is I'll get it in one). Let me know, thanks!

Just added a loop in extractParameters, hopefully what you wanted (since the only other place those parameters come up via a call to extractParameters). Let me know if that's what you want! Also, out of curiosity how does this integrate into the overall code? It's a little hard for a newcomer to understand 😅

Fixed by #153


Set up an automatic dependency update tool

Set up an automatic way for monitoring/updating dependencies of our code-base (like Dependabot).

Dependabot was enable a while ago.


Inconsistent and incorrect behavior

require 'vendor/autoload.php';

$file = Amp\File\openFile('test.txt', 'a');
$file->write("test\n");
unset($file);

Revolt\EventLoop::run();

When using default driver the execution hangs.
When using eio driver the a mode behaves like w.
When using uv driver it works but the file is not automatically closed in underlying destructors. I discovered this by encountering an anomalous chart of open file descriptors (similar code in a real module). Is it meant to be closed manually?

Thanks for reporting! I've committed fixes for 2 and 3. The fix for 1 is in https://github.com/amphp/byte-stream/pull/100.

All of those have been fixed now in the beta versions, I still need to take a look at the current stable version.


email verification not working

When trying to verify my codewars account Ardin17 email<EMAIL_ADDRESS>i get the email but it refuses to verify and gives me a 403 error after being redirected to the codewars site.
┆Issue is synchronized with this Clickup  by Unito

Some users have reported the same issue using some browsers on Android. Can you provide your browser information (the one that opened the confirmation link)?

yes sorry, i missed the reply, the browser it was sent from was chrome, the browser i tried to verify it from was firefox.
Sent with ProtonMail Secure Email.
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Monday, October 12, 2020 8:25 PM, kazk<EMAIL_ADDRESS>wrote:

Some users have reported the same issue using some browsers on Android. Can you provide your browser information (the one that opened the confirmation link)?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

I have the same problem for confirm email , it just show me 403 error said that I have no permission.

Make sure to open the link with the browser you're signed in with. The confirmation link doesn't log you in because that would allow someone to take over your account.
It should be redirecting to the sign-in page instead of showing 403 error, so I'll look into that.


refactor: refact @/cloud package, rename & publish to npm registry

@/cloud is a internal package for cloud function
@/cloud is a invalid package name which npm registry not allow
consider: deno compatibility, tencent-cloudbase compatibility

suggest package names:

@lafos/cloud for @/cloud
@lafos/client-sdk for laf-client-sdk

Has been completed and named @lafjs/cloud.


? Update to latest EtherCis build

Hi Allesandro,
Christian has just updated Ethercis on Github to the latest code build which has a number of fixes. Are you in a position to update the Docker build?
Ian

I will update the image in the upcoming two/three weeks.

Many thanks.
Ian


Display message if unknown extension when dropping file

@lpatiny
I show this message if the dropped file extension not one of ( .dx,.jdx,.json,.mol )
"The file must be jcamp file .dx,.jdx,.json file extension

.dx was not allowed but I added in the commit a6727aee00d68134d97ce4dfb6b41ee9e0ad1712
I think also the way the code is done currently it can never throw an error for unknown extensions because you only call loadFiles for known extensions !
Please change the code so that the rejection is done in the case default behaviour that seems more logical to me: https://github.com/cheminfo/nmr-displayer/blob/a6727aee00d68134d97ce4dfb6b41ee9e0ad1712/src/component/NMRDisplayer.jsx#L199


Record response time at response end instead of response start

Should record response time at response end instead of response start so that we can record whole response time for streaming response.

Also the docs need to be updated (though since this should be a new option,  add new docs). Here is the current docs:

:response-time[digits]
The time between the request coming into morgan and when the response headers are written, in milliseconds.

Hi @bobwei sorry I never came back to this PR. I just realized that instead of landing this as a flag that changes the behavior of an existing token, we should modify the PR to just add a new token with the behavior, which same the same ultimate outcome, but the benefit of being able to include both of these times in the output if that's desired.

I have just written all the code to move this to a :total-time token. If you are agreeable on this approach, I can squash this into your PR here and merge as your PR still.

Hi @dougwilson, sure, please proceed.

Can this be merged please?


Upper case "CREATE" in column name ends table parse

Describe the bug
It looks like you are seeing CREATE and breaking the table DDL parse...
To Reproduce
t1.sql:
DROP TABLE IF EXISTS demo;
--
CREATE TABLE demo
(
     foo                             char(1),
     create_date                     DATETIME2,
     created_BY                      VARCHAR (20)
)
--

t2.sql:
DROP TABLE IF EXISTS demo;
--
CREATE TABLE demo
(
     foo                             char(1),
     CREATE_date                     DATETIME2,
     created_by                      VARCHAR (20),
)
--

sdp -v t1.sql
$ sdp -v t1.sql
Start parsing file t1.sql

File with result was saved to >> schemas folder
[{'alter': {},
  'checks': [],
  'columns': [{'check': None,
               'default': None,
               'name': 'foo',
               'nullable': True,
               'references': None,
               'size': 1,
               'type': 'char',
               'unique': False},
              {'check': None,
               'default': None,
               'name': 'create_date',
               'nullable': True,
               'references': None,
               'size': None,
               'type': 'DATETIME2',
               'unique': False},
              {'check': None,
               'default': None,
               'name': 'created_BY',
               'nullable': True,
               'references': None,
               'size': 20,
               'type': 'VARCHAR',
               'unique': False}],
  'index': [],
  'partitioned_by': [],
  'primary_key': [],
  'schema': None,
  'table_name': 'demo',
  'tablespace': None}]

vs:
$ sdp -v t2.sql
Start parsing file t2.sql

File with result was saved to >> schemas folder
[{'alter': {},
  'checks': [],
  'columns': [{'check': None,
               'default': None,
               'name': 'foo',
               'nullable': True,
               'references': None,
               'size': 1,
               'type': 'char',
               'unique': False}],
  'index': [],
  'partitioned_by': [],
  'primary_key': [],
  'schema': None,
  'table_name': 'demo',
  'tablespace': None}]

Expected behavior
A capitalized substring field name that is CREATE shouldn't break the parser.

Looks like your tokenizer is misidentifying keywords even when they are no separated by white space.

@n2ygk no, ptoblem not in tokenizer, lol, problem in changes that I did in 0.22.0 version to get possible parse statements without ; or GO. I will fix it tomorrow and release the patch version. thanks for catching this

Haha so it was my fault :-)
On Tue, Nov 16, 2021 at 3:21 PM Iuliia Volkova @.***>
wrote:

@n2ygk https://github.com/n2ygk no, ptoblem not in tokenizer, lol,
problem in changes that I did in 0.22.0 version to get possible parse
statements without ; or GO. I will fix it tomorrow and release the patch
version. thanks for catching this
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/xnuinside/simple-ddl-parser/issues/94#issuecomment-970647525,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABBHS52OZX4ILERIL7ABHFLUMK4M5ANCNFSM5IFFIBZQ
.

@n2ygk fixed  in 0.22.2, test added: https://github.com/xnuinside/simple-ddl-parser/pull/95

@n2ygk Alan, I will close the issue - check my comment in PR) it was already released at the morning


Remove Trailing (and Leading) underscores from clean_names

Pandas handles this relatively well but would be a good as a kwarg for jn.clean_names().
The default False or None could leave leading, True or 'both' to remove all and trailing and then pass 'leading' and 'trailing' or similar to remove each.

@JoshuaC3 thanks for raising the issue! So just so I understand correctly, column_name_ and _column_name would become column_name, is that right? If so, please send in a PR!

Yes. Exactly. And _column_name_ would be column_name. Will do PR.
Two quick questions on this;
For removing a single underscore would you rather:

'leading' and `'trailing,
'lead' and 'trail',
'left' and 'right',
'start' and 'end', *
other of your choice?

For removing both:

True, *
'both' or
other of your choice?

my preferences, but please advise :)

I'd suggest using a single keyword argument remove_trailing_underscores, which accepts either one of: ['start', 'end', 'both']. By not having mixed bool and str arguments, it makes for an easier learning pattern for end-users.
Other than that, looking forward to the PR!


Updating the v3 branch

This one is completely unnecessary but i thought it would be good to update the v3 branch to the last production build from mai

PS this updates the version to 3.0.3

PS this updates the version to 3.0.3


Exploring Interesting Places for Valuable Information

PR Description:
We've explored various digital platforms and tools to uncover information that could be critical from a cybersecurity perspective.
Screenshots Included:

M11109816xM11115825-the-harvester.png: This shows our use of The Harvester against a government website, where we discovered 204 hosts. Virustotal was used as the source (using a free API key).

M11109816xM11115825-psdbmp.png: We came across a dump on psbdmp.ws that alarmingly contains SSH credentials.

M11109816xM11115825-pastebin.png: A surprising find on Pastebin revealed secrets related to Tesla.

M11109816xM11115825-doxbin.png: A worrying discovery on Doxbin, revealing someone's password.

M11109816xM11115825-searchcode.png: We used "githubSecretKey" as a keyword on Searchcode and found several exposed keys.

M11109816xM11115825-bitcoin-talks-forum.png: Somebody shared a script to check if your BTC-Key is vulnerable by checking public keys for reused R-signing values which allows generating of the private key of affected PKs. https://bitcointalk.org/index.php?topic=977070.0

M11109816xM11115825-kestrel-huntbook-trial.png: This screenshot is evidence of our hands-on trial with one of the Kestrel huntbook tutorials.

cc: @VStev
Checks

[x] Your YAML file has required fields as listed in Specs
[x] All checks have passed yaml-lint passed
[x] You agree to contribute your work under MIT license

@raymundlin just to add, we found an interesting file during our research. However, we consider it very sensitive data. Therefore, we request to have your PGP public key to encrypt the file before sending it to you, if possible. Thank you.
cc: @VStev

@raymundlin just to add, we found another interesting file during our research. However, we consider it very sensitive data. Therefore, we request to have your PGP public key to encrypt the file before sending it to you, if possible. Thank you.
cc: @VStev

http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x43A33B5BC13D1547


no package config files generated for grpc cmake build on linux

Please answer these questions before submitting your issue.
What version of gRPC and what language are you using?
1.6.x (1.6.6 now)
What operating system (Linux, Windows, …) and version?
x86_64/linux  (Fedora 25, Centos 7, Ubuntu 14 & 16)
What runtime / compiler are you using (e.g. python version or version of gcc)
gcc 7.2.x
What did you do?
If possible, provide a recipe for reproducing the error. Try being specific and include code snippets if helpful.
native (gnu) make build of grpc produces package config files: /lib/pkgconfig/grpc*.pc files;
cmake build of grpc does NO produce them ... (only cmake config files in /lib/pkgconfig/cmake/).
What did you expect to see?
a set of grpc*.pc files in /lib/pkgconfig  directory
What did you see instead?
no such files
Make sure you include information that can help us debug (full error message, exception listing, stack trace, logs).
Anything else we should know about your project / environment?

duplicate of https://github.com/grpc/grpc/issues/10727.


Tailwind css 3.0 should be added to Docusaurus

We recently added tailwind to the components library. One problem though is that it's not being read by the other dependent libraries, specifically, docusaurus.
See:
https://deploy-preview-622--squiggle-documentation.netlify.app/playground
Even better could be a system that loads all the css we use in the components library, and brings that to places that use the styling.
This should include importing the form library, @tailwindcss/forms
This plugin would also be useful:
https://tailwindcss.com/docs/typography-plugin
@tailwindcss/typography
Ideally, this plugin would be used in the documentation. Right now, the headings were removed from the documentation (in that they're not large/styled, which is suboptimal)
Some relevant threads:
https://github.com/facebook/docusaurus/issues/2961

I tried adding it here, but that was Tailwind 2.0, which is wrong for us.
https://github.com/quantified-uncertainty/squiggle/pull/625
This highlights the issue that this is an awkward way to go about this though.

I tried this but didn't get it to work: https://dev.to/sajclarke_62/using-tailwindcss-v3-in-docusaurus-in-5-steps-5c26
There are also some examples around here: https://codesandbox.io/s/rr0qy?file=/plugins/docusaurus-tailwindcss-loader
I'm really not great with webpack stuff.

Sorry, I did notice this and started trying to fix it. Docusaurus is a bit of a pain because it hides a lot of its details and doesn't have a functioning logger. But hopefully I'll get this one patched soon

Actually, I realized that we don't want to import Tailwind.css to Docusaurus. Instead we want roughly this:
https://github.com/quantified-uncertainty/squiggle/issues/636

Should this be closed too?

Yea, thanks for flagging.


Support external link annotation

Summary
Like https://github.com/argoproj/argo-cd/pull/4380, we may want to put external links to workflow templates and workflow execution results from UI as well.
Use Cases

Check the specifications for workflow.
Check logs and performance of workflow execution results

Message from the maintainers:
Impacted by this bug? Give it a 👍. We prioritise the issues with the most 👍.

A feature to add links exists: https://argoproj.github.io/argo/links/
If we had a standard annotation we could modify the code to allow you to link that too. Would you like to submit a PR?

A feature to add links exists: https://argoproj.github.io/argo/links/

This is to provide a link to the argo server, and I am aware that it is not possible to provide a link for individual workflows.

If we had a standard annotation we could modify the code to allow you to link that too. Would you like to submit a PR?

I don't know what the policy is for modifications, so please let me know.
Are you assuming that you don't add any processing on the server side, but modify the front end (probably around here)?

links:
      - name: Example Workflow Link
        scope: workflow
        url: http://logging-facility?namespace=${metadata.namespace}&workflowName=${metadata.name}&startedAt=${status.startedAt}&finishedAt=${status.finishedAt}
      - name: Example Pod Link
        scope: pod
        url: http://logging-facility?namespace=${metadata.namespace}&podName=${metadata.name}&startedAt=${status.startedAt}&finishedAt=${status.finishedAt}

I thinking new formatting things could be added:
links:
      - name: Example Workflow Link
        scope: workflow
        URL: "${metadata.annotations[''workflows.argoproj.io/external-link"]}'

You'd need to modify this func to support it:
https://github.com/argoproj/argo/blob/master/ui/src/app/workflows/components/workflow-details/workflow-details.tsx#L350


Cleanup - Change /ploigos/generated-psr to /ploigos/psr-secrets.yml

Success Criteria:

ploigos-github-workflows reads its shared configuration from a file named /psr-secrets.yml instead of /ploigos/generated-psr
The infra-ops repo has been updated to mount a secret at the new location

Possibly will be done as part of https://github.com/ploigos/infra-ops/issues/34. Adding to sprint in case.

Was done as part of #34.


What is the best way to check if the server is in offline mode?

I already thought of connecting a cracked client to a server and see if it works, but this method seems a bit hacky. Is there any other way?

If the server returns the player list, it will include the UUID for each player too. Then you generate the offline UUID using the player name and compare the two uuids.

Ah, that's a cool idea but unfortunately some servers don't return the player list, so this method is not reliable. Anyways, thanks for your reply :)

why do you have to check?

Because if the server is in offline mode, my program will generate some random minecraft usernames and connect to the server. But if this is not the case, it will authenticate with valid minecraft account credentials and then connect to the server.


Not really a issue, but I need clarification

Hey @codyseibert ,
I had a question, for the main page, LandingPage.tsx is nothing linked except the dashboard, also to link pages do i need to make a separate folder/directory?
Please confirm.
Thanks,
Kendall Does Coding

@KendallDoesCoding for any new page, you need to add it in the client/src/pages directory.  For example, if you want an about page, you need to make and AboutPage.tsx and put it in the client/src/pages directory.  If you want that page to show up, you need to also add it to the  component.

@KendallDoesCoding for any new page, you need to add it in the client/src/pages directory. For example, if you want an about page, you need to make and AboutPage.tsx and put it in the client/src/pages directory. If you want that page to show up, you need to also add it to the component.

Thanks!


AttributeError: 'FrameShiftTruncation' object has no attribute 'aa_alt'

Issue with multiple inheritance of FrameShiftTruncation, where mutant_protein_sequence uses the definition in PrematureStop?
/Users/tavi/drive/work/repos/cancer/topiary/topiary/mutant_epitope_predictor.pyc in predict(self, variant_collection, mhc_alleles, gene_expression_dict, min_gene_expresion_value, transcript_expression_dict, min_transcript_expresion_value, raise_on_variant_effect_error)
    145         fasta_dict = create_fasta_dict(
    146             effects=variant_effects.values(),
--> 147             padding_around_mutation=self.padding_around_mutation)
    148         return mhc_model.predict(fasta_dict)

/Users/tavi/drive/work/repos/cancer/topiary/topiary/mutant_epitope_predictor.pyc in create_fasta_dict(effects, padding_around_mutation)
     20         # TODO: will mhctools take an object key?
     21         key = effect.variant
---> 22         seq = effect.mutant_protein_sequence
     23         # some effects will lack a mutant protein sequence since
     24         # they are either silent or unpredictable

/Users/tavi/.virtualenvs/nejm/lib/python2.7/site-packages/memoized_property.pyc in fget_memoized(self)
     38     def fget_memoized(self):
     39         if not hasattr(self, attr_name):
---> 40             setattr(self, attr_name, fget(self))
     41         return getattr(self, attr_name)
     42

/Users/tavi/drive/work/repos/cancer/varcode/varcode/effects.py in mutant_protein_sequence(self)
    546     def mutant_protein_sequence(self):
    547         prefix = self.original_protein_sequence[:self.aa_mutation_start_offset]
--> 548         return prefix + self.aa_alt
    549
    550

AttributeError: 'FrameShiftTruncation' object has no attribute 'aa_alt'

Can you paste the variant this error came from? I'll add it to my
"problematic variants" unit test.
On Tue, Apr 28, 2015 at 4:53 PM, Tavi Nathanson<EMAIL_ADDRESS>wrote:

Issue with multiple inheritance of FrameShiftTruncation, where
mutant_protein_sequence uses the definition in PrematureStop?
/Users/tavi/drive/work/repos/cancer/topiary/topiary/mutant_epitope_predictor.pyc in predict(self, variant_collection, mhc_alleles, gene_expression_dict, min_gene_expresion_value, transcript_expression_dict, min_transcript_expresion_value, raise_on_variant_effect_error)
145         fasta_dict = create_fasta_dict(
146             effects=variant_effects.values(),
--> 147             padding_around_mutation=self.padding_around_mutation)
148         return mhc_model.predict(fasta_dict)
/Users/tavi/drive/work/repos/cancer/topiary/topiary/mutant_epitope_predictor.pyc in create_fasta_dict(effects, padding_around_mutation)
20         # TODO: will mhctools take an object key?
21         key = effect.variant
---> 22         seq = effect.mutant_protein_sequence
23         # some effects will lack a mutant protein sequence since
24         # they are either silent or unpredictable
/Users/tavi/.virtualenvs/nejm/lib/python2.7/site-packages/memoized_property.pyc in fget_memoized(self)
38     def fget_memoized(self):
39         if not hasattr(self, attr_name):
---> 40             setattr(self, attr_name, fget(self))
41         return getattr(self, attr_name)
42
/Users/tavi/drive/work/repos/cancer/varcode/varcode/effects.py in mutant_protein_sequence(self)
546     def mutant_protein_sequence(self):
547         prefix = self.original_protein_sequence[:self.aa_mutation_start_offset]
--> 548         return prefix + self.aa_alt
549
550
AttributeError: 'FrameShiftTruncation' object has no attribute 'aa_alt'
—
Reply to this email directly or view it on GitHub
https://github.com/hammerlab/varcode/issues/70.

Should be fixed in version 0.3, but if you give me the variant I'll still add it to the unit tests.


Implement compressed transfers

It seems when flashing, loading data is the most time consuming process (just may experience, may not always be the case). Reducing the number of bytes we have to transfer should improve flashing speeds.
Compressing pages of data, copying to target ram, then decompressing on the target ready for writing to flash should net improved flashing performance.
A quick look on crates.io yielded this crate: https://github.com/alexkazik/lzss which may be of use.

On many ARM chips the bottleneck is the actual flash write speed (copying from RAM to flash), not the copy from host to target RAM, given a decent probe (ie not jlink) and decent SWD frequency.
How common is for jtag/swd to be the bottleneck? Maybe there is some low-hanging-fruit optimizing the jtag memory transfers?

I should note that I'm not running into this bottleneck right now, that's for sure!
However, in the case of some Espressif chips (and other chips I image), the spi flash fitted is capable off tens of Megabytes write speed, when in quad io mode at full speed. I'm getting no where near that speed (10kb/s write) with the ram_download example but I am running a jlink so it could be that.
Side note, whats the jtag support like on the hs-probe - I can't lie I am quite tempted by the lil guy :D

Side note, whats the jtag support like on the hs-probe - I can't lie I am quite tempted by the lil guy :D

I can answer this at least! It supports the JTAG_Sequence CMSIS-DAP command, so it can do arbitrary JTAG sequences using something like openocd, at reasonably high throughputs: my ecpdap tool writes to an FPGA over JTAG at 820KB/s, and reads attached SPI flash memory at 750KB/s, for example. The firmware doesn't yet support the JTAG commands used to debug ARM JTDP debug ports, though that's not relevant to the riscv use cases.

spi flash fitted is capable off tens of Megabytes write speed

Wow, tens of MB/s write? What SPI flash is that?

Wow, tens of MB/s write? What SPI flash is that?

My bad, on further inspection although the chips I was looking at support quad writes, thats only at low clock rates only.

I can answer this at least! It supports the JTAG_Sequence CMSIS-DAP command, so it can do arbitrary JTAG sequences using something like openocd, at reasonably high throughputs: my ecpdap tool writes to an FPGA over JTAG at 820KB/s, and reads attached SPI flash memory at 750KB/s, for example. The firmware doesn't yet support the JTAG commands used to debug ARM JTDP debug ports, though that's not relevant to the riscv use cases.

Thanks for the info adam! Unfortunately is seems like JTAG functionality isn't actually supported in probe-rs just yet, I'll keep an eye out :)
Speaking of JTAG support, at some point I'd like there to be support for the usb jtag on the c3(and other upcoming chips); given that this usb interface is Usb Full Speed (1.5MB/s) compression could help here.

1.5MB/s is well enough for any flash algo. The fastest flashes I have seen are in the 100kb/s speed wise. so 15x the overhead is well enough :) But if your flash is really that fast, we can consider it :)

Can the flash algos typically handle flashing and receiving data over JTAG/SWD at the same time?  I've only looked at them for some weird old chips at work, but those don't do anything that fancy.  So, those steps are done in serial, so I think that even if the flashing itself is the slower part of the process there may still be an advantage in compressing over JTAG/SWD.


Issue #286 default qubit state prep. consistency with other plugins

Context: Comparing the various plugin behaviours, there are inconsistencies in whether a state vector prep. operation can be applied only as the first operation in a circuit, and/or act on subsets of qubits. In particular, the default qubit plugin state prep. operations cannot be applied to subsets of qubits, while all other plugins allow this, and does not restrict these operations to being the first in a circuit, while most others do.
Description of the Change: Adds default qubit state vector prep. methods (and accompanying tests) which act on any subset of qubits and only as the first operation in a circuit.
Benefits: Consistency across plugins, ease of use when changing only some qubits when num_wires is large.
Possible Drawbacks: Can no longer apply QubitStateVector or BasisState in the middle of a circuit using default qubit plugin.
Related GitHub Issues: Resolves #286

@josh146 @johannesjmeyer Thanks for all the comments, I've implemented the suggestions you made and will let you know when I come up with an alternative solution for the matrix multiplication (perhaps modelled off qml.utils.expand as Josh mentioned). Regarding Johannes' points above, I agree that state preparation operations should be restricted to the first set of operations in a circuit, and if this goes in PL core as you mentioned then that should make things consistent across plugins. Maybe we should open a new issue for this?

As for tests of the state prep. operation on subsets of qubits, they are in test_default_qubit.py:995 and I can extend them if that would be better.

I added a comment regarding the tests on subsets

Concerning moving those checks to core, I opened issue #386

Hi @josh146 , I've added much more comprehensive tests now for QubitStateVector. I'm not 100% convinced I have an optimal implementation using ravel_multi_index so I'd hold up on merging just now.

Okay the latest commit updates the implementation of QubitStateVector and I think this is ready for some final reviews

Is the PR good to go?

Good to go from my end!

Github is still waiting for an approval from you @johannesjmeyer before we can merge. Any objections?


update application-programming-interface.md

Signed-off-by: Gaurav Shukla<EMAIL_ADDRESS>Describe your changes
I have read application-programming-interface.md this is very beautifully written. But for the people's concern I tried to make it more readable and understandable because there are many peoples who only speak Bangla. So I have tried to contribute. I will wait for your feedback.
Related issue number or link (ex: resolves #issue-number)
Checklist before opening this PR (put x in the checkboxes)

[x] This PR does not contain plagiarism

don’t copy other people’s work unless you are quoting and contributing it to them.

[x] I have signed off on all commits

signing off (ex: git commit -s) is to affirm that commits comply DCO. If you are working locally, you could add an alias to your gitconfig by running git config --global alias.ci "commit -s".

Thanks for the contribution @withgaurav. Have you made contact with the respective localization team on Slack?

Thankyou for your response @iamNoah1. I didn't got any link of Slack to join.

@withgaurav then please make sure to read the contribution guide

@iamNoah1 Contributing guide helped me a lot. Thankyou so much @iamNoah1. Looking forward to contribute more in this organization

@withgaurav you can contact the Bengali localization team on #glossary-localization-bengali

Hey @withgaurav  , kindly create a PR to merge your file to cncf:dev-bn branch. You did create this PR for cncf-main branch which is prohibited


[Feature Request] GCS and S3 support on windows

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Will be required.
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows, all versions
TensorFlow installed from (source or binary): source and binary,
TensorFlow version (use command below): master, and all releases
Python version: python 3+
Bazel version (if compiling from source): -
GCC/Compiler version (if compiling from source): -
CUDA/cuDNN version: -
GPU model and memory: -
Exact command to reproduce: Install tf on windows, and try anything that reads from GCS or S3

Describe the problem
Currently, TF has no support for GCS and S3 on windows. Because they depend on curl, and we have not worked to make curl build and work on windows. Someone needs to dive in and work through the problems, and the rest should be just removing the windows exceptions for GCS and S3 support in configure.py.
This issue is open for community contributions.

Here's an interesting fact I learned recently, which might help the person who implements this warn users about network egress fees, or misconfigured deployments. It's possible to automatically get the GCE host region with curl -sfL metadata.google.internal/0.1/meta-data/zone | sed -n 's!.*/\([^-/]*-[^-/]*\).*!\1!p' and get the GCS $BUCKET region prefix with curl -sfLH "Authorization:Bearer $(curl -sfLH Metadata-Flavor:Google metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token | sed -n 's/.*"access_token": *"\([^"]*\)".*/\1/p')" "https://www.googleapis.com/storage/v1/b/$BUCKET" | sed -n 's/.*"location": *"\([^"]*\)".*/\1/p' | tr A-Z a-z. If the DNS lookup fails, or the first string doesn't start with the second, then GCS egress isn't free. This is especially helpful for TensorBoard users, since it does bandwidth intensive log reading.

I am interested in this feature and I am working on my proposal for this feature. Could you please provide more detail. Thank you 😄

@vnvo2409 I can help you on this issue with respect to S3. If I remember, there were several dependencies that needs to be resolved for S3. curl is one, boringssl could be another one. In tensorflow/io we get the curl working in windows so you may copy the BUILD file and give it a try.
On a side note, there are ongoing discussions about moving S3/GCS support to tensorflow/io once modular file system C API is in place, @vnvo2409 maybe we should start the effort from tensorflow/io? Once the build is in place, we could looking into moving to modular file system. /cc @mihaimaruseac @gunan for input.

Let's keep this on hold for a while longer until Windows modular filesystem is released (if all goes well, at most 3-4 weeks?)

@yongtang I wrote a small C++ program to list the content of a bucket on S3 ( by sending a curl http get request) and it works with openssl and the libcurl provided by Windows. However, I did not manage to build the curl by tensorflow/io on window. Could you please provide more instructions about building it on windows.
@mihaimaruseac If the modular filesystem is released and goes well, will we continue working on this feature ( by resolving the missing dependencies ) or we will write plugins to provide S3 and GCS support. If there is any change, please let me know because the deadline to submit proposal for GSoC will be 31th March.
And finally, beside sending my draft to<EMAIL_ADDRESS>is there anyone could provide feedbacks for me. Thank you 😄

@vnvo2409 Building tensorflow/io on windows is done through Bazel, we have GitHub Actions CI to check every commit: https://github.com/tensorflow/io/actions
The config file of GitHub actions is located: https://github.com/tensorflow/io/blob/master/.github/workflows/build.yml
Inside you can find related Windows part:
https://github.com/tensorflow/io/blob/edd02b2cefd1dbf719990b42ea43de4c4eede2b5/.github/workflows/build.yml#L241-L270
Setting up Bazel on Windows is not exactly straightforward, (that is also why the the issue of porting GCS/S3 to Windows is still open). But we are here to resolve it.
@vnvo2409 Could you open an issue in tensorflow/io  if you encountered any issues in building it on Windows?

@vnvo2409 once we have modular filesystem support for windows, we will start replacing the current ones with plugins for GCS, S3, Hadoop. Plan is for these plugins to reside under tensorflow/io and to work from the start on Linux, Mac and Windows.
Note that there are also two other students that are supposed to start working on filesystems as part of ROSEdu CDL (open source development course)

I built tensorflow on windows with s3 successfully by removing the line clean_dep("//tensorflow:windows"): [], in tensorflow/core/platform/default/build_config.bzl. However, when I tried to test, the exceptions 'file scheme is not implemented' are remains. Could someone please tell me how to remove the exceptions ? Thank you :smile:

Interesting. It shouldn't use the tensorflow/core/platform/default paths, but the tensorflow/core/platform/windows ones if on Windows.

I ask @yongtang  and he told me to remove this line https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/platform/default/build_config.bzl#L656-L660 and It trigger the s3 builds on windows. Anyway, I think that It is built succesfully. However, the exceptions "File system S3 is not implemented" are remains. Do you @mihaimaruseac know where to remove the exceptions ?
In addition, I am considering to change my proposal idea for Google Summer Of Code into write the plugins S3/GCS for your modular filesystem, could you please tell me which filesystems those 2 students work on ?
Thank you 😄

@vnvo2409 The "File system S3 is not implemented"  error might be caused by the fact that s3:// schema is not registered. Beside building successfully the schema also need to be registered. I haven't touched this part of the code for quite sometime so thing might have changed. On a side note I think write plugins for modular file system C API would be better.

When exploring the source code, I think that writing plugins for modular file system would be easier if we already had S3/GCS supports on Windows ( The plugins and the support are somehow similar, aren't they ?) So I will try to register s3:// schema, could you tell me more detail how to register. Even if it is not so useful, it will be a great opportunity for me to learn more about tensorflow source code and bazel :smile:

In addition, Could anyone please tell me who will be the mentor for the GCS and S3 supports on Windows ? It would be great if I could contact the mentor and have some feedbacks for my proposal. Thank you

@vnvo2409 The registration code is in:
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/platform/s3/s3_file_system.cc#L1039
So it should have been registered if the build is successful. You may need to trace more to see where the issue comes from.

In addition, Could anyone please tell me who will be the mentor for the GCS and S3 supports on Windows ? It would be great if I could contact the mentor and have some feedbacks for my proposal.

I don't know who was the original mentor that proposed S3/GCS on Windows. You can contact organizer and send email to tensorflow-developers (google group) email list. If no mentor, I could
sync up with organizer to see if I can be mentor. (Served as GSoC mentor last year but that was a different project).

I think the issue may come from what @gunan say at the open of issues. I did dive into the configure.py but I found nothing

and the rest should be just removing the windows exceptions for GCS and S3 support in configure.py.

Sorry for late update, I went away for a while.
I will probably be the mentor for this project since I proposed the refactoring in tensorflow/community#101
The only lines with clean_dep("//tensorflow:windows") : [] are these 3 and indeed, removing each one of them makes the specific filesystem be compiled on Windows. So, I was wrong above when I said that platform/default/build_config.bzl doesn't need to be changed. Apologies.
Filesystem not registered is because the file that contains this line has not been compiled on Windows (https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/platform/s3/s3_file_system.cc#L1039 as suggested by @yongtang ) Can you try removing this windows deps and compiling again? You can also add a print at this line and print the scheme argument. That should tell you which filesystems are registered and in what order? Or, in a C++ code linking to TensorFlow, you can call Env::GetRegisteredFileSystemSchemes. See an example here
Regarding the modular API, we are still in flux now working on the Windows one. Every modular implementation should be at tensorflow/c/experimental/filesystem/ Please look through the interface defined there and the implementation of the POSIX filesystem.
My other two students haven't yet started implementation of the modular filesystems, so we can start here instead, since I adhere to the philosophy of "whoever writes the code first gets the PR merged"

@mihaimaruseac So I will start implementing the plugins right away. Could I send you my proposal and get some feedbacks about it please ?
Thank you :smile:

I try std::cout the scheme argument and here is what I found

It seems that the s3:// file schema has already been implemented and the exceptions are hard-coded into some file ? I have to put the # to make it into a comment to prevent conflict with auto-generated file.
I will try the Env::GetRegisteredFileSystemSchemes and come back later

I will look over the proposal.
The output seems to come from compiling the code. Question is what is the output when you run the program

the output when I import tensorflow is the same.

However, the compile output of Env::GetRegisteredFileSystemSchemes does not give me s3. I will dive into it.

That's interesting. Can you try again by doing python -vvv -c "import tensorflow" &> log and then sharing the entire log file?

I redirection all the outputs and errors so it is a huge file ( 6mb ~ 68k line )
python_log.txt

I think it might be useful. I put a static variable in the function FileSystemRegistryImpl::Register to count how many time it is called. It turn out that it is called by more than one process (?)

the first process call it 3 times with "s3", "" and "file" ( the last two are for windows ). However, from the second process ( when the static variable is 1 ), "s3" is not registered anymore and therefore the exceptions happen.

Yes, I was thinking to tell you to print the address of registry_, both in FileSystemRegistryImpl::Register and FileSystemRegistryImpl::GetRegisteredFileSystemSchemes

Another thing to try: locate the dll of the s3 filesystem and then try to import it from python, via tf.load_library

ok I will try. Thank you

@mihaimaruseac There is no dll of s3 filesystem, there is only a s3_file_system.lo.lib which can not be loaded into tensorflow. I think that the problems might be come from genrule (you can see the call for Register is from executing genrule, as it needs to generate file, it call FileSystemRegistryImpl::Register for Windows and we lost the s3://. I think I will try to hard-code the REGISTER_FILE_SYSTEM("s3", RetryingS3FileSystem); into this file https://github.com/tensorflow/tensorflow/blob/f7b6793c6611210405d066dde84cc67adca4097c/tensorflow/core/platform/windows/env.cc#L193

Yeah, every genrule is a different process which does another filesystem registration. So this makes sense so far.
It is interesting that there is no dll. I think first step would be to make the necessary build changes to create one. Probably we need a tf_cc_shared_object rule, like I did for the modular windows plugin

I can not figure out how to deal with the genrule. I can not build the dll too. However, there is a good news that @gunan said that he has compiled successfully gcs support on Windows https://groups.google.com/a/tensorflow.org/forum/#!topic/developers/wYlMfe3KIYk . I think the best thing I can do now is waiting for his reply about the exceptions. I will also change my proposal to the second approach as the first approach is nearly done. Could you review my proposal for me when it is done @mihaimaruseac ?
Thank you 😄

Sounds good to me

Certainly a lot happened on this thread. I will try to summarize my responses.
In TFs codebase, now curl builds on windows. I have resolved all issues a couple weeks ago.
I finally also have tensorflow/core/platform/cloud:gcs_file_system_test passing, too.
about the changes to platform/default/build_config.bzl. You cannot simply remove the tensorflow:windows": [] I think more than filesystems depend on that. You will need to look deeper into what build targets use these macros.
About the exceptions, as @yongtang mentioned, File system S3 is not implemented means S3 module is just not built. When the module is built into the binary, it registers the file name schema for s3, and the error goes away.

I think there may be an ODR violation, file system registry may be linked twice. Because even when I explicitly built GCS support, I seem to be missing it.

So you have the same problems with me ? @gunan

@mihaimaruseac I finished my proposal. Thank you for review it one more times :smile:

I can not build //tensorflow/core/platform/s3:s3_file_system_test becasue of this error

I think the root of it is there are two file 1 and 2 which both contain the rule to build bfloat16.dll
Could anyone help me ? Thank you

Interesting. But, afaik, the two rules should generate two different files in two different paths.
I need to set-up a Windows environment and check

That is a known issue.
Due to path length restrictions, bazel has these name clashes on windows
when running bazel test.
You need to add "--dynamic_mode=off" flag to your bazel test commands on
windows to be able to tests
On Wed, Mar 25, 2020 at 9:30 AM Mihai Maruseac<EMAIL_ADDRESS>wrote:

Interesting. But, afaik, the two rules should generate two different files
in two different paths.
I need to set-up a Windows environment and check
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/issues/19297#issuecomment-603944278,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AB4UEOLTHFS7QXYWKS5FDUDRJIWT7ANCNFSM4E77X7UA
.

I run the test and not a single test pass. However, there is a good news that the registry_ only linked one times and the s3:// works. I think we have to edit the path to make it works on windows

Another question, could anyone show me how to set up a build trigger process ? I see that tensorflow using cloud build API by GCP but I am not sure how to set up for my own

Good news, S3 has passed all unit test except HasAtomicMove which does not compile because NeedTempsLocation is not belong to s3fs. I have to set up my own bucket on S3 sever to test. Is there a way to mock test it ?
Anyway, the only thing left to do is solving the exceptions 🚀

That's great.
We would prefer to not mock the filesystem tests because that could result in divergent behavior between actual use and the mock.
Regarding the missing method, that might be because when it was added recently no one also ran the S3 tests.

Ok. I am working on the exception and I will update when I find something ?
Do you have any suggetion where to dig in for me ? @mihaimaruseac

@vnvo2409 In tensorflow-io we use localstack, which is a AWS stack emulator, to test AWS related ops. It has been quite useful to us. This might be suitable for tensorflow as tensorflow's code base and test running time is already quite long. But once s3 is migrated to tensorflow-io we could reuse localstack to setup the test for s3.

I see that the registry_ used by Env::GetFileSystemForFile is the fifth, not the very last one as I thought ( I've already tried many times )
@mihaimaruseac I think that I need help now because it seems very werid to me as I think that the next registry_ should override the last one. I could not get any further debug information. Could you explain the order of linking registry_ for me
Thank you.

In theory it should be just one global variable. But because it is used across multiple shared objects, there is a possibility that one shared object gets a registry_ initialized at a different address. The keyword here is ODR violation (one definition rule). We need to debug the build and find out which DLLs provide the alternate definition

Good news every one. I have made s3 worked on Windows. I have to add to this rule deps
select({
        "//tensorflow:windows": ["//tensorflow/core/platform/s3:s3_file_system"],
        "//conditions:default": [],
})

https://github.com/tensorflow/tensorflow/blob/907a55ebadce3c78bef73148613b327ed84d92e4/tensorflow/python/BUILD#L6070-L6081

@mihaimaruseac Now I will make one PR for s3 and another ones for gcs or I should make two filesystems in one PR ?

This addition works, but it has a downside. Now the filesystem is linked always on Windows, even for users who might not want to use S3.
Preferably we should provide a tf_cc_shared_object rule to build the DLL of the S3 filesystem on Windows (and an .so on POSIX) and then in Python we should use load_library to load the plugin on demand.

I build tensorflow on Linux and s3 is also loaded when we import tensorflow. So I think this addition is more suitatble for the tensorflow source code right now. About the idea of building the DLL of the S3 filesystem, it is more suitable when we finish the implementation of filesystem modular. Because right now, in the source code of tensorflow, we always register a filesystem by a macro even for users who do not use s3.
https://github.com/tensorflow/tensorflow/blob/ca6e96effcd3d3bcb9eec96e09832519b1acda70/tensorflow/core/platform/s3/s3_file_system.cc#L1039
I think also find the cause of ODR violation, it is because on windows tensorflow use 2 folder core/platform/default and core/platform/windows instead of only core/platform/default like on Linux. To solve the ODR violation, I think we have to re-organize the 2 folder.
@mihaimaruseac  I would like to ask you, what should I do, try to solve the ODR violation on Windows or begining to work on the plugins for modular filesystem ?

I think I finally found the root cause of ODR violation. The problem is about the different of bazel config between Windows and Linux. This line is default on Windows but it is not on Linux
https://github.com/tensorflow/tensorflow/blob/5daf0e00e9913a0f7bd262f5f37ea70d34b00811/.bazelrc#L119
On linux, we have a shared object named libtensorflow_framework.so. which hold a common instance of Env::Default() and is imported into all shared objects that require a static instance of Env::Default(). So If there is a need of accessing to filesystem, it can use the common Env::Default() rather then creating a new instance.
On Windows, that line is considered into the bazel build config, which means it does not build libtensorflow_framework.dll. The _pywrap_tensorflow_internal.so is imported into these shared objects, but it does not export the symbols Env::Default() so these shared objects must create a new instance of Env::Default().

On Linux, when I build with --config=monolithic, the behavior is extractly the same on Windows. s3 is not linked into tensorflow and there are several ODR violations, just like windows

I think the plugin has to be loaded manually via load_library. At least on Linux the shared object can be built.

@mihaimaruseac Could you tell me more about the current implementation of C API Modular Filesystem. So that I can have a plan working on the plugin and not to put too much efforts here

That sounds good.
The entire implementation is under c/experimental/filesystem. There is a test suite, the API interface plugins needs to be compiled against (with a lot of comments) and some other glue files.
The plugins are under c/experimental/filesystem/plugins. POSIX is fully implemented as of now, Windows is just the starting stub, but expect more changes to come there in the coming weeks.
Please read at the least the API (takes precedence over tensorflow/community#101) and let me know if there are questions

Everything you send me require me a MOMA Single Sign On and I do not have the account 😢

@vnvo2409 The files are located on the master branch of the tensorflow repo. You can replace name mentioned by @mihaimaruseac with the repo, e.g., :
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/c/experimental/filesystem

thank you

Oh, sorry, I was using the links from my work computer. You can also replace the prefix of the links from osscs.corp.google.com, to cs.opensource.google

@mihaimaruseac Can I use AWS sdk for C++ for s3 plugin or I should use as much C as possible ? A small note, I have to delete 2 lines to able to test libposix_filesystem.so
https://github.com/tensorflow/tensorflow/blob/3364a141f2a7dc151154652e2e231516d6ee43cc/tensorflow/core/platform/default/env.cc#L215-L216

Apologies for the delay in answering.
Regarding the deletion of the two lines: yes, that is expected. Otherwise the filesystem in use is the old one. See #37734 for the latest updates w.r.t. testing. I should actually create a guide for how to test these but was planning on doing that after Windows support was ready.
Regarding language usage: the plugin could use C++ and libraries but the interface must be valid C.

@mihaimaruseac I have one more question, Could I reuse some part of tensorflow core or not ? I see the current implementation of s3 filesystem is heavily depended on the lib in tensorflow/core/. I think it is not fit to the principle of Modular Tensorflow.

Regarding 1: preferably not because then the plugin and the core of TensorFlow will still be entwined to the same compiler version and compile flags, breaking any guarantee that we want to achieve with the modularization. Instead, we should identify what are the common things and expose those to an interface, if there are multiple users.
Regarding 2: We would prefer to not initialize static global variables which are not trivially destructible. Do we need shared_ptr on the types? Keep in mind that the opaque structs in the API can be used to hide plugin specific implementation on the plugin side

I think we need something to keep a single instance of variable of type S3Client to use when s3 plugins is loaeded because allocating a new S3Client each time we need would be a waste of resource.

Agree. But the plugin is only loaded a single time, it cannot be unloaded

So I suggest using a std::weak_ptr
Something like
// s3_filesystem.cc
namespace tf_random_access_file {

typedef struct S3File {
	const char* bucket_;
	const char* object_;
	std::shared_ptr<Aws::S3::S3Client> s3_client_;
};
}

// s3_helper.h
namespace tf_s3_filesystem {
  std::shared_ptr<Aws::S3::S3Client> GetS3Client();
  std::shared_ptr<Aws::Transfer::TransferManager> GetTransferManager();
  std::shared_ptr<Aws::Utils::Threading::PooledThreadExecutor> GetExecutor();
}  // namespace tf_s3_filesystem

// s3_helper.cc
// use anonymous namespace to hide these pointers
namespace tf_s3_filesystem {
  namespace {
    std::weak_ptr<Aws::S3::S3Client> s3_client_;
    std::weak_ptr<Aws::Transfer::TransferManager> transfer_manager_;
    std::weak_ptr<Aws::Utils::Threading::PooledThreadExecutor> thread_executor_;
  }

  std::shared_ptr<Aws::S3::S3Client> GetS3Client() {
    if(s3_client_.get() == nullptr ) {
      //...
      s3_client_ = std::shared_ptr<Aws::S3::S3Client>(...);
    }
    return s3_client_;
  }
}  // namespace tf_s3_filesystem

The global static will be reference by a weak_ptr<Aws::S3::S3Client>. If something need to use it, we could return a shared_ptr<Aws::S3::S3Client>. When it is no longer use anymore, It will be destructible right at that moment, and we don't have to keep it throughout the program as the plugin can not be unloaded.

This looks like something that should be done in the function passed to init.
In the plugin, you can have a private struct that keeps these pointers (naked pointer should be ok, I don't see a reason to have them be std::weak_ptr or other wrappers at the moment) and then you attach that struct to the opaque TF_Filesystem.
For example, POSIX init does nothing, but you could have something like
namespace tf_s3_filesystem {

typedef struct S3Data {
    Aws::S3::S3Client *s3_client;
} S3Data;

static void Init(TF_Filesystem* filesystem, TF_Status* status) {
  auto s3data = static_cast<S3Data*>(filesystem->plugin_filesystem);
  s3data->s3_client = ...;
  TF_SetStatus(status, TF_OK, "");
}

}  // namespace tf_s3_filesystem

Then, when you create a file:

namespace tf_s3_random_access_file {

typedef struct S3File {
	const char* bucket;
	const char* object;
	const Aws::S3::S3Client *s3_client; // not owned
};
}  // namespace tf_s3_random_access_file

namespace tf_s3_filesystem {

static void NewWritableFile(const TF_Filesystem* filesystem, const char* path,
                            TF_WritableFile* file, TF_Status* status) {
  auto s3data = static_cast<S3Data*>(filesystem->plugin_filesystem);
  ...
  file->plugin_file = new tf_random_access_file::S3File({bucket, object, s3data->s3_client})
  TF_SetStatus(status, TF_OK, "");
}

}  // namespace tf_s3_filesystem

This way you don't need globals.

Thank you. I have misunderstood the usage of const TF_Filesystem* filesystem

Regarding the pointers, yes, if the API already uses wrapped ones then we should incorporate them too. I was not checking the AWS API.
Regarding the second point: that's great, I like the lazy loading approach you suggested.

Here is what I have done with the s3 plugins

I have created a s3 filesystem plugins based on the current implementation of s3 filesystem.
Here is the test log. attempt_1.log. As you can see. 94/103 test passed. Here is the link to my test bucket on s3. I have removed some test:

TestTranslateName because the default TranslateName remove the bucket name ( s3://bucket/tmp/test -> /tmp/test/) So I have to write my own TranslateName.
Any test related to MemoryRegion. Seg fault so I will work on it later.
TestStatPathIsInvalid I use Stat to check if path is invaild or not so this test can not be run.

Test time is very long. It is because most of the tests are not compability with how s3 filesystem work and they just add the complexity to the operation of s3 plugins filesystem. s3 does not have the directory concept. s3 has a flat structure instead of a hierarchy like in a normal filesystem. Take an example, with only one bucket bucketvnvo without any "directory"

aws s3api put-object --bucket bucketvnvo --key test/vnvo/file
aws s3api put-object --bucket bucketvnvo --key test/vnvo/file/dir

These 2 command create 2 objects with have the key test/vnvo/file and test/vnvo/file/dir. It does not care if the "parent directory" is exist or point to a file or something as there is no "parent directory". So TestCreateFileNonExisting, TestCreateFileExistingDir, TestCreateFilePathIsInvalid, etc are not so useful. I will remove some path pre-checker in the s3 plugins filesystem and I think we should rewrite the modular filesystem test for s3
A side note, I have some exams so I will be less active till the student announcement (May 5) of the Google Summer of Code.

Since S3 does not support directories those operations should return the not implemented status. Then tests would be marked as passing and skipped fast.
The goal for the tests is to provide a checker that all plugins adhere to the specification of the API in the modular interface.
Do you have a PR with the code so we can look through that?
No worries about the delay, good luck on the exams

@yongtang I see you have fixed the gcs_filesystem linking problems on Windows by this PR https://github.com/tensorflow/tensorflow/pull/38959. Do you think I should make a PR for s3_filesystem on Windows as well ( it has already passed all tests ) and we can close this issue.

That sounds good, @vnvo2409. Please assign/mention me in the PR.

@mihaimaruseac I have made a PR, please review it.
https://github.com/tensorflow/tensorflow/pull/39124

Closed by #39124


Remove dom from node-only libs

As disussed here: https://github.com/open-telemetry/opentelemetry-js/issues/585#issuecomment-610516259
We should remove the possibility of using dom globals in libs we know to be node-only by setting libs in tsconfig

Consolidating issues and conversation to keep it from being as fragmented:
From @Flarna in #585

Maybe the projects should be configured more strict. e.g. plugin-http is node only therefore lib dom should be not needed/used at all. As there is no lib config in tsconfig.json of the plugin-http nor in tsconfig.base.json typescript includes dom on default.

I'm not sure this would help since several packages like api and core which are required to use otel at all would pull the dom types in.


nheqminer 0.4b for Linux download

Statically compiled on Fedora 24 (+ all updates installed) for -march=sandybridge.
Must be compatible with all existing 64bit Linux distros: Ubuntu, Mint, Fedora, Arch, etc.
Found/Download here
sha1sum: e3fa9b0807ecc3d11ee3cbd0c1f71f04d649fe1b
It's a CPU version!

How to use on a VPS ?

Issue related to older version.
Closing...


Create pre processing py file

Creating a pre-processing py file.
So far, we could load red and white wine data and combine those data sets vertically.
After selecting our model, additional code might be needed for pre-processing.
@wang-rui I haven't merged #Target distribution plot part in the script. If you think that part should be in pre-processing script, please let me know.

Hi @rissangs
My comments:

For download script: suggest to change the command instruction to the following to save the two data files under '/data/raw' folder instead of '/data' folder

python src/download_data.py --url="https://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/winequality-red.csv" --out_file="data/raw/winequality-red.csv"

python src/download_data.py --url="https://archive.ics.uci.edu/ml/machine-learning-databases/wine-quality/winequality-white.csv" --out_file="data/raw/winequality-white.csv"

For the preprocessing script:

To add command line to run the scripts to incorporate to Usage section of README.md
The documentation on ---out_dir should change:
from

--out_dir=<out_dir>          Path (including file name) of where to locally write the file

to
--out_dir=<out_dir>          Path (excluding file name) of where to locally write the file

as I think you are asking for the directory not the filename.

Please help debug as when I run the following command

python src/pre_processing_wine.py --in_file_1="data/raw/winequality-red.csv" --in_file_2="data/raw/winequality-white.csv" --out_dir="data/processed"

I got the following error
file_2="data/raw/winequality-white.csv" --out_dir="data/processed/processed.csv"
Traceback (most recent call last):
  File "C:\Users\truon\miniconda3\envs\wine_quality\lib\site-packages\pandas\core\in
dexes\base.py", line 2895, in get_loc
    return self._engine.get_loc(casted_key)
  File "pandas\_libs\index.pyx", line 70, in pandas._libs.index.IndexEngine.get_loc
  File "pandas\_libs\index.pyx", line 101, in pandas._libs.index.IndexEngine.get_loc

  File "pandas\_libs\hashtable_class_helper.pxi", line 1675, in pandas._libs.hashtab
le.PyObjectHashTable.get_item
  File "pandas\_libs\hashtable_class_helper.pxi", line 1683, in pandas._libs.hashtab
le.PyObjectHashTable.get_item
KeyError: 'quality'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "src/pre_processing_wine.py", line 51, in <module>
    main(opt["--in_file_1"], opt["--in_file_2"], opt["--out_dir"])
  File "src/pre_processing_wine.py", line 35, in main
    (data['quality'].eq(3) | data['quality'].eq(4)),
  File "C:\Users\truon\miniconda3\envs\wine_quality\lib\site-packages\pandas\core\fr
ame.py", line 2906, in __getitem__
    indexer = self.columns.get_loc(key)
  File "C:\Users\truon\miniconda3\envs\wine_quality\lib\site-packages\pandas\core\in
dexes\base.py", line 2897, in get_loc
    raise KeyError(key) from err
KeyError: 'quality'

Thanks for your review @athy9193.
I will send another PR with updating README.md.
For the bug above, you missed / at the end of --out_dir. Please try the code below.
python src/pre_processing_wine.py --in_file_1="data/raw/winequality-red.csv" --in_file_2="data/raw/winequality-white.csv" --out_dir="data/processed/"


riscof validateyaml fails due to special characters in schema_isa.yaml

Hi,
I am seeing the same errors as have been previously reported in this issue #17.
> riscof validateyaml --config=config.ini
    INFO | ****** RISCOF: RISC-V Architectural Test Framework 1.25.3 *******
    INFO | using riscv_isac version : 0.18.0
    INFO | using riscv_config version : 3.17.0
...
ruamel.yaml.reader.ReaderError: unacceptable character #x0080: special characters are not allowed
  in "/work/kjostera/experiments/riscof/venv_py310/lib/python3.10/site-packages/riscv_config/schemas/schema_isa.yaml", position 18507

Here are some more details on the environment where this has been reproduced.
> python3 --version
Python 3.10.6
> riscof --version
RISC-V Architectural Test Framework., version 1.25.3
> riscv-config --version
Version: 3.17.0

This is due to the special tick characters that are used in multiple description fields. Note that I do not see this error on all platforms so there might be some platform dependent encoding configurations that play a part in this issue. I can try to create a PR which replaces these characters with the ' character instead.

PR can be found in the riscv-config project: https://github.com/riscv-software-src/riscv-config/pull/168


Fixes #65 Use constraint layout in activity_media_details.xml

Fixes #65

Modified the activity_media_detail layout  from LinearLayout to ConstraintLayout .

Congrats on your first PR @MarinaTaran

Thank you!


[BEAM-5744] Cherrypick PR 6686  to 2.8.0. release branch:

Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

It will help us expedite review of your Pull Request if you tag someone (e.g. @username) to look at it.


release: Bump version to 0.13.3

@Lusitaniae what steps do I need to take to trigger a new release?

need to bump VERSION file and create a tag/release (I usually do it from the interface https://github.com/Lusitaniae/apache_exporter/releases/new


Error while running the code

'T_student': config.get('T_student') : 'NoneType' object has no attribute 'get'
''lambda_student't': config.get(''lambda_student'') : 'NoneType' object has no attribute 'get'
I got T_student is a temperature that is used for soften probability values.
What is lambda_student? From where (T and lambda) values are coming?
Please tell to fix above issue.
Thanks

Fixed this issue.


Cleaned up AD FS custom auth development documentation

Removed duplicated code and changed code blocks to csharp blocks
Removed random periods in gacutil commands

Checking with engineering.

Hi @themattman18
Unfortunately I haven't heard back from engineering yet.  So, I am going to go ahead and create a work item to track this.
For now, I am going to go ahead and close the PR and I will re-open it once I have heard back.
Thanks!
Bill
#please-close

The review process should be quite a lot easier if you use the view Hide whitespace changes (https://github.com/MicrosoftDocs/windowsserverdocs/pull/4779/files?diff=unified&w=1) to see that most of the changes are formatting improvements.
The formatting improvements are straightforward and can be approved quite easily.
(I tried the rich diff view, but it leaves me with more questions than answers vs. the duplication removal, because this topic is not within my area of expertise.)
The large section of code deduplication is a bit more cumbersome for me to review, since I don't do any coding worth mentioning.
This is about the 2 big red blocks (Lines 203-222 & Lines 224-344):  https://github.com/MicrosoftDocs/windowsserverdocs/pull/4779/files?diff=unified&w=1#diff-95188fd5c2ced5f1a2b6e90c61177c2ef06e0f8586128c37fe9b438e02d69474L203-L344
To be able to verify that it is OK to remove that part, I would have to know this topic well enough to use it straight away as a lab work example.
I can only assume that the suggestion to remove the unique (for the page) text in Lines 203-207 is based on the fact that the code following step 9 is already used in the preceding examples (this is the part which could be easily verified by someone familiar with this particular form of scripting).