Issue using cub reduce on more than elements than fit into a 4 byte integer

Reduce (void *d_temp_storage, size_t &temp_storage_bytes, InputIteratorT d_in, OutputIteratorT d_out, int num_items, ReductionOpT reduction_op, T init, cudaStream_t stream=0, bool debug_synchronous=false)
my issue seems to be that num_items is of type int so when I try to  reduce more elements than fit into a 4 byte integer it overflows and the code obviously doesn't work properly. Given that GPUs are both growing in RAM size and that we can now oversubscribe by using cudaSharedMalloc are there any plans to change that number to be able to receive type size_t?

So, an explanation and a solution:
CUB's algorithms actually aren't hard-coded for 32-bit counts -- they are specialized by template parameter.  However, 64b offsets require twice the register file as 32b offsets, and many of the algorithms (prefix sum, radix sort, etc.) have pervasive bookkeeping offsets and counts, so specializing for 64b counts often reduces performance as RF pressure reduces occupancy.  So... the outer interface specializes everything for 32b int counts because the majority of people aren't reducing/scanning/sorting more than 2 billion items.
If you want to use a 64b count (e.g., int64_t or size_t` or whatever), you can simply invoke the more generic interface underneath.  See:
https://github.com/NVlabs/cub/blob/1.8.0/cub/device/device_reduce.cuh#L148
for example (which is where the outer interface specializes int as the offset type). Let me know if that unsticks you,
Duane

Ok yes i see how to do this now. This does unstick us thank you.
Felipe

So maybe I'm missing something here, but it appears that num_items is still int. Is there a way to relax that constraint? It would be useful to have something like size_t here instead. Thoughts? 🙂

@jakirkham I guess what @dumerrill meant is to invoke DispatchReduce::Dispatch() ourselves with num_items being size_t?

Ah sorry. I got overly focused on the highlighted line. So IIUC we should be looking here. Is that right?

Yeah I guess so.

Also if using 32-bit is significantly more performant than 64-bit, what is the recommendation for doing reductions that exceed the size of 32-bit signed integers?
Additionally I understand that 32-bit has special value here, but why is a signed value used instead of an unsigned one? Switching would double the size of allowed values without affecting the number of bits used.

Closing as this is part of a larger issue being tracked in #212.


codepipeline: There should be a way to insert a stage before an existing one (and not append)

Currently, Pipelines only allows appending stages. This creates a limitation in usability. For example, we have a general purpose construct that defines a pipeline for code library builds. The current pipeline has two stages: build and release. We wish to implement a method .addTest(...), which will conditionally "inject" another stage between "build" and "release" only if it's called. If there are no tests, there should still be two stages.
Maybe we can do something like new Stage(pipeline, 'Bla', { insertBefore: someOtherStage }).

Moved to PR #460 after problems with the fork when the repo was changed to public.


Large partition key using ARM templates.

FYI:  when defining a container using an ARM template, adding "version":2 in the partitionKey definition will deploy the new version with large partition keys enabled.
I didn't see this here, or in the ARM template reference docs here: https://docs.microsoft.com/en-us/azure/templates/microsoft.documentdb/2015-04-08/databaseaccounts/apis/databases/containers#ContainerPartitionKey
I just guessed that it might work, tried it, and... it worked!  Might be helpful to document it somewhere.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: c628b14e-b23f-3d87-5456-bbcf40ced3f9
Version Independent ID: 6f9d5320-e3f3-277a-b141-c795ea1a58d3
Content: Create Azure Cosmos containers with large partition key using the Azure portal and various SDKs.
Content Source: articles/cosmos-db/large-partition-keys.md
Service: cosmos-db
GitHub Login: @markjbrown
Microsoft Alias: mjbrown

@joshlang Thanks for the feedback. We are checking this and will get back to you.

@joshlang We are checking this with our doc author.
@markjbrown @SnehaGunda Please confirm if we have ARM template support for creating large partition keys. If yes, update the document to reflect the same.

@joshlang we document the versions to be used for large partition key here: https://docs.microsoft.com/en-us/azure/cosmos-db/large-partition-keys ARM template is missing from the list, will add it.
Just confirming again, you added the version property to the PartitionKey object right?
"partitionKey": {
"paths": [
"/myPartitionKey"
],
"kind": "Hash",
"version":2
},

@markjbrown Is it okay to update the ARM template to use a large partition key by default?

Just confirming again, you added the version property to the PartitionKey object right?
"partitionKey": {
"paths": [
"/myPartitionKey"
],
"kind": "Hash",
"version":2
},

Yes - I used  "version": 2 and it worked.
When "version" is missing, it defaults to version 1 (or whatever you call the old version).

Added a note to the doc in the above PR.

We will now close this PR if there aren't further questions.
#please-close


Bug: Cannot archive pdf page.

Describe the bug

I was trying to archive a URL that just returned a PDF file. I have only singlefile and PDF enabled, but archivebox didn't archive anything except the title, headers, and the favicon.
Steps to reproduce

Running archivebox in docker with everything disabled exception singlefile and pdf
Archive a webpage that gives back a pdf (https://ifs.host.cs.st-andrews.ac.uk/Books/SE9/Web/ExtraChaps/Documentation.pdf) for example.
See what actually gets saved in the archive directory

Screenshots or log output

Result of doing the same with the cli:
[i] [2022-07-25 21:53:38] ArchiveBox v0.6.3: archivebox add https://ifs.host.cs.st-andrews.ac.uk/Books/SE9/Web/ExtraChaps/Documentation.pdf
    > /data

[+] [2022-07-25 21:53:38] Adding 1 links to index (crawl depth=0)...
    > Saved verbatim input to sources/1658786018-import.txt
    > Parsed 1 URLs from input (Generic TXT)
    > Found 1 new URLs not already in index

[*] [2022-07-25 21:53:38] Writing 1 links to main index...
    √ ./index.sqlite3

[▶] [2022-07-25 21:53:38] Starting archiving of 1 snapshots in index...

[+] [2022-07-25 21:53:38] "ifs.host.cs.st-andrews.ac.uk/Books/SE9/Web/ExtraChaps/Documentation.pdf"
    https://ifs.host.cs.st-andrews.ac.uk/Books/SE9/Web/ExtraChaps/Documentation.pdf
    > ./archive/1658786018.446854
      > title
      > favicon
      > headers
        4 files (241.4 KB) in 0:00:01s

[√] [2022-07-25 21:53:40] Update of 1 pages complete (1.71 sec)
    - 0 links skipped
    - 1 links updated
    - 0 links had errors

ArchiveBox version

ArchiveBox v0.6.3
Cpython Linux Linux-5.10.0-15-amd64-x86_64-with-glibc2.31 x86_64
IN_DOCKER=True DEBUG=False IS_TTY=True TZ=UTC SEARCH_BACKEND_ENGINE=ripgrep

[i] Dependency versions:
 √  ARCHIVEBOX_BINARY     v0.6.3          valid     /usr/local/bin/archivebox
 √  PYTHON_BINARY         v3.10.4         valid     /usr/local/bin/python3.10
 √  DJANGO_BINARY         v3.1.14         valid     /usr/local/lib/python3.10/site-packages/django/bin/django-admin.py
 √  CURL_BINARY           v7.74.0         valid     /usr/bin/curl
 -  WGET_BINARY           -               disabled  /usr/bin/wget
 √  NODE_BINARY           v17.9.0         valid     /usr/bin/node
 √  SINGLEFILE_BINARY     v0.3.16         valid     /node/node_modules/single-file/cli/single-file
 -  READABILITY_BINARY    -               disabled  /node/node_modules/readability-extractor/readability-extractor
 -  MERCURY_BINARY        -               disabled<EMAIL_ADDRESS>
 -  GIT_BINARY            -               disabled  /usr/bin/git
 -  YOUTUBEDL_BINARY      -               disabled  /usr/local/bin/yt-dlp
 √  CHROME_BINARY         v100.0.4896.127  valid     /usr/bin/chromium
 √  RIPGREP_BINARY        v12.1.1         valid     /usr/bin/rg

[i] Source-code locations:
 √  PACKAGE_DIR           24 files        valid     /app/archivebox
 √  TEMPLATES_DIR         4 files         valid     /app/archivebox/templates
 -  CUSTOM_TEMPLATES_DIR  -               disabled

[i] Secrets locations:
 -  CHROME_USER_DATA_DIR  -               disabled
 -  COOKIES_FILE          -               disabled

[i] Data locations:
 √  OUTPUT_DIR            6 files         valid     /data
 √  SOURCES_DIR           994 files       valid     ./sources
 √  LOGS_DIR              1 files         valid     ./logs
 √  ARCHIVE_DIR           1239 files      valid     ./archive
 √  CONFIG_FILE           81.0 Bytes      valid     ./ArchiveBox.conf
 √  SQL_INDEX             5.2 MB          valid     ./index.sqlite3

As archive box is seen as a self-hosted version of archive.org and archive.org has no problems archiving a PDF I do believe that archive box should be able to do that too.

You need to enable the wget extractor method to download staticfiles directly using their URL. The PDF extractor is designed to convert pages that aren't PDF into PDF archives, it's redundant if the file is already a PDF (which is what the wget extractor is designed for, staticfiles).

…
I could have come up with that solution myself 🤦♂


http cache-control different when image resize filters applied

Is there a config that needs to be applied in order to apply the same cache control headers when a query param is applied?

You'd like to pass a query parameter, for example: /image.png?w=400&cache_control=..., and transform the parameter into a header, correct?
In that case, you should be able to use an example configuration in Caddy with a header that retrieves the value from the query. Here’s how it could look:
If you are using file_server
localhost:80 {
    root /your/path/test-dataset

    @hasCacheControl query cache_control=*
    header @hasCacheControl Cache-Control {query.cache_control}

    file_server
}

If you are using reverse_proxy
localhost:80 {
    @hasCacheControl query cache_control=*
    reverse_proxy @hasCacheControl localhost:4000 {
        header_down Cache-Control {query.cache_control}
    }
    reverse_proxy localhost:4000
}

Ok, it seems that Caddy generates the ETag header using modification time and file size.
Then, it defers ETag deletion if response doesn't match.
In our case, the ETag is deleted because the Content-Length differs from the initial value.
I need to reimplement a specific ETag generation mechanism to handle resized files.
This assumes that the ETag cannot be the same as the initial reverse-proxied value, but I can generate a specific ETag for each resized image

I can generate an ETag, but the image processor doesn’t implement a caching mechanism to check if the newly resized image matches the client’s ETag.
As a result, the underlying caching mechanism cannot be accurate.
That said, I can generate an ETag, but I cannot handle a 304 Not Modified response when the content matches the client’s version without regenerating the image.

I'll try to develop, by the end of the year, a caching system that takes into account the initial ETag returned by the proxy or file_server.
The goal is to cache a specific version based on the processing parameters, to avoid reprocessing the image for every request.
This would help save server resources by preventing unnecessary resizing.
However, it might take me some time to implement.

Ok, I've implemented a custom mechanism to handle the ETag and If-None-Match headers according to the specific image processing for each image, if your initial proxy/file server returns an ETag specific to the file.
See https://github.com/quix-labs/caddy-image-processor/releases/tag/2.2.2

Ok, I've implemented a custom mechanism to handle the ETag and If-None-Match headers according to the specific image processing for each image, if your initial proxy/file server returns an ETag specific to the file.
See https://github.com/quix-labs/caddy-image-processor/releases/tag/2.2.2
Regarding the cache-control, it seems more legitimate to reuse the one provided initially by the proxy/file server by default. Therefore, I suggest keeping the previous logic to overwrite it with a custom cache.
I don't see the point of globally bypassing the initial cache configured on the remote proxies.

Yeah I wasn't trying to override the cache response. I was just seeing two different results with and without query params. Thanks


Build "Add Issuer" flow

This generates a public key and sends it to the issuer.

This is mostly done. The UX for the Add Issuer form is coming along nicely.

Unfortunately, the key part of it is currently faked inside an IssuerCreationRequest. Once I figure out how to properly parse data from that response, this should be good to mark as done.

New request models have been built for creating an issuer and introducing a recipient to an issuer. We have a few unit tests with mocked URLSession objects to help refine the programming model while the server that actually handles these responses is built.

We have this tested & working in the app against an instance of https://github.com/blockchain-certificates/cert-intro .


Provide Better Interface for the ffs.setDefaultConfig

Currently user needs to know the proto implementation of DefaultConfig message definition. However this is probably not good interface for the end user.

Fixed in https://github.com/pygate/pygate-gRPC/commit/3cc260907a87ae6007d060ba501b40598364d77e


Initialize global input source variable

Overview
We didn't initialize the global input source variable initially, causing warnings to appear.
Changes
Fixed error introduced by #9770

/azp run


Show hex representation of variable while debugging

I work on low-level code frequently and use VSCode as my primary editor and debugger. I'm hoping that it might be possible to configure the debugger, when viewing variables in the left panel and while hovering over the variables in the code editor space, that the hex representation of the variable's value can also be shown. Currently, I can only see the decimal and ASCII representations.

/duplicate #28025


MERGE http verb used without const

As mentioned in #563 review, there are a couple instances of hardcoded "MERGE " values being used as http verbs. Documentation for "MERGE" can be found here.
It is not included in the stdlib's http library, so it didn't make sense to replace those instances until we decide on the best place to put const MethodMerge = "MERGE".

@mcardosos & @jhendrixMSFT, thoughts on putting it in Azure/autorest?

There is a work item on my plate (I can't find it right now) to refactor constants into their own source file so they aren't duplicated; I think we can include it there yes?

https://github.com/Azure/azure-storage-go/issues/9 It is this one

^Nice. Yeah, lets go ahead and use that bug to track this item. I'll close this out.


Different length audio input for infer mode

Hi, thanks for the interesting work!
I have a question about the infer mode in htsat.py. When training, the length of audio input will always be 10 seconds. When inference, the model needs to handle variable-length audio input which could be longer or shorter than 10 seconds, but for the infer mode in the htsat.py,
 if infer_mode:
            # in infer mode. we need to handle different length audio input
            frame_num = x.shape[2]
            target_T = int(self.spec_size * self.freq_ratio)
            repeat_ratio = math.floor(target_T / frame_num)
            x = x.repeat(repeats=(1,1,repeat_ratio,1))
            x = self.reshape_wav2img(x)

What if the length of input frame_num > target_T (should be 256x4=1024 here)? If so, repeat_ratio will be 0. So how does the model process the audio longer than 10 seconds for inference here?

Hi,
Thank you for your question, it is possible to handle an audio sample that has more than 'target_T'.
Our method is to cut the audio sample into different slices, each of which has the length 'target_T'.
For example, a 15-sec piece can be sliced as [0-10] and [5-15]. And of course you can control the overlapping.
Then we send these slices into HTS-AT and get the result just like the "voting".
We did not implement this code in this repo since audio samples in our exps are always less than 10sec. But we face this question when we try to use this model in other scenario, where I actually implement this code at:
https://github.com/LAION-AI/CLAP/blob/d2d5dae8ea8f1ee02ac40242418a36d1d567943a/src/open_clip/model.py#L484-L533
You can check it and adapt it into your code.
Best,
Ke

Hi Ke, thanks for your quick reply! I have some other questions for HTS-AT.

As the paper mentioned in Section 2.1.1, mel-spectrogram is split into patch windows, then patches are split inside each window. The implementation in the code actually reshapes the mel-spectrogram from a rectangle (1024x64) to a square ((1024/4)x(64x4)=256x256), following the setting of Swin-transformer for image classification with square inputs.
I was wondering have you compared your patching method to another way that directly split non-overlapping patch without reshaping the mel-spectrogram to a square shape (I guess this patching method is also workable for swin-transformer)? By the way, I guess one of the reasons might be that reshaping the input to 256x256 can use the pre-trained Swin-T model on ImageNet.

I am assuming that the training detail mentioned in section 3.11 in the paper is for training HTS-AT from scratch. Could you provide more training details about using pre-trained Swin-T/C24?

Thank you!


Copy all files in project root directory during make setup

In make setup, instead of
	neuro cp $(REQUIREMENTS_APT) $(REQUIREMENTS_APT_STORAGE)
	neuro cp $(REQUIREMENTS_PIP) $(REQUIREMENTS_PIP_STORAGE)

we should do
	neuro cp -r . storage:

so that all files are copied, including setup.py

Yes, we should not use setup-name-of-the-project instead of copying directories separately (because they should be copied in different modes, code and notebooks: rw, data: rw?). But we definitely need to copy files in project root. I'll rename this issue

or at least copy files that we created: setup.py, etc.

better: for file in $(PROJECT_PYTHON_FILES); do neuro cp ./$$file $(PROJECT_PATH_STORAGE)/$$file; done
better to enumerate these files explicitly rather than use neuro cp * (not to copy unnecessary files like Makefile and not to fail while trying to copy directories non-recursively


Use Ivy Repository Layout for hosting Umple Artifacts

Problem
Currently, we only the latest Umple version at http://try.umple.org/scripts/umple.jar. This link is not overly obvious and forces our build scripts to manually wget the umple.jar.
Proposed Solution
Ideally we could use Ivy to actually fetch based on versions. All that Ivy requires is a directory structure that matches a pattern. For example, if we stored umple versions like: dist.umple.org/org.umple/[module]/jars/[artifact]-[revision].[ext]. Where:

[module] would be 'umple-sync', 'umple', etc.
[artifact] name of the jar
[revision] version of the artifact

This would let us use Ivy to fetch the Umple dependencies, as well. Additionally, other projects outside of umple can depend on the project and use our Ivy repository to bring umple into their projects through Maven or Gradle.
This would need to be attached to Cruise Control's build process to put the jars in the correct directories as they are built. Additionally, there should be a org.umple/[module]/jars/[artifact]-latest.[ext] version which always holds the most recent version. Unfortunately, implementing this would require access to the cruise control server.

With the work going on for a Gradle plugin in #751, I think this should also be revisited. Having Umple in a maven/ivy repository means that others can include Umple in their build chains. With the Gradle plugin coming, it will need access to multiple versions of umple.
This looks like a good place to start: https://maven.apache.org/guides/mini/guide-central-repository-upload.html

Conforming to the above, the list of Umple jars is now automatically maintained at:
http://cruise.eecs.uottawa.ca/dist/org.umple/umple/jars/
Files are umple-[revision].jar
[revision] includes 1.pp.q for definitive releases for stability. These are also found on Github
and 1.pp.q.rrrr.xxxxxx for every merge to master where xxxxxx is the hash abbreviation
Also umple.jar and umple-latest.jar always point to the most recent merge to master.
The umple.jar (latest) is the same one found at http://try.umple.org/scripts/umple.jar
http://cruise.eecs.uottawa.ca/dist/org.umple/umplesync/jars/ has the same structure for artifact umplesync
Both artifacts can be accessed starting with the root, as specified at the start of this issue
http://dist.umple.org/org.umple/
This would normally be be best URL pattern to use for portability
I will move on now to other stages of integration with Ivy and Maven

@nava2 Can you give me suggestions for next steps. I do [plan to work on issue #1009, but as the proposer of this issue, are there any steps I should take now the directories are present as requested?

This is an awesome first step.
A great second step would be to point the current build to use the ivy repo in it's ivy.xml instead of downloading it from the scripts/ folder in umple online (sorry if thats been done already!). We should do the same for the gradle project.


Automerge feature for edit

Hi,
Please support automerge feature for edits, by example If I want to make one change on my token (if I own the repository for sure) I should be able to do it.
Thanks

Please read the SPL-token README, and ensure your SPL-token info uses the correct Uniswap format.

The problem with edits is proving ownership - this is why edits need manual approval for now.
Discussions are in progress on how to best handle this going forward

Yes yes I understand, but just owning repository should be good, I mean if you listed the coin originally its a good ownership proof!

Maybe give an automerge result directly after commit and proceed them with CRON, because we have to wait 1 hour to check if we did any mistake.


1.0.5

This version implements string based filters for scans.

The PR has been updated.  Newest version does object based scan filters.

:accept:


Question: CacheStore Configuration using DDL

Hi,
We are currently working on creating cache using the Ignite DDL statements and came across a scenario where we need to map the cache data to external Database like postgres.  We tried using the xml Config and Java based code but what we are looking for is if there is any way we can configure already existing cache which was created using DDL and add the CacheJDBCBlobStore config change without bringing the nodes down or downtime. Please let us know if this possible?
Any suggestions will be helpful. thanks

It is not possible to change cache store configuration for an existing cache.
Zero downtime workaround is to create a new cache with a different configuration, copy the data, drop the old cache.

thank you for the response @ptupitsyn.

With respect to CacheJDBCBlobStore do we need to deploy the POJO class for key and value Objects we use for cache persistence or entries as we cannot peer class load these models. And if we don't upload these classes on each server we are getting below ERROR.

class org.apache.ignite.binary.BinaryInvalidTypeException: x.x.x.MyClass and java.lang.ClassNotFoundException: x.x.x.MyClass

Can we make use of UriDeploymentSpi to deploy the model classes on Ignite servers? or is there any other option to do so without the need for stopping all nodes and manually uploading these classes and starting them again?

Any suggestions will be helpful thanks.

Yes, POJO classes need to be deployed on all servers
I'm not sure about UriDeploymentSpi

@ptupitsyn - thank you for the response


Validate custom file upload control on form submit.

I've developed a custom form control for file uploads, and while the validations function correctly when a file is uploaded or removed, they fail to trigger upon form submission. I have prepared a Stackblitz to demonstrate my current approach. Could you please help identify what I might be doing wrong?
Additionally, I would appreciate any suggestions for improving the existing setup. Thank you!
Stackblitz:
https://stackblitz.com/edit/ngx-formly-ui-material-guaenq

@tHe-AK  If you want validations to be shown like required lets say you need to add wrapper to your custom type:
  types: [
        {
          name: 'fileupload',
          component: FormlyFileupload,
          wrappers: ['form-field'],
        },
      ],

There are prebuild wrappers https://formly.dev/docs/guide/custom-formly-wrapper but you can also create custom one.
 <formly-validation-message [field]="field"></formly-validation-message>
this component handles showing this validation messages.

Hi @bmickoski, I have explored formly-validation-message already and we are using it for other fields. I cannot use it for file upload as its design has errors at multiple levels (file level error[file size, file type], file category error[max, min files count], whole control level error[totalFilesSize]), which are to be displayed in different places.

So @tHe-AK i don't follow then... what is main problem? That you don't see required msg for example when you try to submit without uploaded file or i misunderstood you?

Oh i see.. you say that validations only works if you choose file otherwise they are not triggered when submitting.

You should use form-field wrapper, alos ensure to import FieldType from @ngx-formly/material.
You may check the following example https://stackblitz.com/edit/ngx-formly-ui-material-b9ahsaez


Revisit BootstrapLogger

In #1008 we discovered an occasion where the BootstrapLogger added in #803 is potentially not being used 100% effectively... We need to make sure it is fully compatible with any places we're providing runtime replacements for logging that was setup before the service container was fully built, and that we're using it consistently everywhere we can (possibly only relevant to config providers?)

After reviewing the state of the codebase, we are content with the current ways BootstrapLogger is built and used


Unable to install

I have tried installing this on Mysql 5.7 and Mysql 8.0 (both support JSON columns).  When the container first loads it is in a 'starting' state.  When I look at the logs it is filled with this error:
migration failed with error: create table auth (id int unsigned not null auto_increment primary key, created_on datetime not null, modified_on datetime not null, user_id int unsigned not null, type varchar(30) not null, secret varchar(255) not null, meta json not null default '{}', is_deleted int unsigned not null default '0') - ER_BLOB_CANT_HAVE_DEFAULT: BLOB, TEXT, GEOMETRY or JSON column 'meta' can't have a default value,
migration file "20180618015850_initial.js" failed
How can I get this to work?  I have tried relaxing the sql_mode of Mysql in the hopes of letting it go in and then put that back but that doesn't seem to have an effect.

To be honest I've done most of my testing with the latest Maria database engine which I looks like supports default blob values of any string whereas mysql JSON data types only accept default values written as expressions. The KnexJS library doesn't know how to handle the different between them yet so I've updates the migration to remove these and handle defaults in a different way.
Pull the docker image again to get it working.


docs: example apps for React & Next.js

Changes

Adds example apps for React & Next.js
Closes WID-31

@paolodamico I fixed most of the tests in this branch, but after merging with the main branch, they do not work again. Maybe I should send test fixes to the main branch?

@paolodamico I made a new clean pull request with sample apps: https://github.com/worldcoin/world-id-js/pull/39
I propose to close this request.


2024-03-07 (7 Mar 2024) - Morphir Project Meeting

Date
Thursday 07 Mar 2024 - 11am ET / 3pm UK
Untracked attendees

Name
Firm
Comment

Meeting notices

FINOS Project leads are responsible for observing the FINOS guidelines for running project meetings. Project maintainers can find additional resources in the FINOS Maintainers Cheatsheet.

All participants in FINOS project meetings are subject to the LF Antitrust Policy, the FINOS Community Code of Conduct and all other FINOS policies.

FINOS meetings involve participation by industry competitors, and it is the intention of FINOS and the Linux Foundation to conduct all of its activities in accordance with applicable antitrust and competition laws. It is therefore extremely important that attendees adhere to meeting agendas, and be aware of, and not participate in, any activities that are prohibited under applicable US state, federal or foreign antitrust and competition laws. Please contact<EMAIL_ADDRESS>with any questions.

FINOS project meetings may be recorded for use solely by the FINOS team for administration purposes. In very limited instances, and with explicit approval, recordings may be made more widely available.

Agenda

[ ] Convene, roll call, welcome new people (5mins)
[ ] Display FINOS Antitrust Policy summary slide
[ ] Review Meeting Notices (see above)
[ ] Approve previous meeting minutes - https://github.com/finos/morphir/issues?q=is%3Aissue+is%3Aclosed+label%3Ameeting
[ ] Add Items Here
[ ] AOB, Q&A & Adjourn (5mins)

Decisions Made

[ ] Decision 1
[ ] Decision 2
[ ] ...

Action Items

[ ] Action 1
[ ] Action 2
[ ] ...

Zoom info
Join Zoom Meeting

https://zoom.us/j/99313804465
Meeting ID: 993 1380 4465
Passcode: 038632

Dial-in

Find your local number: https://zoom.us/u/avGA76WoI

Github Repo: https://github.com/finos/morphir/
Mailing List: Email<EMAIL_ADDRESS>to subscribe to our mailing list

Damian Reeves / Capital One

Karl Moll / FINOS

Michelle Chan / Capital One

Nwokafor Choongsaeng / Morgan Stanley

Siddharth / Credit Suisse

Here is the survey for signing up for the NYC CDM workshop
https://www.surveymonkey.com/r/7QLNWMZ

Edward Peters/Capital One


storage: fsync writes

Codecov Report
Attention: 5 lines in your changes are missing coverage. Please review.

Comparison is base (bcbeed0) 47.10% compared to head (53c3529) 47.12%.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #206      +/-   ##
==========================================
+ Coverage   47.10%   47.12%   +0.01%
==========================================
  Files          87       87
  Lines        3821     3830       +9
==========================================
+ Hits         1800     1805       +5
- Misses       1705     1707       +2
- Partials      316      318       +2

Files
Coverage Δ

internal/storage/filesystem.go
47.82% <54.54%> (+1.15%)
:arrow_up:

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Add Coordinator field to JobSet spec

Part of #617. I plan on adding validation logic for this new field in a follow up PR, as well as more tests.

Add coordinator field to JobSet spec, allowing the user to define the ReplicatedJob, JobIndex, and PodIndex of the pod they wish to be the global coordinator.
Add a new label/annotation, jobset.sigs.k8s.io/coordinator, which will store the stable network endpoint where the coordinator pod can be reached, when the jobset.spec.coordinator field is defined.
Minor refactoring of test code so it is more flexible, allowing us to do a "set or merge" operation when adding labels/annotations to Job or Pod wrappers.

I have a small question: Does this feature require the use of featureGate?

I have a small question: Does this feature require the use of featureGate?

i don't think we'll add a feature gate for this one since it is isn't a risky feature. It just sets a new annotation and label on jobs and pods if the field is set.

cc @kannon92 would you ming taking a look at this if you have time? Abdullah is OOO currently.
Happy to discuss the feature further if you'd like.

/hold
/lgtm
Code looks good. I have some minor nits. Feel free to unhold.

@kannon92 can you reapply LGTM when you have a sec please? I made changes to address the comments and it was removed

/hold cancel
/lgtm


Core throughput: CTC 1090881

Implement:
1090881 - Given a core throughput dataset consisting of M clean frames (nominally 1024x1024) taken at different FSM positions, the CTC GSW shall estimate the pixel location and core throughput of each PSF.
NOTE: the list of M clean frames may be a subset of the frames collected during core throughput data collection, to allow for the removal of outliers.

There's some significant work done towards closing this issue in the ct branch. On the other hand, the current FDD for CT is going through a review that affects this req. and may change its text. It will be finalized after Jan 10th. I'll likely post the PR review once the new version is accepted. In the mean time, I'll clarify with the customers a couple of questions too.


<!--[if mso]> conditional comment breaks mj-raw

Describe the bug
When using an [if mso] conditional inside an mj-raw tag (mjml 4.3) the first closing mj-raw tag, mjml element that follows, and subsequent opening mj-raw tag (needed to hold the closing mso conditional tag) is treated as one comment. The mj-element doesn't render as expected.
Problematic snippet (large red square in screenshot):
<mj-raw><!--[if mso]></mj-raw>
        <mj-text font-size="20px" color="#F45E43" font-family="helvetica">Hello World</mj-text>
<mj-raw><![endif]--></mj-raw>

My try-it live:
https://mjml.io/try-it-live/BkLgDMSYN
Screenshots

Expected behavior
Something similar to the [if !mso] conditional comment where the opening comment allows the mj-text element to render. Mj-raw tags are also parsed correctly by the mjml engine leaving only the contents of the mj-raw tags outputted into the html.
<mj-raw><!--[if !mso]><!-- --></mj-raw>
<mj-text font-size="20px" color="#F45E43" font-family="helvetica">Hello World</mj-text>
<mj-raw><!--<![endif]--></mj-raw>

Additional context
Related issue 1561 I think:
https://github.com/mjmlio/mjml/issues/1561

I'm not sure if i understood completely but this might be fixed in 4.4.

On 5 Apr 2019, at 21:08, Brett Wilson<EMAIL_ADDRESS>wrote:
Describe the bug
When using an ) is treated as one comment. The mj-element doesn't render as expected.
Problematic snippet:
Bolded lines above are treated as a comment and not rendered by mjml 4.3.
My try-it live:
https://mjml.io/try-it-live/BkLgDMSYN
Screenshots
Expected behavior
Something similar to the [if !mso] conditional comment where the opening comment allows the mj-text element to render. Mj-raw tags are also parsed correctly by the mjml engine leaving only the contents of the mj-raw tags outputted into the html.
<!--[if !mso]>
Hello World

Additional context
Related issue 1561 I think:
#1561
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Thanks, some of my original post wasn't rendering correctly but I think I fixed everything.

Well I think this is not fixed at all in MJML4.4 the fix on ending tag has
been shipped in MJML4.3. That's a bit annoying and I honestly don't have
any idea on how to handle that properly might to check if we're able
somehow to hack htmlparser for that :(
On Fri, Apr 5, 2019 at 10:20 PM Brett Wilson<EMAIL_ADDRESS>wrote:

Thanks, some of my original post wasn't rendering correctly but I think I
fixed everything.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/mjmlio/mjml/issues/1565#issuecomment-480409669, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAizzWWws4ZkyjvBvflw-MKkNzEqQcgwks5vd7AWgaJpZM4cfjSF
.

@WilsonBrett  After further examination, it turns out that even if we do hacky stuff to parse conditional comments in a different way than regular comments, the mjml components you'll put between them probably won't work.
Most mjml components use conditional comments when rendering, and i suppose nesting conditionals wouldn't work well.
If you want a mj-text that only displays on outlook (or the opposite), the best way would be to create a custom component that extends MjText, and override the render function so that the whole output is in one single conditional

Ok thanks @kmcb777 for looking into and for the suggestion.

There is no real solution for this problem, it would require tricky hand-made parsing and the result still wouldn't work properly as it is. The best solution is a custom component, so i think we can close this


code coverage?

we have a testing code, but how get testing code coverage after we finish testing?

this post matches this project a lot<EMAIL_ADDRESS>


Computer lock key

Description of the new feature / enhancement
I have a Microsoft Bluetooth Keyboard.  Directly above the Backspace key is a dedicated key that locks the computer.  More often than not, I hit the computer lock key by mistake, and I have to reenter my passcode to unlock it.  It's annoying.
Scenario when this would be used?
I would like to use Keyboard Manager in PowerToys to disable the computer-lock key.  When you press the computer-lock key, I'm pretty sure it sends a Ctrl-L shortcut to the computer.  Is there a way to use Keyboard Manager to disable the key?
Supporting information
No response

Nope. Win+L is reserved by the OS. More chance of success by simply popping out that key.

Thanks.  I was afraid of something like that.  Definitely a poor key placement in a reasonably good product.
From: Jay @.>
Sent: Thursday, May 12, 2022 1:07 PM
To: microsoft/PowerToys @.>
Cc: DaleMWebb @.>; Author @.>
Subject: Re: [microsoft/PowerToys] Computer lock key (Issue #18246)
Nope. Win+L is reserved by the OS. More chance of success by simply popping out that key.
—
Reply to this email directly, view it on GitHub https://github.com/microsoft/PowerToys/issues/18246#issuecomment-1125330880 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AZER7L2KPSJFGDMVF72F4WDVJVJGXANCNFSM5VZFMDSA .
You are receiving this because you authored the thread.  https://github.com/notifications/beacon/AZER7L3MORNYDYVR5M44DV3VJVJGXA5CNFSM5VZFMDSKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOIMJS7QA.gif Message ID: @.*** @.***> >

yup, can't remap that key combo.  https://www.microsoft.com/en-us/download/details.aspx?id=102134 maaaay be able to do it.

@DaleMWebb Just got the same keyboard. Same issue. For me, though, it's prominent in that I have to use my IDE's go to definition shortcut which is set to F12, which in every keyboard I owned in the past was above the backspace key. And it's a key I use about 200 times in any given coding session and have been doing since I started coding 10 years ago. You can thus imagine how infuriating this keyboard is for me. Doesn't help that the app MS provides to work with this keyboard limits the configurable keys to only 6 of the entire layout - of which I need none (and even then the broken app won't let you modify them).
Anyway this is the closest thing to a makeshift solution I found. It'll leave you wanting, but I haven't found a better alternatives since MS dropped the ball on this one. https://stackoverflow.com/a/27975295/1018172.

Thank you for your information.


Can't give a code_block custom style

I'm able to override the code_block with the following code.
rules = {
    code_block: (node, children, parent, style) => {
        return <Text>I am a code_block</Text>;
    }
}

<Markdown rules={rules}>
    {myString}
</Markdown>

However, I can't give code_block a custom style (for example, a different text color) with similar code, even though I can modify the style of other components.
style = {
    root: { color: "red" }, // works on its own
    code_block: { color: "red" } // doesn't work on its own
}

<Markdown style={style}>
    {myString}
</Markdown>

Any thoughts?
I'm doing this because I'm setting the "root" text color to be white, but the background color of the code_block is close to white, so it's impossible to read. I'd like to either change the text color or the background color of the code_block, but I can't seem to do it.

This is because of bad / confusing naming of the rules and the styles
take a look at the readme and you will see that the style for code blocks is not the same name as the rules:
const copy = `
# Boom title

Some text

\`\`\`
  Code block
\`\`\`

\`inline code thing\`
`;

const styles = StyleSheet.create({
    root: { color: "red" },
    codeBlock: { color: "blue" }
});

I would like to clean this up, but because of backwards compat with react-native-markdown-renderer its really tough.

Thanks, I appreciate it.

just to let you know @keanfreeman i am sick of the difference between the styles and render rules so 6.0.0 just changes them all to match

Render Rule
Style(s)

body
body

heading1
heading1

heading2
heading2

heading3
heading3

heading4
heading4

heading5
heading5

heading6
heading6

hr
hr

strong
strong

em
em

s
s

blockquote
blockquote

bullet_list
bullet_list

ordered_list
ordered_list

list_item
list_item - This is a special case that contains a set of pseudo classes that don't align to the render rule: ordered_list_icon, ordered_list_content, bullet_list_icon, bullet_list_content

code_inline
code_inline

code_block
code_block

fence
fence

table
table

thead
thead

tbody
tbody

th
th

tr
tr

td
td

link
link

blocklink
blocklink

image
image

text
text

textgroup
textgroup

paragraph
paragraph

hardbreak
hardbreak

softbreak
softbreak

pre
pre

inline
inline

span
span

Seems like a good change!

Hi, I can get code_inline working, but not code_block:
Source:

Result:

My style sheet:
const styles = StyleSheet.create({
  heading1: {
    fontSize: 32,
    // backgroundColor: '#000000',
    color: '#E94B3CFF',
  },
  heading2: {
    fontSize: 24,
    color: '#E94B3CFF',
  },
  heading3: {
    fontSize: 18,
    color: '#E94B3CFF',
  },
  heading4: {
    fontSize: 16,
    color: '#E94B3CFF',
  },
  heading5: {
    fontSize: 13,
    color: '#E94B3CFF',
  },
  heading6: {
    fontSize: 11,
    color: '#E94B3CFF',
  },
  code_inline: {
    color: '#E94B3CFF',
    backgroundColor: '#2D2926FF',
  },
  code_block: {
    color: '#E94B3CFF',
    backgroundColor: '#2D2926FF',
  },
  blockquote: {
    color: '#E94B3CFF',
    backgroundColor: '#2D2926FF',
  },
});

@JosuGZ I got it working using fence instead of code_block:
 fence: {
    color: '#E94B3CFF',
    backgroundColor: '#2D2926FF',
  },

@JosuGZ I got it working using fence instead of code_block:
 fence: {
    color: '#E94B3CFF',
    backgroundColor: '#2D2926FF',
  },

Thanks, it works!
Aparently, this:

    Code here

is not the same than this:
```
Code here
```