Use C11 standard atomics

Hi,
I notice some of the code uses instructions like GCC's __atomic_fetch_add. This seems unnecessary; these built-ins are primarily for standard C library implementers to implement atomics. If the only reason the stats have to be atomic is to avoid problems with multiple threads writing at the same time, making the struct members of type atomic_size_t and incrementing just using, say, stats.free++ should suffice. I could send a pull request if you could help me get my head around this.

To be honest I have no idea why I used the __atomic intrinsics instead of the standard ones.  I think originally I had written the code as part of a separate project that is C99 so I didn't rely on <stdatomic.h.>  But yeah I have no objections to using the standard ones.


Error in ciftify_subject_fmri with  --func-ref option

File "${CODEPATH}/ciftify/bin/ciftify_subject_fmri.py", line 522, in define_func_3D
cifify.meants.verify_nifti_dimensions_match(settings.ref_vol.path,
NameError: name 'cifify' is not defined
After correcting the typo, running into below issue
ciftify.meants.verify_nifti_dimensions_match(settings.ref_vol.path,
AttributeError: 'Settings' object has no attribute 'ref_vol'
Here is the command that is used

ciftify_subject_fmri --surf-reg FS --ciftify-work-dir ${PROCESSDIR} --FLIRT-to-T1w  --func-ref ${DATAPATH}/fmri_map1.nii.gz fmri_mapAll.nii.gz SUBJECT mapname

Please assist if you see any issue in the usage
Thanks

Looks like a bug on my end.. will debug ASAP
On Sun, Jun 17, 2018 at 1:42 PM, NeuroImagingWorld<EMAIL_ADDRESS>
wrote:

File "${CODEPATH}/ciftify/bin/ciftify_subject_fmri.py", line 522, in
define_func_3D
cifify.meants.verify_nifti_dimensions_match(settings.ref_vol.path,
NameError: name 'cifify' is not defined
After correcting the typo, running into below issue
ciftify.meants.verify_nifti_dimensions_match(settings.ref_vol.path,
AttributeError: 'Settings' object has no attribute 'ref_vol'
Here is the command that is used
ciftify_subject_fmri --surf-reg FS --ciftify-work-dir ${PROCESSDIR}
--FLIRT-to-T1w --func-ref ${DATAPATH}/fmri_map1.nii.gz fmri_mapAll.nii.gz
SUBJECT mapname
Please assist if you see any issue in the usage
Thanks
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/edickie/ciftify/issues/65, or mute the thread
https://github.com/notifications/unsubscribe-auth/AMl59lX2X8gKXn5vo7Q1QBxHcxKGvcl-ks5t9pT4gaJpZM4Uq6Yv
.

Bug fixed. It should be fine with version 2.0.3


Sad times.

I have a bigger hammer, so let's see.

/benchmark plutus-benchmark:validation

Core looks promising for readKnown now (I haven't updated makeKnown yet, let's see how readKnown goes first).

On a serious note though, it does kinda make sense. The inlined version has all the constraints explicitly discharged while the non-inlined one doesn't (the class methods have those constraints).  I don't like that what's in here does not allow us to remove val from TypeScheme (it only needs to be in RuntimeScheme), however what I'm going to do in order to make deferred unlifting faster should solve the val-in-TypeScheme problem too. I'm not sure about the order of the PRs, though. I can work on deferred unlifting first or I can finish this one first, if you find what's in here reasonable.

I can work on deferred unlifting first

Yes, that's higher priority.
I also wonder if we should try inlining read/makeKnown into TypeScheme independently. That seems to have got us most of the speedup here, so I wonder if this PR is now a combination of an improvement and a regression, so the improvement masks the regression... perhaps better to just do the improvement independently.

/benchmark plutus-benchmark:validation

+0.9%. Sucks, I guess. Still gonna see if I can remove val from TypeScheme and ToBuiltinMeaning.

Closing in favor of #4533.


Saving and loading issues when transforms are attributes in a class

🐞 Bug

When a Transform is an attribute of a class and it saved inside it, an OSError: source code not available error is got when loading it.

Code:

@transform
def times_two(x):
    return x * 2

@transform
class MyClass:
    def __init__(self, a):
        self.a = a
    def __call__(self, x):
        return self.a + x

@transform
class Test:
    def __init__(self):
        self.t2 = MyClass(6)
        self.t2.pd_save('t2.padl', force_overwrite=True)
        times_two.pd_save('t4.padl', force_overwrite=True)

t4 = load('t4.padl') # This works
t2 = load('test2.padl')

Error:

@eramdiaz Could you update the above code so that it can be run to reproduce the error? It looks like there are a few typos.

Looking into this it looks like the issue results from saving a transform that includes in it's source code a pd_save call. So when calling self.t2.pd_save('t2.padl', force_overwrite=True) the resulting transforms.py file looks like
from padl import transform

@transform
class MyClass:
    def __init__(self, a):
        self.a = a
    def __call__(self, x):
        return self.a + x

@transform
def times_two(x):
    return x * 2

@transform
class Test:
    def __init__(self):
        self.t2 = MyClass(6)
        self.t2.pd_save('t2.padl', force_overwrite=True)
        times_two.pd_save('t4.padl', force_overwrite=True)

_pd_main = Test()

so when you try to load t2 = load('t2.padl') you again call pd_save which I think causes the issue.

@eramdiaz Could you explain the scenario in which you would like to call the pd_save function from within in a class or a transform?

This could though may have other problems, as it would not be possible to ensure that every source code will not have pd_save.
Perhaps, we should disable pd_save when a load is called? To ensure no overwriting is done is attempted until safely loaded.

At the moment there must be ways to overcome this problem, but if there are two Transform and one is only used or makes sense inside other, we could want to use this functionality. For example, if we had a tokenizer and a padding we might want to include the tokenizer call inside the padding call, and for that we need to save the first.

At the moment there must be ways to overcome this problem, but if there are two Transform and one is only used or makes sense inside other, we could want to use this functionality. For example, if we had a tokenizer and a padding we might want to include the tokenizer call inside the padding call, and for that we need to save the first. I consider this is a desirable behaviour.

Hmm I'm not sure I understand. Could you not either 1) have the tokenizer feed into the pad transform t = tokenizer >> padding or 2) you create a single class transform that does both tokenize_and_pad without inheriting transforms?
I understand that you could also make a new transform that inherits the transforms tokenize and padding, but then you would probably save the parent class, not the individual transforms inside right?

This could though create other problems, as it would not be possible to ensure that every source code will not have pd_save. Perhaps, we should disable pd_save when a load is called? To ensure no overwriting is done is attempted until safely loaded.

I agree that some sort of check or warning should be raised in this case. I'm not sure if disabling by itself is enough, because the loaded transform would not act in the same way as when the original transform was created.

@eramdiaz If you think it's worth solving this for Phase 2 please add it to the project board.

This could though create other problems, as it would not be possible to ensure that every source code will not have pd_save. Perhaps, we should disable pd_save when a load is called? To ensure no overwriting is done is attempted until safely loaded.

I agree that some sort of check or warning should be raised in this case. I'm not sure if disabling by itself is enough, because the loaded transform would not act in the same way as when the original transform was created which could be confusing.

yup, i agree, but just disabling pd_save inside load should not create an problem (unless there is certain functionality that relies on things written to disc). We can handle this with some kind of context manager.
with stage=save
     do things where 'pd_save' is an empty function

This will keep everything same after loading, but just for loading part make pd_save empty function

This is related to my following error when creating a saving and loading test:
def test_save_and_load_1(self, tmp_path):
    self.try_transform_1.pd_save(tmp_path / 'test.padl')
    t1 = padl.load(tmp_path / 'test.padl')
    assert t1.infer_apply(4) == 1

where I get the following error when loading the Transform

:
Appareantly, it is interpreting self to be the varname of the Transform since it is being called like self.try_transform_1.pd_save.

This is related to my following error when creating a saving and loading test:
def test_save_and_load_1(self, tmp_path):
    self.try_transform_1.pd_save(tmp_path / 'test.padl')
    t1 = padl.load(tmp_path / 'test.padl')
    assert t1.infer_apply(4) == 1

where I get the following error when loading the Transform  :
Appareantly, it is interpreting self to be the varname of the Transform since it is being called like self.try_transform_1.pd_save.

Could you post the complete code with minimal reproducible example?

@eramdiaz are the two things really the same? If not, please open another issue with the second problem with some minimal reproducible example.
As for the first issue. I think it comes from a misunderstanding of how the saving works: If a transform contains another transform as an attribute, the latter will automatically be saved, there is no need to explicitly do that. Especially calling save inside the init will cause a loop. We could perhaps document this better, but I don't there's anything that needs to be fixed.

closing this, @eramdiaz to check if the second problem still occurs and if so open a new issue


Search options combinations seems buggy

Describe the bug
I report here a few things regarding the dataset search feature, the combination of the search options between theme/organisations and keywords:
1/ the number of dataset is updated under theme/organisations when a keyword is entered -> this seems to work
But this number should also be updated when just clicking on the "Include draft datasets" checkbox.
Clarification: when just clicking on the "include draft" checkbox, then the number of datasets is not updated, but it should
2/ when a keyword is entered, and then a theme/organisations is selected -> what is happening ?
Because the keyword seems automatically deleted, but it seems that it is still used to filter the search results.
The correct behavior should be: keep the keyword in the field, show only the datasets that correspond to the keyword+selected theme/organisation.
Complement: when doing it the other way round it seems to work properly (first select a theme/organisation, and then enter a keyword)
3/ Another special case where I do not understand what is happening:
Steps to reproduce the behavior:

Go to 'https://test.visualize.admin.ch/en'
Enter "list" in the search field, and select "include drafts"
The categories show "1" for "National Economy"
Click on "National economy" and then, the results are empty:

Environment (please complete the following information):

Visualize environment and version: TEST v3.4.1 + https://visualize-ad-testingv3--dl6d2r.herokuapp.com/en
Browser and version: Firefox 97.0.1

LGTM! 👍


Neither authentication method work.

I have set up the the FLICKRAW_API_KEY and FLICKRAW_SHARED_SECRET environment variable, but authentication does not work either way.
(byebug) FlickRaw::Flickr.new
*** FlickRaw::FlickrAppNotConfigured Exception: No API key defined!

nil
(byebug) FlickRaw::Flickr.new ENV['FLICKRAW_API_KEY'], ENV['FLICKRAW_SHARED_SECRET']
*** ArgumentError Exception: wrong number of arguments (given 2, expected 0)

nil

Hi olav,
The documentation you see at the head of the master branch does not match the version of the gem you're using. I assume you're using 0.9.10? The documentation for that is here: https://github.com/hanklords/flickraw/tree/V0.9.10
BTW, we moved FlickRaw to the Flickr namespace a few months ago, and development continues there: https://rubygems.org/gems/flickr


fix f-string usage for Python 3.11 and earlier

Nested double quotes in f-strings are only supported in Python 3.12 and later.
Downstream bug: https://bugs.gentoo.org/943225

Thank you. Being on Arch, I keep doing this mistake.


Update existing Rich Schema HIPEs; Propose HIPEs for Rich Schema's Encoding, Mapping and CredDef

@brentzundel
Thank for review and suggestions! I believe I've applied all of them.


refactor: site enrich to send to placement queue

When a site is received its related placements should be sent to the
placement queue instead of performing the existing enrich on all
placements at once.
This allows individual placements to be processed instead of the site
triggering large batches of placements which can not easily be
processed.
TIS21-1673

Creating as draft due to dependency on #126

Would I need to pull the containers again in order to try it empirically locally with localstack?

I'm assuming the populateSiteDetails() is happening elsewhere now.

Everything will just get treated as a placement load.
So it will happen as part of enrichPlacementWithRelatedSite called by the top level placement enrich

Would I need to pull the containers again in order to try it empirically locally with localstack?

You'll need to pull the latest dev-handbook changes for the extra queue in localstack. Other than that only this sync project has changed, so no need to pull latest containers of any other services.


✅ [migration2viatot2.0]: Migrated SalesForce

Summary

Importance

Checklist

This PR:

[ ] follows the guidelines laid out in CONTRIBUTING.md
[ ] links relevant issue(s)
[x] adds/updates tests (if appropriate)
[x] adds/updates docstrings (if appropriate)
[ ] adds an entry in CHANGELOG.md

We are restarting the work on viadot 2 migration. @djagoda881 please, have a look at it!

@Diego-H-S Please rebase this branch with 2.0 because it has conflicts

I also don't have access, maybe another person should do that. @burzekj assign the PR also to @trymzet.

I'm guessing this can be closed since a duplicate was created https://github.com/dyvenia/viadot/pull/1036


Mention min clang format version and format all cpp and hpp files with clang-format version 9.0

fixes: #1207

Storage team has a major concern about you making ~300 lines of code change into storage sdk, and merging it without storage team's approval.


dict objects should not be a list

@pangeo-forge-bot uses Pydantic typing to validate meta.yamls (as of opening this PR, the code for this lives in the private repo pangeo-forge/registrar).
Typing for the meta.yaml recipes section looks like this
class MetaYaml(BaseModel):
    ...
    recipes: Union[List[RecipeObject], RecipeDictObject]
    ...

If dict objects are provided as a YAML list in meta.yaml (as currently indicated in ADR 2), this causes a validation error, because recipe dict objects should not be a list. This scenario was observed in https://github.com/pangeo-forge/staged-recipes/pull/31#issuecomment-1058307750.
This PR changes ADR 2 to indicate that dict objects should not be passed as a list, but rather a simple mapping.

@charlesbluca 👋 😄

You're welcome, @yuvipanda!

sorry, autocomplete fail haha


Update footer backgroud transperancy to 0.95 from 0.3

Note: It's recommended to give your branches descriptive names

Closed in favor of #11


add https://github.com/apptension/saas-boilerplate to our list of django templates

https://github.com/apptension/saas-boilerplate

Hey @undernewmanagement Can you assign this to me?


D110 printer works only with B1 device type.

D110 printer works only with B1 device type.
I think even the library itself detects this printer as B1 type.

Please post printer info (Gear icon near printer name). Maybe you have D110_M (300 dpi version of D110). It uses same print task format as B1.

Yes, you are right.
Printer info:
connectResult: 3
protocolVersion: 4
modelId: 2320
serial: G621120613
mac: 12:06:13:fc:10:02
charge: 4
autoShutdownTime: 3
labelType: 1
hardwareVersion: 0x0401
softwareVersion: 0x040a
Model metadata:
model: D110_M
id: 2320
dpi: 203
printDirection: left
printheadPixels: 120
paperTypes: 1,5
densityMin: 1
densityMax: 5
densityDefault: 3
Heartbeat data:
paperState: -1
rfidReadState: -1
lidClosed: false
powerLevel: 4

So it's expected behavior. A task name does not necessarily mean a certain model. This is just the first model I came across for this protocol.


Add getDatabaseSize() method to Sqlite-KV to retrieve size of database

I will be using this in my metrics

I have read the CLA Document and I hereby sign the CLA

The code looks fine, although I have to admit I'm not sure what this will be used for that we couldn't have used SqlStorage::getDatabaseSize() for. I assume that will make more sense after seeing the code that uses this, but want to call it out just in case you missed it.

The KV api layer operates on the DB object directly as opposed to going through SqlStorage, so need this method on the DB object itself.
We could remove the ones in SqlStorage and let it call db.getDatabaseSize(), I did not do that in this change

But what part of the KV API layer is going to be calling this code? That's what I'm missing.

SqliteKv is a small utility class that is intended only to implement the KV storage API on top of Sqlite. getDatabaseSize() doesn't seem to have anything to do with KV storage specifically, so it doesn't seem like it belongs here.
Also, the workers runtime (internal codebase) already tracks the current database size at all times through a different method (by intercepting and inspecting WAL writes), so obtaining it by executing a pragma seems unnecessary -- we can just use the value we already have.
Look for lastCommittedDbSize in the internal codebase.

But what part of the KV API layer is going to be calling this code? That's what I'm missing.

Here is how I plan to do the changes:
actor-state.c++ is where we emit the metrics (here). I want to emit the databaseSize here.
I plan to get the databaseSize returned from of getCache(..).get(key,value).
In the kv implementation of get(from above), I need the getDatabaseSize() method so I can return the value.

I think it could make sense to extend ActorStorageInterface with a getDatabaseSize() method, which the internal codebase can implement in terms of lastCommittedDbSize, but workerd could implement using the pragma. But that'd still happen in ActorSqlite, not SqliteKv.

I don't think the lastCommittedDbSize is the same what we get from the SQL query using pragma.  I believe lastCommittedDbSize is the size of the DB on disk, while the pragma-based calculation removes free pages from the calculation, so it should be strictly smaller.

We use auto-vacuum so I think free pages are eliminated immediately?

Decided to take a different approach for this task, the PR is no longer needed


Adds HttpRequestSampler and backports HttpSampler to use it

We recently introduced the HttpClientRequest and HttpServerRequest
types. These are used during propagation extraction, for example as a
parameter to Extractor.extract(request). This was to facilitate
secondary sampling, which occurs just prior to the primary sampling as
performed by HttpSampler.
One feedback was that it is intuitive to use types like
HttpRuleSampler uniformly across primary and secondary sampling.
This change introduces HttpRequestSampler which aligns types so that
rules can be defined with similar code regardless of primary or
secondary decision. This also takes care to not break api. In order to
reduce complexity around starting a span with old or new types, this
pushes some logic to adapters. Rationale is adapters are straightforward
to test and easier to reason with than other logic.
See https://github.com/openzipkin-contrib/zipkin-secondary-sampling/issues/8

This will need some more work, a lot of it done locally. ignore for now.

the code is done here, just it needs to be polished (test coverage itself). I'll ping when I have a corresponding pull request in the secondary sampling project that uses it.

This all works now, https://github.com/openzipkin-contrib/zipkin-secondary-sampling/pull/18
FYI: @marcingrzejszczak and @trustin @geminiKim @minwoox @ikhoon, I re-used terminology from sleuth as "rate" has caused confusion in the past when applied to a probability. This didn't break api or anything it just impacted naming of method parameters.

NEXT STEP: backfill unit tests

Thabks for the heads up. Let us rename rate to probability.

@marcingrzejszczak and @trustin @geminiKim @minwoox @ikhoon FYI: I moved the probability vs rate change to https://github.com/openzipkin/brave/pull/995 in order to reduce the heft of this PR

eventhough the other code was extracted, there's still some coverage issues here due to the increase of adapters.
NEXT STEP: backfill coverage so that it is as good or better than before


Show number of attempts

https://github.com/xenon4er/xa-ai-images/commit/b453a976d601cee7963bd164757a81c1587948ad


Android Multiple Flavor Dimensions

Android Multiple flavor dimensions seems to be not supported and document does not contain this as a limitation.

This is about the appcenter build service.The build service in appcenter is unable to discover android flavor dimensions properly and errors out. Thought a mention in the documents on the limitation would help people. More information about it can be found here: https://github.com/Naitbit/AppCenterGradleFlavors

Hi @navinilavarasan, thanks for brining this up. We will add this to the documentation.


Add key for reporting

We need to know if a public subscription was converted to the full set for the following reports:
% conversion from Public to Full Message set total / per month
% conversation from Public to Full Message set per weekly reminder (1/2/3/4).
I am assuming that if a public subscription is complete and then a full subscription is created that it is not a conversion. <- This was confirmed.

@rudigiesler we would be able to filter on the message set and get mother conversions and household conversions separately.


Is it possible to upgrade from cStor to Mayastor seamlessly?

Mayastor so efficient compared to cStor that I can't wait to put it in production environment, can anyone tell me if Mayastor v2 is mature enough for production? And if it's possible upgrade(switch) to Mayastor from cStor? Appreciate it!

Hi @kvzn, We have customers who are running Mayastor in production.
Coming to the second question, there is no upgrade path from cStor to Mayastor as both are completely different engines with different tech stacks.

@Abhinandan-Purkait Got it, thank you!

@kvzn Closing this PR, as this doesn't have any action items. Please feel free to reopen this in case you have any further queries or concerns.
Thank you for considering MayaStor.


Errors Shown in VSCode Problems panel for Non-Existent Files

Issue: diagnosticFilters does not get applied to files that are created/deleted
I discovered the issue while using Maestro. Maestro creates a ton of files under maestro/generated. However, the linter thinks these files are located under components/maestro/generated when the actual path is src/components/maestro/generated.
The files produce 1082 errors. If you click on any of the warnings you receive an editor error "Cannot open because file not found" because the path is incorrect

I think something similar may have been mentioned like this relating to erroneous errors/warnings after moving/renaming files?
The suggestion I remember was to try the 1.0.0 alpha, but I had issues trying that and the lang server kept crashing 🙁 (I haven't had time to debug that, sorry, just noting that I couldn't get that to work)

bsconfig.json
{
  "sourceMap": true,
  "stagingDir": "build",
  "retainStagingDir": true,
  "rootDir": "src",
  "files": [
    "manifest",
    "source/**/*.*",
    "images/**/*.*",
    "fonts/**/*.*",
    "meta/**/*.*",
    "components/**/*.*",
  ],
  "autoImportComponentScript": true,
  "createPackage": false,
  "diagnosticFilters": [
    "**/roku_modules/**",
    "**/maestro/**",
    "**/RALETrackerTask.*",
  ],
  "plugins": [
    "maestro-roku-bsc-plugin",
    "rooibos-roku",
    "roku-log-bsc-plugin",
    "@rokucommunity/bslint",
    "./build-scripts/nba-bsc-plugin/plugin.js"
  ],
  "maestro": {
    "excludeFilters": [
      "**/RALETrackerTask.*",
      "**/roku_modules/**/*",
      "**/node_modules/**/*",
      "**/rooibos/**/*"
    ],
    "buildForIDE": true,
    "extraValidation": {
      "doExtraValidation": true,
      "doExtraImportValidation": true,
      "excludeFilters": []
    }
  },
  "rooibos": {
    "isRecordingCodeCoverage": false,
    "testsFilePattern": null
  },
  "rokuLog": {
    "strip": false,
    "insertPkgPath": true,
    "removeComments": true
  },
  "logLevel": "error",
  "transpileOptions": {
    "removeParameterTypes": true
  }
}

Updated the original post with revised information and bsconfig


Add Hangers

Adds Hangers (a multiplayer word game) to builtwithelm!
🫶 TY!

@jawnyawns Thanks for sharing.


Unable to fetch twitter profile

After successful authorization it gives Unable to fetch twitter profile

This issue comes up occasionally...
It will disappear if retry the login process.
And here is the scopes,

{ scope: ['tweet.read', 'users.read', 'offline.access'] }

2024-01-05 02:02:08 | default | error | InternalOAuthError: Failed to fetch user profile
    at /.../...//node_modules/@superfaceai/passport-twitter-oauth2/dist/strategy.js:111:29
    at passBackControl (/.../...//node_modules/oauth/lib/oauth2.js:132:9)
    at IncomingMessage.<anonymous> (/.../...//node_modules/oauth/lib/oauth2.js:157:7)
    at IncomingMessage.emit (node:events:526:35)
    at IncomingMessage.emit (node:domain:488:12)
    at endReadableNT (node:internal/streams/readable:1589:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)

Okay, it may be a rate limit issue.
Logging the error,
  oauthError: {
    statusCode: 429,
    data: '{"title":"Too Many Requests","detail":"Too Many Requests","type":"about:blank","status":429}'
  }

Then how to dynamic change the skipUserProfile to false


Crash at start -SOLUTION- (not Visual C++)

My program crashed like #148 #147
after trying to upgrade it.
Deleting his folder or upgrading Visual C++ did not help.
Solution:
Deleted Settings folder in %LOCALAPPDATA%\TumblThree
cleaned log manually and saved
For developers! Thank you for you work !

Hello,
thanks for your feedback.
Do you still have backups of the settings folder? If so, you could copy the files one by one to the settings folder (make a backup of your current folder if needed) to see after which one the app no longer starts. Maybe that way we can find out what is the difference which causes these problems.


Fix edit page buttons

The "Edit This Page" button at the bottom of the page is broken. Currently, it uses /docs/editdocs#, e.g. https://kubernetes.io/docs/editdocs#docs/contribute/style/style-guide.md, but it should be like the one at the top of the page, e.g. https://github.com/kubernetes/website/edit/master/content/en/docs/contribute/style/style-guide.md
Also, the edit button at the top of the page has the en language hardcoded into the URL.

Preview: https://deploy-preview-10517--kubernetes-io-master-staging.netlify.com/docs/contribute/style/style-guide/

/lgtm
/approve


Introduce auth interceptor context for C#

To ensure future API stability with respect to #4154. (populating AuthInterceptorContext with new properties is an additive API change).

ping?

Sorry. I thought I had already merged this for some reason.


pipeline: keep shard files isolated when merging

Codecov Report

Merging #48 (64b6ade) into master (ab0bedd) will increase coverage by 0.89%.
The diff coverage is 13.33%.

@@            Coverage Diff             @@
##           master      #48      +/-   ##
==========================================
+ Coverage   39.73%   40.62%   +0.89%
==========================================
  Files          79       79
  Lines        3043     3035       -8
==========================================
+ Hits         1209     1233      +24
+ Misses       1582     1543      -39
- Partials      252      259       +7

Impacted Files
Coverage Δ

internal/pipeline/merging.go
17.55% <5.00%> (+2.59%)
:arrow_up:

internal/pipeline/file_receiver.go
34.88% <30.00%> (+2.83%)
:arrow_up:

internal/upload/sftp.go
62.62% <0.00%> (-2.43%)
:arrow_down:

internal/schedule/cutoff.go
90.69% <0.00%> (+53.48%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ab0bedd...64b6ade. Read the comment docs.


Dask doesn't work with pyarrow > 0.12.1

Code Example
I have dask 2.0.0 and pyarrow 0.13.0 installed. I got an error message saying that "Please install either fastparquet or pyarrow when reading a parquet file using dask. Downgrading pyarrow to 0.12.1 works.
import dask.dataframe as dd
dd.read_parquet('/path/to/parquet/file')

Working on this in https://github.com/dask/dask/pull/4995 I believe. I don't see a specific issue for this, so we'll leave this one open.

I don't believe there is any plan to support pyarrow 0.13.0, but 0.13.1 should work both now and after the refactor.  With that said, 0.13.1 will probably require you to build pyarrow from source until a pip/conda package becomes available.

@rjzamora Thank you! I won't bother to compile from the source code by myself. I will use 0.12.1 until 0.13.1 is available on PyPi.


example code calls getAccessStr() on a dict

Hi, following along with the example code in
https://github.com/basespace/basespace-python-sdk/blob/master/examples/2_AppTriggering.py#L77
before running this line, if I check the class of myReference it turns out to be a dict, so it does not have the getAccessStr() method defined on it and running the line above fails:
In [138]: myReference.__class__
Out[138]: dict

In [139]: print myReference.getAccessStr(scope='write')
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-139-62a91ddc162e> in <module>()
----> 1 print myReference.getAccessStr(scope='write')

AttributeError: 'dict' object has no attribute 'getAccessStr'

Has the API changed and the documentation not been updated? I am running version 0.3 of BaseSpacePy.
So what is the recommended way to get an access string in the latest version of the package?
Incidentally the same issue happens on the
Python SDK Overview page. There are other issues there (for example no object BSapi is ever defined, but it is used in the code example).

It seems the way around this is:
myReference = myAppSession.References[0]
projId = myReference.Content.get('Id')
# Then retrieve the project with:
proj = myAPI.getProjectById(projId)
# Now you can run the line that was causing problems before:
proj.getAccessStr(scope='write')

Perhaps the documentation and examples could be updated using this (or similar) approach?


What's new for .NET 5

[x] For SEO reasons, I would consider changing the title to What's new in .NET 5 and move evolution to the introduction. While very accurate, I don't think most customers will be searching on evolution. Searching for What's new in .NET 5 doesn't return this doc in the first couple of pages.

[ ] ~.NET 5 extends these characteristics, making incremental improvements:~
Container size optimizations -> That's not an incremental improvement AFAIK. Can you provide a link to this issue or otherwise provide more information?

[x] What .NET 5 is not - consider What's not included in .NET 5 - My guess is that will MT better and have better SEO
Consider changing text to:  NET 5 isn't a complete replacement for .NET Framework. - After all, we want customers to migrate from .NET Framework to .NET 5, and for many it is a replacement.

[x]  Web Forms replacement should be Blazor  or Razor Pages
There are no plans to port the following technologies from .NET Framework to .NET 5, but there are supported alternatives ~included in .NET 5~:
Consider changing column heading from Recommendation to something like Alternative, or recommended replacement, recommended alternative, etc.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f8175dfa-c812-afef-c018-a5927995ef43
Version Independent ID: 4fd72a98-c51f-cbd9-0578-d6448bab4749
Content: The evolution of .NET Core to .NET 5
Content Source: docs/core/dotnet-five.md
Product: dotnet-core
GitHub Login: @IEvangelist
Microsoft Alias: dapine

Thank you for the feedback @Rick-Anderson - much appreciated.

Hi @Rick-Anderson - curious about a few of your statements.

Container size optimizations -> That's not an incremental improvement AFAIK

How is that not incremental improvement? Reducing the size of a container with every release is absolutely incremental improvement. I'm leaving this as is.

@IEvangelist  thanks for accepting my other suggestions, as @BillWagner  stated, it's great work

How is that not incremental improvement? Reducing the size of a container with every release is absolutely incremental improvement. I'm leaving this as is.

Unless I'm not understanding the container size changes, (which is very possible), the layering approach is new to 5.0 and not an incremental change.  With layering, 99+% of the bytes from pulling the ASP.NET Core image is eliminated and 36MB of the SDK. That's 100MB improvement.
Looping in @mthalman the expert on layering.
I also made another request, provide more information than  the single bullet item

Container size optimizations

Unless I'm not understanding the container size changes, (which is very possible), the layering approach is new to 5.0 and not an incremental change. With layering, 99+% of the bytes from pulling the ASP.NET Core image is eliminated and 36MB of the SDK. That's 100MB improvement.

The strategy of layer sharing is not something new to 5.0.  This had been done in prior releases.  We just applied it to all of the SDK container images for 5.0.  There are a variety of historical reasons why it hadn't been done pre-5.0.  I would consider this to be an incremental improvement.

We still need more info.

Container size optimizations

Doesn't provide enough information.

@IEvangelist  please add a link to https://github.com/dotnet/dotnet-docker/issues/1814#issuecomment-625294750

Container size optimizations

by itself doesn't give enough information. The comments in the link tell the story.


snyc error

Hitting https://doclets.io/sync shows:
Err {"message":"Bad credentials","documentation_url":"https://developer.github.com/v3"}

I believe that happened, because of the following steps:

Login to doclets.io, granting access
Revoking the access for doclets.io on the github side (I did that because organization repos weren't showing up before knowing it was due to #50)
Logging in to doclets.io again, granting access again
Clicking the Re-Sync Repositories button.

thanks for reporting! sorry for the trouble. there is no work around yet...
working on this.

this is a duplicate of #47


[Proposal] Create a Group from Yup users

Yup is extracting the best of Farcaster, Lens, Mirror, NFTs, Twitter, and all of web3 in one curated feed.
Creating the Yup Group will enable to generate new Groups from Yup users.
Specifications
Expected Group: list of all addresses using Yup.
Tips on how to create the Group
The easiest way to extract the list of addresses is by using Yup API.
Have a look at Sismo documentation to understand how to generate a group.
Have fun!

Waiting for a response on the Yup data provider Q to do this one https://github.com/sismo-core/sismo-hub/issues/1433

The Yup knowledge base describes an API to extract all addresses - it doesn't exist in reality.
It appears there's no way to get "all" yup users, only a specific user.

Okay, thanks for the info!
Then, I'm closing this issue


Pin jupyter server proxy for windows

It fails to run on windows due to the jupyter server proxy issue, this is a fix to it.

thanks!


Add staging deployment action for the API

Description

This is a follow up to https://github.com/WordPress/openverse-infrastructure/pull/353 which fixes the deployment action for multiple containers.
This PR modifies the API deployment actions to:

Publish images to ghcr.io on every push to main (currently on every push but that's just for testing, I'll come back and fix that after I know it works)
Deploy to staging on ever push to main (same caveat as above)

Testing Instructions

I'm hoping to test this as part of this PR.
Checklist

[x] My pull request has a descriptive title (not a vague title like
Update index.md).
[x] My pull request targets the default branch of the repository (main) or
a parent feature branch.
[x] My commit messages follow best practices.
[x] My code follows the established code style of the repository.
[ ] I added or updated tests for the changes I made (if applicable).
[ ] I added or updated documentation (if applicable).
[ ] I tried running the project locally and verified that there are no visible
errors.

Developer Certificate of Origin

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

It works!! Woohoo! 🥳 I'll remove the test code so that it only deploys on commits to main


Fix error in watch mode

It looks like you forgot the rename the chunk property to chunks in the disconnect implementation which lead to build failures in watch mode for me.

Thanks


Validating the timestamp value during oauth

When OAuth occurs, the server must validate that the HMAC matches.
That's fine, but what about the timestamp? Shouldn't there be some validation to check that the timestamp parameter is within some certain tolerance threshold to avoid someone trying to reuse a valid HMAC (found in some logs for example)?

Can we please get a reply on this topic?
This seems like a potential security issue.

Hello? I was under the impression that Shopify paid a lot of attention to security?

@clement911 Thanks for raising this. We are looking into this!

@cquemin any updates?


Update the default behavior of zero_grad(set_to_none=None)

Now, the behavior of the wrapped optimizer is that the gradient is cleared by default when set_to_none=None. This aligns with torch.optim.Optimizer and saves memory.
What does this PR do?

Fixes # (issue)
Before submitting

[ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
[x] Did you read the contributor guideline,
Pull Request section?
[ ] Was this discussed/approved via a Github issue or the forum? Please add a link
to it if that's the case.
[ ] Did you make sure to update the documentation with your changes? Here are the
documentation guidelines, and
here are tips on formatting docstrings.
[ ] Did you write any new necessary tests?

Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

This is a proposed update for issue #2471

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@yongchanghao can you rebase from main to check if these failing tests pass? Thanks!

I have sync the code with the main branch.

Thanks!


Add materials for geo aggregation

Add materials for geo aggregations including the new polygon functionality when that is available.   Also point to any external developer relations information that we produce as extra learning material.

Actually these are in ft.search now which is better.


Support DataFrame.info()

Support DataFrame.info()

Related Issues
Fixes #1436

Please add some tests for added operand.

Please rebase master to solve the conflicts.


secure Memcached with TLS

Deployments of Memcached in cloud environments may need to encrypting its network traffic. We can implement SSL/TLS on top of the TCP protocol to achieve this goal. TLS can also help with authentication and ensuring the data integrity.

So good to see TLS getting done. Security compliance very often depends all traffics, even internal, to be encrypted.
Do you have a release date for this big feature?

No release date. I am finally getting time to properly review this this month though.
This might have two stages:

releasable but mixed/poor high end performance.
later optimized for the typical openssl lock scaling (maybe requiring something like amazon s2n) + interleaving handshake latency bubbles.

this is...  out in 1.5.13.


__hash__ TypeError in v0.12.1

Describe the bug
After upgrading from v0.12.0 to v0.12.1 my test suite fails with a lot of these errors:
self = Service({'id': None, 'created_at': None, 'updated_at': None, 'more sensitive fields')
name = 'sensitive', value = UUID('53cadcf0-cdce-4101-8218-22c96d3edd39')

    def __setattr__(self, name: str, value: Any) -> None:  # noqa CCR001
        """
        Overwrites setattr in pydantic parent as otherwise descriptors are not called.

        :param name: name of the attribute to set
        :type name: str
        :param value: value of the attribute to set
        :type value: Any
        :return: None
        :rtype: None
        """
>       prev_hash = hash(self)
E       TypeError: __hash__ method should return an integer

.venv/lib/python3.11/site-packages/ormar/models/newbasemodel.py:184: TypeError

I'm not sure, but I think every ormar model initialization fails with the same error. Any clue as to why? :slightly_smiling_face:

Can you provide a sample code to reproduce? Cause the whole tet suite pass on win and linux for python 3.7-3.10 😅

Looks like the issue goes away if I specify id on my models. The id field is auto-incrementing and defined on a base model like this:
class ModelBase(ormar.Model):
    id: int = ormar.Integer(autoincrement=True, primary_key=True)
    created_at: datetime = ormar.DateTime(timezone=True, server_default=func.now())
    updated_at: datetime = ormar.DateTime(timezone=True, server_default=func.now(), onupdate=func.now())

    class Meta:
        abstract = True

When I run MyModel.objects.create(...) without an explicit id it fails with the above error.
In the __setattr__ method, for prev_hash = hash(self), self is a model instance with id set as None. Might that be relevant?

Just adding this here in case that's enough to shed some light on the issue. Will try to dedicate some more time to creating a reproducible example later if needed 👍

Well, it's calculated as this:
    def __hash__(self) -> int:
        if getattr(self, "__cached_hash__", None) is not None:
            return self.__cached_hash__ or 0

        if self.pk is not None:
            ret = hash(str(self.pk) + self.__class__.__name__)
        else:
            vals = {
                k: v
                for k, v in self.__dict__.items()
                if k not in self.extract_related_names()
            }
            ret = hash(str(vals) + self.__class__.__name__)

        object.__setattr__(self, "__cached_hash__", ret)
        return ret

It might be the case that it does not play well in some cases with inheritance (although we have tests for that too, not sure if base classes have id's though, don't remember).

Ok, I'll come back with a full reproducible example when I get a chance 👍

I'm sorry @collerek, this is a false positive. After a little bit of digging, I found this 😓
class ModelBase(ormar.Model):
    id: int = ormar.Integer(autoincrement=True, primary_key=True)
    created_at: datetime = ormar.DateTime(timezone=True, server_default=func.now())
    updated_at: datetime = ormar.DateTime(timezone=True, server_default=func.now(), onupdate=func.now())

    class Meta:
        abstract = True

   # about 100 lines of code here

    def __hash__(self) -> int:  # <-- this is the culprit!
        return self.id

My bad. Thanks for the help!

No worries, glad that it's sorted out ☺


ShapeNetCore_4096.h5

Hello, thank you very much for your excellent work, can you please provide ShapeNetCore_4096.h5?

Hi, do you have access to ShapeNetCore_4096.h5. if so, please ask if you can share it, thank you very much.


Reexamine the "getting started" documentation

We recently had a user attempt to install RAIL, and theyy found the instructions quite complicated. We should revamp the instructions to make them more user friendly.

@drewoldag will sync with the user to get a concrete list of problem points.

Relatedly, the README currently tells users to get permission from the DESC PZ WG conveners to use RAIL for science, which seems counterproductive (and potentially in conflict with our license), so we should also take the opportunity to revise that to reflect the open nature of the code and limited applicability of the DESC Publication Policy to the broader community.

Adding some notes here while it's fresh in mind:

Drew and I have both run into problems with healpy following instructions as written; for me at least, it can be resolved with a conda install healpy before installing anything with pip
(TODO: adding a couple others after double checking them a bit, after lunch)

There seems to, in general, be a lag in updating pip vs conda for some packages. For my M1 mac I've had occasional issues with some/all of: pywavelets, healpy, fitsio, and jax/jaxlib when installed from pip.  So, I generally conda install these before I install pip install RAIL.  I think a few of these have now been "fixed" (i.e. the people managing the projects updated their pip distros to match the ones on conda).
This isn't really a bug, but it should probably be mentioned in the install instructions to aid users.  Because this could change at any time, maybe a phrasing more like:
We note that developers sometimes do not update their PyPI installers as regularly as they do conda.  If you see an install error on a particular package (for example, we've run into issues with pywavelets, healpy, jax/jaxlib, and fits), installing with conda rather than pip with e.g.
conda install -c conda-forge healpy
may solve the issue

Oh, and adding another point of confusion that I've seen: we had a user wanting to run Flexzboost, which has now been moved from the core RAIL package to its own package.  She did not see Flexzboost in RAIL, and wondered how to install it, or even if it had been removed.  So, we apparently need some clarification in the install instructions stating that BPZ, FlexZBoost, and Delight have been moved to their own packages, and can be pip installed separately.

I left some notes about what it took for me to go from a brand new, pristine, virtual environment to a completed run of goldenspike in this journal. https://docs.google.com/document/d/1HGv0UuT0khlXtAUJF2IIPabs_Gsr77AOw-tG0rXI7y4/edit#heading=h.fs1v8hq3cdc4
If anyone happens to pick up this ticket in the near future, they can use those instructions to help guide them.

As noted in today's tag-up, this ought to include some description of our workflow, e.g. merging PRs, tagging issues, etc. @eacharles Please feel free to add anything I missed from the call!

Today's tag-up brought up healpy installation issues: users may need to conda install healpy instead of pip (potentially just Mac M1, M2 users)


Decrease LFS size, gif -> webm.

This should reduce quite a bit the size of git LFS and loading the
napari website.
It converts a few of the gifs (or gifs depending on how you pronounce) to
webm.
Webm can be put in img tag in most recent browser and will behave like a
video. https://caniuse.com/webm current says 95.36% of global users,
97.64% if we restrict to desktop ones.
Conversion was done using:
ffmpeg -i {base}.gif -c vp9 -b:v 0 -crf 40 {base}.webm

With a quality of 40. You can see some color quantisation by looking
well, though personally on an old and slow machine this video is nicer
than gif as it is more fluid because of the smaller size I guess ?
Other advantage, I get a play/pause/scrubber with webm, I don't get with
Gif, so it's easier to replay and find when it loops.
I also add webm to .gitattributes so that all webm should be tracked by
LFS automatically.
Technically I'm not sure this actually reduce the LFS payload as the old
files are likely still there, but I'm going to assume git lfs is smart
enough sometime to just pull the ones from current commit (?) and/or
that we can purge later.
I'm also not super good at git lfs, so I'd love for someone else to try
this commit manually to see if I haven't screwed up anything.
$ git lfs ls-files -s  | grep tutori |grep -v png | cut -f2 -d'-' | sort
 docs/tutorials/assets/tutorials/dask1.gif (4.3 MB)
 docs/tutorials/assets/tutorials/dask1.webm (111 KB)
 docs/tutorials/assets/tutorials/dask2.gif (13 MB)
 docs/tutorials/assets/tutorials/dask2.webm (364 KB)
 docs/tutorials/assets/tutorials/painting.gif (3.6 MB)
 docs/tutorials/assets/tutorials/painting.webm (235 KB)
 docs/tutorials/assets/tutorials/point_annotator_demo.gif (18 MB)
 docs/tutorials/assets/tutorials/point_annotator_demo.webm (105 KB)
 docs/tutorials/assets/tutorials/tracks_isbi.gif (11 MB)
 docs/tutorials/assets/tutorials/tracks_isbi.webm (1.5 MB)
 docs/tutorials/assets/tutorials/viewer_layout.jpg (141 KB)
 docs/tutorials/assets/tutorials/viewer_pan_zoom.gif (3.7 MB)
 docs/tutorials/assets/tutorials/viewer_pan_zoom.webm (221 KB)

More things I don't know what I'm doing:
$ git lfs ls-files  | cut -f2 -d '*' | xargs  du -k | awk '{ sum += $1 } END { print sum }'
132936 (HEAD) vs 183528 (HEAD~1)

which I assume if KB, but du on mac is weird, so it looks like a non-negligible reduction, and the difference seem to match with the sum of the size of the above gifs...
It's also relatively easy to redo with a different compress ratio... and to extend to other gifs and 9 of the top 10 files in lfs are gifs:
$ git lfs ls-files  | cut -f2 -d '*'| xargs ls -lhS | head -n 10
-rw-r--r--  1 bussonniermatthias  staff    20M Mar  6 15:02 docs/images/pathology.gif
-rw-r--r--  1 bussonniermatthias  staff   8.6M Mar  6 15:01 docs/images/editing_points.gif
-rw-r--r--  1 bussonniermatthias  staff   6.8M Mar  6 15:01 docs/images/editing_shapes.gif
-rw-r--r--  1 bussonniermatthias  staff   6.4M Mar  6 15:01 docs/images/nD_vectors.gif
-rw-r--r--  1 bussonniermatthias  staff   6.0M Mar  6 15:01 docs/images/smFISH.gif
-rw-r--r--  1 bussonniermatthias  staff   5.2M Mar  6 15:01 docs/guides/images/mitosis.gif
-rw-r--r--  1 bussonniermatthias  staff   4.9M Mar  6 15:01 docs/images/shape_vertex_editing.gif
-rw-r--r--  1 bussonniermatthias  staff   4.8M Mar  6 15:01 docs/images/LLSM.gif
-rw-r--r--  1 bussonniermatthias  staff   4.6M Mar  6 15:01 docs/images/shape_resizing.gif
-rw-r--r--  1 bussonniermatthias  staff   3.5M Mar  6 15:01 docs/tutorials/assets/tutorials/launch_ipython.png

It's also a redo of https://github.com/napari/napari.github.io/pull/314

I'm going to save this until after release. We are currently above our lfs bandwidth, so I want to make sure the cached docs builds work during release and avoid actions that will invalidate the cache.

I'm going to save this until after release. We are currently above our lfs bandwidth, so I want to make sure the cached docs builds work during release and avoid actions that will invalidate the cache.

Sure, no objections and that make sens. One of my main questions is should I also do this on the rest of the GIFs of the repo ?

I'm working on shrinking the rest of the gifs in https://github.com/Carreau/napari/pull/new/shrinklfs2, it decreases the total size of lfs on HEAD from 164 Mb to 32Mb (not including the png crushing in the other PR)

#4208 reduces the size by an extra ~11 Mb to a 33Mb total size for head:
$git lfs ls-files  | cut -f2 -d '*' | xargs  du -kc | grep total
33600	total

That's awesome @Carreau! So to clarify, push is broken right now? It looks like GitHub has finally cut us off (we are actually at 690GB against a 450GB quota). Please keep your eyes peeled for other failures. Hopefully we can catch all the places that we are using lfs inadvertently now.

So to clarify, push is broken right now?

Yep, at least was yesterday.

(we are actually at 690GB against a 450GB quota)

Ouch. When is the reset ?
Is it worth considering a git-filter-branch to remove old pointers ? Or ask them to delete some old objects ?
Should we also look into making sure new PNGs/Gifs are webm'ed or crushed ?

@jni should we revisit now as 0.4.15 is out?

LFS bandwidth resets today, I don't see a reason not to merge this and #4208 ahead of moving away from LFS. It'll certainly give us a bit more breathing room to implement our new docs fix.


Error when using DGL_GCN

Hello, I got below errors when constructing DTI predicting model using GCN implemented in DeepPurpose (DGL)
Running Code
from DeepPurpose import utils, dataset
from DeepPurpose import DTI as models
from DeepPurpose.utils import *
from DeepPurpose.dataset import *
import warnings
warnings.filterwarnings("ignore")

drug_encoding = 'DGL_GCN'
target_encoding = 'Transformer'
X_drugs, X_targets, y= dataset.read_file_training_dataset_drug_target_pairs('./toy_data/DeepPurpose_Inhouse.txt')
y = y.astype(int)

train, val, test = data_process(X_drugs, X_targets, y,
                                drug_encoding, target_encoding,
                                split_method='random',frac=[0.4,0.3,0.3])

config = generate_config(drug_encoding = drug_encoding,
                         target_encoding = target_encoding,
                         train_epoch = 1,
                         LR = 0.001,
                         batch_size = 128,
                         result_folder = "./BI/GCN_Transformer_Inhouse/"
                        )

device = torch.device(f'cuda:{GPU_NUM}' if torch.cuda.is_available() else 'cpu')
torch.cuda.set_device(device) # change allocation of current GPU
model = models.model_initialize(**config)
model.train(train,val, test)

Error Message
Drug Target Interaction Prediction Mode...
in total: 34308 drug-target pairs
encoding drug...
unique drugs: 92
encoding protein...
unique target sequence: 381
splitting dataset...
Done.
Using backend: pytorch
Let's use 8 GPUs!
--- Data Preparation ---
--- Go for Training ---
Traceback (most recent call last):
  File "GCN_test.py", line 29, in <module>
    model.train(train,val, test)
  File "/mnt/hyojin0912/DeepPurpose-master/DeepPurpose/DTI.py", line 436, in train
    score = self.model(v_d, v_p)
  File "/mnt/hyojin0912/anaconda3/envs/DeepPurpose/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/mnt/hyojin0912/anaconda3/envs/DeepPurpose/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 168, in forward
    outputs = self.parallel_apply(replicas, inputs, kwargs)
  File "/mnt/hyojin0912/anaconda3/envs/DeepPurpose/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 178, in parallel_apply
    return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)])
  File "/mnt/hyojin0912/anaconda3/envs/DeepPurpose/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 86, in parallel_apply
    output.reraise()
  File "/mnt/hyojin0912/anaconda3/envs/DeepPurpose/lib/python3.6/site-packages/torch/_utils.py", line 434, in reraise
    raise exception
RuntimeError: Caught RuntimeError in replica 0 on device 0.
Original Traceback (most recent call last):
  File "/mnt/hyojin0912/anaconda3/envs/DeepPurpose/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 61, in _worker
    output = module(*input, **kwargs)
  File "/mnt/hyojin0912/anaconda3/envs/DeepPurpose/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/mnt/hyojin0912/DeepPurpose-master/DeepPurpose/DTI.py", line 52, in forward
    v_f = torch.cat((v_D, v_P), 1)
RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 128 but got size 2 for tensor number 1 in the list.

Sorry for the late reply. it seems there are bugs on multi-gpu for DGL based encoders. Could you try 1 GPU for now since your dataset looks small?


Support for more than one key per (chain-specific) keyring

KMS was designed to support a one-keyring-per-chain model, where the same key can exist in multiple keyrings if so desired. However, at the level of an individual chain, the "keyring" isn't of much use as only one key per keyring is supported:
https://github.com/tendermint/kms/blob/master/src/keyring.rs#L96
For proposed features like key rotation, it would be immensely helpful if multiple keys could be active at the same time:
https://github.com/cosmos/cosmos-sdk/pull/5233/files
An open question is what is responsible for determining which key is "active". The KMS could potentially decide this via some value easily changed at runtime, or Tendermint could potentially signal which key to attempt to use, and then fall back to an old one in the event the new one fails. This would require including a key ID / public key in messages like SignProposal/SignVote:
https://github.com/tendermint/kms/blob/master/src/session.rs#L162

An open question is what is responsible for determining which key is "active". The KMS could potentially decide this via some value easily changed at runtime, or Tendermint could potentially signal which key to attempt to use, and then fall back to an old one in the event the new one fails. This would require including a key ID / public key in messages like SignProposal/SignVote:

If I understood ADR 016 correctly, Tendermint will have no knowledge of the change, so this leaves us only one possibility - "some value easily changed at runtime".

Correct @melekes. Unless that is, of course, we modify Tendermint. To keep things as simple as possible (initially), I would consider a runtime configuration/signaling method.


Access to an image with Basic authorization header

Hi,
How can I access an image that is protected with the authorization header ? I saw a pull request that has never been merged with an addition to get MWPhoto through a NSURLRequest instead of NSURL which seems to me fine, but it has been created 2 years ago.

Hi I need to access an image that is protected with the authorization header. How did you achieve that?


Application Insights client-side implementation

Does anyone have any guidance or advice for implementing Application Insights on the client-side, I asked this question here https://github.com/aspnet/JavaScriptServices/issues/565 but I can't even make the basics work, it's down to my experience with Angular 2 I think.
I'm really after the browser, user agent, page tracking telemetry that you get with MVC etc.
I'm working on it so if I turn out something which works I'll gladly share here, had anyone beaten me to this ?

As far as I know, it is already working in this project and it is already configured.
This feature is working for server side code.
And Angular2 is working in client side.
It needs only some REST API to communicate with the server.
So, for those API calls, you will get Application Insights.
I hope you have your answer @johnkattenhorn

Please have a look in here-
https://docs.microsoft.com/en-us/azure/application-insights/app-insights-asp-net
You can find insight option for REST API here.

Sorry I didn't make myself very clear, I want to use Application Insights client side Angular 2. It seems the library in the documentation refers too is still Angular 1. I'll see if I can convert it easily enough.

Sorry for the interruption.
Can u please share the code you are referring to, please?
I am sorry for not finding or understanding the place or code you are referring to @johnkattenhorn

It's currently not implemented client-side in the template, only server side. I was just asking if there was anyone who had already implemented this so that we had appinsights both server side and client side.

As far as I know there is no such thing available for client side with angular.
If anything available in your knowledge, please share the link please so that we can integrate that.
Thanks @johnkattenhorn

Looking at the Application Insights example from Angular1.x, it looks like you need to do a few things, but we might be able to make it work.
Add the MS insights code onto the _layout.cshtml page
<!--
To collect end-user usage analytics about your application,
insert the following script into each page you want to track.
Place this code immediately before the closing </head> tag,
and before any other scripts. Your first data will appear
automatically in just a few seconds.
-->
<script type="text/javascript">
    var appInsights=window.appInsights||function(config){
    function i(config){t[config]=function(){var i=arguments;t.queue.push(function(){t[config].apply(t,i)})}}var t={config:config},u=document,e=window,o='script',s='AuthenticatedUserContext',h='start',c='stop',l='Track',a=l+'Event',v=l+'Page',y=u.createElement(o),r,f;y.src=config.url||'https://az416426.vo.msecnd.net/scripts/a/ai.0.js';u.getElementsByTagName(o)[0].parentNode.appendChild(y);try{t.cookie=u.cookie}catch(p){}for(t.queue=[],t.version='1.0',r=['Event','Exception','Metric','PageView','Trace','Dependency'];r.length;)i('track'+r.pop());return i('set'+s),i('clear'+s),i(h+a),i(c+a),i(h+v),i(c+v),i('flush'),config.disableExceptionTracking||(r='onerror',i('_'+r),f=e[r],e[r]=function(config,i,u,e,o){var s=f&&f(config,i,u,e,o);return s!==!0&&t['_'+r](config,i,u,e,o),s}),t
    }({});
    window.appInsights=appInsights;
</script>

Now we need a basic InsightService:
import { isBrowser } from 'angular2-universal';
import { Injectable } from '@angular/core';

@Injectable()
export class InsightLoggingService {
    logCustomEvent(fileName: string, methodName: string, eventName: string, msg: string, customData: {}, numericParams: {}): void {
        if (isBrowser) {
            (<any>window).appInsights.trackEvent(eventName, customData, numericParams);
        }
    };
}

Now add it to your app.common.module (so it's both in server&client automatically, but we're just ignoring the server rendered app build with isBrowser)
Then I'd imagine you could just use it like so, in a Component:
import { LoggingService } from './logging.service';
    @Component({
        // ....
    })
    export class SomeComponent {
        constructor(private loggingService: LoggingService) { }

        ngOnInit () {
            this.loggingService.logCustomEvent('SomeComponent', 'OnInit', 'Testing Angular2 - CustomEvent', 'Test Message', customData, numParams);
        }
    }

You can see here, we did everything manually, but you could also hook this into RoutingEvents (for example you could add it in here, on NavigationEnd), and just do all of this here, with the data generate through each Routing event. You could make this as simple or detailed as you'd like!

I'll be working on making an Ng-library here:
https://github.com/MarkPieszak/angular-application-insights
so it can just be npm install'd and added to your root NgModule that way.

That's awesome @MarkPieszak thanks! Feel bad now, given you more stuff to do :-)
I'd gladly test, give feedback whilst I get up to speed with Angular 2.

@johnkattenhorn Not a problem, it's all in good fun 👍
I'll definitely need help testing to make sure it works! I just updated the repo with extremely basic stuff, still nothing on npm but hopefully soon I'll have something you can mess around with and see if it works. It'll be easy to contribute to it as well (as it's really just a Service) so if something is off you'll be able to tweak it and we can publish the fix quickly :)

FYI, one of our dev's is actively working on this https://github.com/MarkPieszak/angular-application-insights and we will report back there the work we are doing.

Oh great! Working on that repo itself? Let me know what you guys need.

Yep, we will split at the work from our project and PR into the new repo once we have it working, making progress today on page tracking.

@johnkattenhorn @abrarjahin
https://github.com/MarkPieszak/angular-application-insights if you haven't seen it, we went 2.0 with the library, works with Angular 4+ and AoT now. Let me know over there if any issues come up for you guys!
🎈

Sure.
Thanks @MarkPieszak 🥇


Glu values mostly zero in PR-STEAM sequence processing and missing group correction in Quantify results

Hello,
I'm currently working on processing PR-STEAM sequences and I've noticed that most of the fitted Glu values are 0. I'm wondering if this is an issue with the parameters or if it's related to the basis set generated by the sim_steam.m function from FID-A, which doesn't appear to have been tested. Could this also be affecting the results?
Additionally, I've noticed that the OspreyQuantify results don't include any group correction results. Have I missed any settings or configurations?
I would really appreciate any feedback on these issues.
Thank you.

Hi @shaokunzhao,
The accuracy of the basis set has major impact on the quantification, especially for highly j-coupled metabolites. The sim_steam.m function uses ideal excitation and I'd advise you to use  sim_steam_shaped.m instead to generate fully-localized basis spectra.
Right - this only worked for GABA-edited MEGA-PRESS. I have just added this functionality for unedited data now.
Best,
Helge


Create a Rating Component

Design and implement a rating component that allows users to provide ratings or feedback.

Can you please assign this issue to me?? I would love to work on this issue

Assigned!

@Keerat666 I have created a PR on this kindly review it


MAINT: missed triang edge case.

@rgommers sorry, I missed an edge case for stats.triang
>>> from scipy.stats import triang
>>> triang.pdf(1, 1)
/Users/anz/miniconda3/envs/dev3/lib/python3.6/site-packages/ipykernel_launcher.py:9: RuntimeWarning: invalid value encountered in true_divide
  if __name__ == '__main__':
Out[58]:
array(nan)

I rectified that, and added more tests to cover the missed edge case (testing all pdf/cdf edge cases now).

This edge case was picked up when I rebased #8201.

LGTM, thanks Andrew.


Make use of the citation graph to improve labeling

Summary of Issue
The citation graph of papers is potentially powerful information that could be used to improve labeling accuracy.
Issue Type

[ ] Bug
[x] Enhancement

Description
The methods used in the paper, SPECTER: Document-level Representation Learning using Citation-informed Transformers from the Allen Institute could be used to improve labeling.
Look into using it using the code they provide.
Example
N/A
Environment
N/A

From Lucy: The citation graph could be useful later on but not right now


Wrong deprecation warning when using availability guard

Description
We are preparing our OSS packages to support visionOS warning free but are currently running into a misdiagnose of a deprecation warning. Following code shows an example of this
if #available(macOS 10.15, iOS 13, tvOS 13, watchOS 6, *) {
    print("We hit this branch")
} else {
    let trust: SecTrust? = nil
    SecTrustEvaluateAsync(trust!, DispatchQueue(label: "io.swiftnio.ssl.validationCallbackQueue")) { (_, result) in
    }
}

The above code will produce a warning 'SecTrustEvaluateAsync' was deprecated in xrOS 1.0: renamed to 'SecTrustEvaluateAsyncWithError(::_:)’. However, we are executing the if branch and not the else branch.
This is currently blocking us from properly supporting visionOS warning free.
rdar://111149445

cc @tshortli

@FranzBusch Swift version and target OS please?