403 ERROR

Greetings,
Thanks for the nice API ,
while i was trying to use it for a course project today it suddenly starts to give me
Status:403
hmmm, what's going on ?
btw i am only requesting animelists.
Thank you.

There were some problems with Jikan being blocked by MyAnimeList.net today. In such case you have the following options:

wait for the block to be invalidated
install Jikan on your own server so you are connecting to MAL from your own/private IP address

Installing Jikan (with docker or traditional way):
https://github.com/fethica/jikan-rest-docker
https://github.com/jikan-me/jikan-rest

Hi @AliD201 ,
We had some issues with the API earlier today. It should be working now though.
Can you confirm whether you're facing the same issue again?

Hi @AliD201 ,
We had some issues with the API earlier today. It should be working now though.
Can you confirm whether you're facing the same issue again?

Yes now it's working!
Thank you.

@irfan-dahir
it seems the problem is still on, as if some requests goes and some others get blocked.

@AliD201 I can't seem to reproduce the issue on my end. I've tested the servers' connectivity with MAL and they're not showing any issues.
Can you provide some request/response logs?

Closing due to inactivity. Feel free to open a separate issue if the problem occurs again.

Hi!
First thank you for your continuous work, on the other hand comment that this error is being replicated in many calls:
GET: https://api.jikan.moe/v3/schedule/monday
{   "status": 403,   "type": "BadResponseException",   "message": "Something went wrong, please try again later",   "error": "403 on https://myanimelist.net/anime/season/schedule" }
GET: https://api.jikan.moe/v3/anime/9253/videos
  "status": 403,   "type": "BadResponseException",   "message": "Something went wrong, please try again later",   "error": "403 on https://myanimelist.net/anime/9253/_/video" }
These two calls are some examples, but there are many more with this problem. Thank you so much for everything
Regards!

Hello @Jeluchu ,
MAL actually blocked every cloud provider/VPN/etc IP. This has happened before, we're communicating with MAL on the duration of this block. Last time it lasted for about 8 hours.
This is the reason any uncached requests on Jikan's servers return 403 as Jikan is unable to connect to MAL.
Additionally we're also working on a long term fix (less depending on MAL) which should be out by later this month of July.
Thanks for your patience. You can check out the #status channel on our discord for updates.

Thank you very much for your quick response @irfan-dahir
I did not want to open a new issue because I saw that it had previously happened, but I still wanted to report it so that you were aware

Thank you for reporting either way. Meanwhile I've re-opened and pinned this issue for anyone else that would like to follow. Will update the issue as the situation develops.

Hi, so here's a bit of an update.
MyAnimeList is still blocking cloud providers so Jikan is still unable to fetch the respective data. According to them, they're still looking into a solution.
This message on discord details what Jikan is doing to continue the service during this block: https://discordapp.com/channels/460491088004907029/461020417998454785/719365075172851732
tl;dr we're indexing cache into a database format to generate responses for endpoints.
The following endpoints have currently been restored
✅ Anime
✅ Manga
✅ Anime Search
✅ Manga Search
✅ Top Anime & Manga
✅ Anime & Manga Genre
✅ Producers
✅ Magazines
✅ Anime Schedule
All these endpoints are currently possible by looking up locally indexed Anime & Manga that we have salvaged based on existing local cache.
And unfortunately, we are still lacking Characters & People entries at the moment as this is all that could be salvaged from existing cache. We're looking into what can be done for the missing entries.
We're currently working on restoring the seasonal, people, characters, and all search endpoints.
Known Issues
Note: There may be some issues since we're working on local data. Here are some known ones.
⚠Anime & Manga endpoints excludes detailed data from sub-endpoints such as characters, staff, stats, pictures etc. Which is why it's not working at the moment.
@JanPlazovnik
ℹ While advanced search filters have been restored although some filters may not work due to the way caching has been done up till now. These include the following filters letter, genre.
ℹ While Top endpoint filters have been restored although some filters may not work due to the way caching has been done up till now. These include the following filters bypopularity, favorite. Anime/Manga type filters work fine.
ℹ You may still face 403s until a missing resource has been populated from existing cache data.
Or cache for that particular resource may not exist (we're also working on making uncached data available).
e.g https://api.jikan.moe/v3/anime/2 - This anime does not exist however since this request attempts to connect with MAL, it returns a 403 - where-as the correct response should be a 404. We're working on it.
ℹ There may be some missing entries or outdated information. We're working on updating these.
Thanks for your continued patience. I'll keep this thread updated with the information available in #status over on our Discord channel.

hello! season endpoint is functioning for me right now, thank you for the good work!!

Update:
✅ Seasonal endpoint has been completely restored (/season, /season/{year}/{season}, /season/later, /season/archive)

Jikan can connect to MAL again, looks like we're unblocked.
Closing issue for now.

I can't access the API from my ip. When I'm trying to access with VPN, it's all wokring fine.


CloudWatch dashboard widgets

🔖 Summary
This plugin idea is to bring AWS monitoring capabilities to Backstage. Idea is not to replace existing monitoring but instead give a fast overlook about how things are running.
Components (or why not other entities) could add some annotation / other metadata that would determine which CloudWatch widgets should be shown in the component page. As components may not use same AWS services, these should be configurable per component. Cross-account calls should of course be supported.
🌐 Project website (if applicable)
Example implementation https://github.com/aws-samples/aws-cloudwatch-building-dashboard-outside-aws-console
✌ Context
No response
👀 Have you spent some time to check if this plugin request has been raised before?

[X] I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

[X] I have read the Code of Conduct

Are you willing to submit PR?
None

Any progress on this?

Contributions welcome! 🚀


Python Version Specifics - better aligning prereqs for all mlflow docs.

There seems to be a lot of missing prereqs for the MLFlow and Databricks workflows. As there are lots of overlap in tools and options it might be good to build a matrix for current supported python/databricks/conda configurations?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d6910eb2-9a76-916c-133c-6d1b37cd077d
Version Independent ID: adb35769-d52c-1294-872a-78cf9ef538ec
Content: MLflow Tracking for ML experiments - Azure Machine Learning
Content Source: articles/machine-learning/how-to-use-mlflow.md
Service: machine-learning
Sub-service: core
GitHub Login: @shivp950
Microsoft Alias: shipatel

@timmyreilly Thank you for your feedback. We will investigate and update the thread further.

@shivp950 Hello Shivani, could you please check on this issue and update the document? Thanks.

@timmyreilly Hello Tim, I will forward this issue to author for more details. Thanks.
We will now proceed to close this thread. If there are further questions regarding this matter, please respond here and @YutongTie-MSFT and we will gladly continue the discussion.


Apps should provide an icon for usage in small, rectengular spaces

Every app in a Giant Swarm catalog can provide an icon URL. Icons allows users to recognize and distinguish an items quicker and easier in a list of many items.
So far we have set app icons in a way that was optimized for the use in happa's app catalog list views. Example:

Examples

These icons often work best in a rectangular-shaped container that have a width:height ratio of 2:1.
However, so far we don't have a way to provide an icon which is guaranteed to work well in a space-constrained square shape container. The following example illustrates such a use case.
Example of a list of app without icons:

Example of a similar list of app with icons:

In our app metadata, we are currently limited to 1 icon URL in the Helm chart (example).
TODO

[x] Communicate change of spec internally:

[x] intranet PR
[x] announcement

[ ] Adapt most apps we expect to be installed in WC (default apps, preinstalled, Giant Swarm catalog)
[x] Display app logos for catalogs in the web UI if the logo is provided

Some options we have:

Always use the one icon metadata property to specify the icon that fits in a square space. (As a result, we would loose the more expressive logos that work well in a larger space.)
Define an additional icon annotations for the new purpose
Define some implicit lookup logic. E. g. if the icon URL ends in logo.svg, we also check for an icon with the URL ending in logo_icon.svg.
Use SVG fragment identifiers. With this option, all available icon formats would be included in the same SVG file. We would define a set of views with standard names. A view is basically a rectengular part of an SVG's content. For this to work, all icon files would have to implement all views.
For each of our apps, have additional icons hard-coded into happa

Would be great to get some input from Honey Badger on this @gianfranco-l .

left a note on @giantswarm/team-honeybadger board to provide input on this

Hmm... I prefer explicit solutions, so if you really want to implement this, I would introduce a new annotation (Chart.yamls allow for arbitrary annotations and AFAIR we're automatically copying them from chart's metadata to ACE CRs).

I would go with the logo meta data in Chart.yaml. It seems to be standard way and will work with custom apps created by users for private catalogs.

Based on your feedback, and with some distance, I've been re-thinking this. How about this combined solution:

We use the icon metadata field in Chart.yaml for a square icon. Because "icons" in the classical sense were exactly that: little square images.
We introduce a new, optional annotation for an additional catalog image URL. This one is supposed to work well in the landscape image size ratio.

Example
Icon:

Catalog image:

We just discussed the above proposal (https://github.com/giantswarm/roadmap/issues/1248#issuecomment-1208201606) in Rainbow and decided to pursue it this way. There will be an intranet update where other teams are invited to review. After that is merged, we'll start adapting the most important apps.

Intranet documentation for providing icons for apps in happa has been updated. We think the task of updating icons for apps could be a good task for Team Rainbow's working student, and the appropriate label has been added to this issue.

Notes

in web-assets, we want to use a naming cnovention to distinguish icon and logo. Suggestion: e. g. logo_dark.svg and icon_dark.svg
We reference the dark version in Chart.yaml
Based on Grafana app usage data we add icons where they are missing, starting with the most used apps

Ideally we would also target all cluster-PROVIDER and default-apps-PROVIDER app repos and add an icon. They are important and would look strange without one.

They are on my list and some already got their icon :+1:

FYI @mogottsch Xavier (@vxav) wrote me that he is working on adding the icon to Cloud Director apps.

@vxav Yes, looks good!

closing this issue


Removed inclusion of unit definitions

@gtfierro, assuming this doesn't break something else I'm unaware of, this change reduces the line count of Brick.ttl from 52,044 to 43,490 by suppressing the (redundant) definition of all the QUDT units, since they are now imported.

It seems the checks do indeed notice the absence of the unit definitions. The question is whether we can get the system to see the imports where such things are defined.

Hi @steveraysteveray -- I just pushed a fix to this branch that uses the ontoenv library to import the dependencies into an RDFlib Graph (see tests/test_examples.py). This fixes some of the tests--I'll take a look at the others soon but it should be the same fix.
One issue I'm getting is qudt:Unit-supercededBy is used as the object of sh:property but is never actually defined anywhere. The pySHACL library doesn't like this, so validation fails on graphs that import QUDT because pySHACL wants to know that qudt:Unit-supercededBy is actually a PropertyShape. I believe this should be inferred from sh:property, but I need to double check how the standard is worded

Thanks! I assume the same ontoenv snippet in analyze.py will fix the submeter test failure.
Meanwhile, I'll remove the definition of qudt:Unit-supercededBy. We are using dcterms:isReplacedBy instead, and never cleaned up.

Thanks! I assume the same ontoenv snippet in analyze.py will fix the submeter test failure.

It should! If you change the definitions of any of the imported ontologies, remember to run make (to update the imports folder)

Also, got the following warning when running "pytest analyze.py"
/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/brickschema/graph.py:231: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
logger.warn(report)

This shouldn't affect any of the tests -- it's something I can address in another PR but it is really just as simple as adding ing onto the warn() method calls

@gtfierro, I think this PR is good now. The units do not explicitly show up in Brick.ttl.
There is one minor side-effect that has come up because of some refactoring in the QUDT repo. Namely, all Currency units have been moved to a separate graph. They are still instances of qudt:Unit. However, if you do not choose to import the Currency units, then we get a validation error caused by qudqk:Currency referring to a bunch of units that are unknown. If you are OK with importing Currency units, there's no problem, but I'd like to hear your thoughts about this as a sample user of QUDT. If you really don't want to include Currency units, QUDT.org may reconsider how we package the graphs.

@gtfierro, I think this PR is good now. The units do not explicitly show up in Brick.ttl.
There is one minor side-effect that has come up because of some refactoring in the QUDT repo. Namely, all Currency units have been moved to a separate graph. They are still instances of qudt:Unit. However, if you do not choose to import the Currency units, then we get a validation error caused by qudqk:Currency referring to a bunch of units that are unknown. If you are OK with importing Currency units, there's no problem, but I'd like to hear your thoughts about this as a sample user of QUDT. If you really don't want to include Currency units, QUDT.org may reconsider how we package the graphs.

I'm not strictly against including the Currency graph -- it might be helpful for modeling tariff structures for example. It should be ok to include it for now, but it does raise the question of how to bound the size of these graphs. I wonder if there is some sort of tree shaking procedure we could use to remove unused parts of the graph (esp when we take imports into account)

OK, great. In that case, merging #495 includes the Currency graph, and includes this merge.
I'll think about "tree shaking"...


BUG Issue 训练时加入 dropout gpu内存不断增加

环境
1.系统环境：ubuntu 18.0.4
2.MegEngine版本：0.3.1
3.python版本：3.7.6
使用gpu训练mnist数据集 Lenet时，占用gpu内存不断增加，直至gpu内存溢出（4g内存，batch 128）
请提供关键的代码片段便于追查问题
示例训练代码 gpu训练mnist数据集 Lenet时 加入两层dropout
去除drop 不会出问题
class LeNet(M.Module):
    def __init__(self):
        super(LeNet, self).__init__()
        # 单信道图片, 两层  5x5 卷积 + ReLU + 池化
        self.conv1 = M.Conv2d(1, 6, 5)
        self.relu1 = M.ReLU()
        self.pool1 = M.MaxPool2d(2, 2)
        self.conv2 = M.Conv2d(6, 16, 5)
        self.relu2 = M.ReLU()
        self.pool2 = M.MaxPool2d(2, 2)
        # 两层全连接 + ReLU
        self.fc1 = M.Linear(16 * 5 * 5, 120)
        self.relu3 = M.ReLU()
        self.drop1 = M.Dropout(0.5)
        self.fc2 = M.Linear(120, 84)
        self.relu4 = M.ReLU()
        self.drop2 = M.Dropout(0.5)
        # 分类器
        self.classifer = M.Linear(84, 10)

    def forward(self, x):
        x = self.pool1(self.relu1(self.conv1(x)))
        x = self.pool2(self.relu2(self.conv2(x)))
        # F.flatten 将原本形状为 (N, C, H, W) 的张量x从第一个维度（即C）开始拉平成一个维度，
        # 得到的新张量形状为 (N, C*H*W) 。 等价于 reshape 操作： x = x.reshape(x.shape[0], -1)
        x = F.flatten(x, 1)
        x = self.relu3(self.fc1(x))
        x = self.drop1(x)
        x = self.relu4(self.fc2(x))
        x = self.drop2(x)
        x = self.classifer(x)
        return x

请提供完整的日志及报错信息
egBrainError: MegBrain core throws exception: mgb::MegDNNError
curand call failed: status=102(CURAND_STATUS_ALLOCATION_FAILED) call=curandGenerateUniform(m_curand_handle.gen(), dst.ptr<dt_float32>(), dst.layout.total_nr_elems())

您好，该问题是本次版本的已知问题 #5，我们会尽快修复这个问题。您可以暂时使用静态图来进行训练，非常感谢您对 MegEngine 项目的关注：）

TESTIMONY ON HOW I BECAME INSTANT MILLIONAIRE WITH THE HELP OF THE ILLUMINATI. TELEGRAM +4915219211439 Or EMAIL THE COUNCIL AT<EMAIL_ADDRESS>)
Hello everyone, i have to write this testimony because of people out there who also will like to join the Illuminati, because i am really happy to be member of Illuminati cult society today, i have been searching for the right hand to join this organization of Illuminati i finally got this nice Prof.Monday and Barr. Mikhail ,who help me in joining even when i was asked to buy all the items my self which i did for the initiation. though it cost me some money, but every thing went out successfully, and an instant sum of 1,000,000,000.00 US dollars was given to me with a free home of my choice in CALIFORNIA USA. And also, power, influence, good position fame including $300,000,000.00 dollars monthly as a salary for me. If you are interested contact the Illuminati full membership council through this email address above . Today my family and i are  happy for following the Light. See you there. You may WHATSAPP CHAT HELPING MEMBERS  at +13252429274 for special information on how to be a full member


Image Import Issue

Adding this to provide more info once gathered.
Basically, image renditions from AEM are not accurately importing into Franklin. This is seen most prevalently in galleries, where a horizontal image that fills the slide is importing as a vertical image that does not fill the slide. It is also seen less so in listicles and general articles wherein an image that has a large original size that is renditioned down to fit the page/component, is importing as the lesser quality image and size in the Franklin page. This is leading to degradation in the image quality as well as messing with the desired gallery display.
Examples:

Updated based on 2/14/23 call:

Henry to remove the rend portion of the image URL
Henry to adjust import so that it is pulling in OG image
Henry to re-run the import for all galleries/listicles

@icaraps I am not sure if this ticket is valid. We are importing original images now. And the particular document mentioned in this ticket now fails to upload to GD due to file size. This is a separate issue we need to address by referencing images instead of storing them in the document

Closing this ticket. The following has been done a while ago.

Henry to remove the rend portion of the image URL
Henry to adjust import so that it is pulling in OG image
Henry to re-run the import for all galleries/listicles


Third SqlTypeModifier outputs instance name in createTable

Adding 3 SqlTypeModifiers to a createTable call results in it outputting it's instance name instead of properly appending it's text to the query:
        db.createTable("tools", true,
                "mac" to TEXT + PRIMARY_KEY + UNIQUE,
                "type" to TEXT,
                "lastAdvertisement" to TEXT)

Error Code : 1 (SQLITE_ERROR)
Caused By : SQL(query) error or missing database.
(near "org": syntax error (code 1): , while compiling: CREATE TABLE IF NOT EXISTS tools(mac TEXT PRIMARY KEY org.jetbrains.anko.db.SqlTypeModifierImpl@82ed13f, type TEXT, lastAdvertisement TEXT);)

Turns out to have been a pretty simple oversight and a quick fix:
#374

Thank you for the contribution!

I think this isn't fixed in the 1.10.1 source file (nor built through Gradle).
The master is fixed, but the changes are not included in the release somehow

This is true, just a small comment that the version is 0.10.0 that works and 0.10.1 doesn't have the fix.

Fixed with a9fb4771c4ac8a211839bdb9174bf967595a36aa


Sync Updates

The two options given to sync custom profile properties are a few years old. You may want to add a link to this newer method, https://docs.microsoft.com/en-us/sharepoint/dev/solution-guidance/bulk-user-profile-update-api-for-sharepoint-online#call-the-import-api-from-powershell

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 089de41e-64c6-4698-5a2e-deafe6fac8ea
Version Independent ID: 99812440-95be-2284-4b90-3088f55b3772
Content: Add and edit user profile properties in SharePoint Online
Content Source: SharePoint/SharePointOnline/add-and-edit-user-profile-properties.md
Service: sharepoint-online
GitHub Login: @kaarins
Microsoft Alias: kaarins

@kaarins Any updates on this issue? Thanks!

This change was published immediately, and I just missed closing the issue. :)


Fix KerbinSideRemasteredTLA install location

Fixup of #8636
This mod belongs into GameData/ContractPacks/, as it is in the zip. Also the ships belong where they are in the zip, they are part of the contracts, not user-playable ships: craftURL = "ContractPacks/KerbinSideRemasteredTLA/Assets/Ships/" + @/wreckCraft.
ckan compat add 1.8

Oops, sorry about that, and thanks for fixing it!


Add mention icon

Closes #155

@TedGoas No I added it in because I thought I hadn't done it yet. I guess I had though and forgot to close the open GH issue.


actions: Use GITHUB_TOKEN and other fixes

Use GITHUB_TOKEN for ghcr.io pushing (packages GA!!)
Grammar: image -> image(s)
Add more caching
Make the devcontainer support ARM64
Make the devcontainer finally pull from ghcr.io
Add nano to the devcontainer so I don't lose my mind

There's no easy way to do a test run of some of these things like pushing (Thankfully the test builds are okay), so just sanity check the yaml changes and let's see what happens.

Pushing in this way seems to work with a different project under the same conditions: https://github.com/Erisa/Cliptok/runs/2890527795
So hopefully should be safe to do.

Actually, an afterthought - could the caching added be enough to close #83?

Actually, an afterthought - could the caching added be enough to close #83?

All I did here was enable workflow caches in some places where they weren't for some reason. It would still be a good idea to optimise the use of cache with those bundler layer changes. Need to get around to looking at it sometime.


Add support for suppressionURL (with default)

The issue
Currently false positives will only be updated/removed by newer releases of this plugin.
Describe the solution you'd like
It would be nice if the plugin would automatically try to fetch the latest suppression file from github or a different configurable source (proxy). This could be achieved by adding and using a suppressionURL (s) property that defaults to https://raw.githubusercontent.com/jeremylong/DependencyCheck/master/core/src/main/resources/dependencycheck-base-suppression.xml (?)
Describe alternatives you've considered
Manually share the suppression file with all developers (already possible but error-prone).

Pretty sure this is supported already.  Just put a full URL into the suppression file path.  Probably needs to be documented better.
For another option see https://github.com/jeremylong/DependencyCheck/issues/1878

I wasn't aware, that you can refer to URLs there already.

Probably needs to be documented better.

I would appreciate that. Thanks for your help.

Please also update the javadocs for the suppresionFIle/suppressionFiles properties in BaseDependencyCheckMojo as they will be converted to the help that will be displayed in IDEs.


Set isRunning of Process to false before calling the termination handler

Cherry-picks https://github.com/apple/swift-corelibs-foundation/pull/3000 to release/5.5.

Otherwise, we end up in a race condition where isRunning might still be true when accessed from inside the terminationHandler. If this is the case, accessing terminationReason or terminationStatus inside the termination handler crashes because their precondition that the task has finished are not satisfied.
Fixes rdar://78035044

@swift-ci Please test


api,client: mark SSN as optional on CreateCustomer

See: https://github.com/moov-io/quick-enroll/issues/12

Codecov Report

Merging #161 into master will decrease coverage by 3.04%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #161      +/-   ##
==========================================
- Coverage   64.66%   61.61%   -3.05%
==========================================
  Files          43       43
  Lines        2493     2493
==========================================
- Hits         1612     1536      -76
- Misses        615      699      +84
+ Partials      266      258       -8


Moped won't retry write operations when primary went down (failover)

I noticed that max_retries or retry_interval will only work with read operations and write operations will not be retried even when primary node is dead.   [souce] (https://github.com/mongoid/moped/blob/master/lib/moped/read_preference/selectable.rb#L68)
Even when the master went down and other secondary become new primary in the replica set, node info will never be refreshed. Insert, update, remove commands keep throwing error for 5 minutes REFRESH_INTERVAL
Is this expected behavior?
Only when the read option is  primary, the read will also fail and the node info will be refreshed immediately.  To reproduce this bug, read option needs to be primary_preferred, secondary, secondary_preferred, or nearest.

This is fixed in #352


TypeError: 'NoneType' object is not subscriptable

Hello, I tried the test model on my computer(GTX 1070), but I found the mean fps is only 14.
So I run the time profile tools to test the performance. Then the error went out.
I'm not familiar with tensorflow and segmentation. Could you tell me how to fix this problem?
Here is the error information:
Trt engine file: ./checkpoint/bisenetv2_cityscapes_frozen.trt has been generated
WARNING:tensorflow:From tools/timeprofile_bisenetv2.py:90: calling import_graph_def (from tensorflow.python.framework.importer) with op_dict is deprecated and will be removed in a future version.
Instructions for updating:
Please file an issue at https://github.com/tensorflow/tensorflow/issues if you depend on this feature.
Traceback (most recent call last):
File "tools/timeprofile_bisenetv2.py", line 319, in
pb_file_path=args.pb_file_path
File "tools/timeprofile_bisenetv2.py", line 157, in time_profile_tensorflow_graph
image_feed = image[:, :, (2, 1, 0)]
TypeError: 'NoneType' object is not subscriptable
By the way, I'm searching for a segmentation model as the preprocess of a semantic slam system.
Can I deploy your work with tensorflow framework to the project just like what PaddlePaddle do?
Thank you.

@kkycj 1.No image loaded in your testing process. Check if you have input the right image file path. 2. Sorry I did not get your intention. What do you mean by deploy tf just like what paddle paddle do? :)

@MaybeShewill-CV Thank you for your reply.

I noticed that I copied the command in README.md,as
python tools/timeprofile_bisenetv2.py --input_image_path ./data/test_data/test_01.png

where ./data/test_data/test_01.png should be ./data/test_img/test_01.png. And I solved this.
2.I'm sorry I didn't explain my propose clearly. I'm trying to combine the segmentation programme with a slam system written by C++. As far as I known, PaddleSeg provide a way to export the model by C++ and deploy it in user's own project. I just want to know how can I achieve that in this case.

@kkycj 1.Thanks for correction. 2. TensorFlow also support cpp inference and you may google how to use it. 3. Other framework such as MNN or NCNN also can achieve what you want. 4. I have uploaded a tool to convert the pb model into tensorrt engine and you may use TensorRT to deploy the model. 5. PaddlePaddle have a useful tool named X2Paddle(or something like this) which may help you to convert the model from other framework into Paddle format. :)

@MaybeShewill-CV Thank you for your help. I greatly appreciate it!

@kkycj Welcome:)


using lookahead in zkd

Scope & Purpose
This is an experimental improvement of the ZKD based index iterator. While querying a multidimensional range of values one can now choose (instead of computing the next intersection point of the Z-curve with the range) to try first several documents that are immediately after the current document. If one of them is in the range, the computation of the next intersection point is not necessary, otherwise the point is computed.
A new parameter can be passed from the query: lookahead (a non-negative number). It tells how many documents after the current one should be checked for being in the search range before looking for the next point where the Z-curve intersects the search range. The lookahead is passed from the query to, finally, RocksDBZkdIndexIterator. However, it possibly should depend on the dimension of the index and on the limits of the query (see function numNextTries). The main change is in the function RocksDBZkdIndexIterator::nextImpl().

[ ] :hankey: Bugfix (requires CHANGELOG entry)
[ ] :pizza: New feature (requires CHANGELOG entry, feature documentation and release notes)
[x] :fire: Performance improvement
[ ] :hammer: Refactoring/simplification
[ ] :book: CHANGELOG entry made

Backports:

[ ] No backports required
[ ] Backports required for: (Please specify versions and link PRs)

Related Information
(Please reference tickets / specification / other PRs etc)

[ ] Docs PR:
[ ] Enterprise PR:
[ ] GitHub issue / Jira ticket number:
[ ] Design document:

Testing & Verification
(Please pick either of the following options)

[ ] This change is a trivial rework / code cleanup without any test coverage.
[x] The behavior in this PR was manually tested
[ ] This change is already covered by existing tests, such as (please describe tests).
[ ] This PR adds tests that were used to verify all changes:

[ ] Added new C++ Unit tests
[ ] Added new integration tests (e.g. in shell_server / shell_server_aql)
[ ] Added new resilience tests (only if the feature is impacted by failovers)

[ ] There are tests in an external testing repository:
[ ] I ensured this code runs with ASan / TSan or other static verification tools

Link to Jenkins PR run:
Documentation

All new features should be accompanied by corresponding documentation.
Bugs and features should furthermore be documented in the CHANGELOG so that
support, end users, and other developers have a concise overview.

[ ] Added entry to Release Notes
[ ] Added a new section in the Manual
[ ] Added a new section in the HTTP API
[ ] Added Swagger examples for the HTTP API
[ ] Updated license information in LICENSES-OTHER-COMPONENTS.md for 3rd party libraries

There are zdk index js tests. we should run another set of those tests with lookahead enabled to check that it does not miss any documents. See tests/js/server/aql/aql-optimizer-zkdindex-multi.js

Added testing with loolkahead 32.


Release

When merge, use merge commit. Squash method will fail to release. :)
Thank you for the resetting.

Lets also include the fix for #157 before merging this


dockerize the ksm server

It's very useful when we can run by using Docker.

Install docker & docker-compose
Run: docker-compose up

#10

Codecov Report

Merging #11 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #11   +/-   ##
=======================================
  Coverage   81.81%   81.81%
=======================================
  Files           3        3
  Lines         341      341
=======================================
  Hits          279      279
  Misses         37       37
  Partials       25       25

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d5f15e7...366362c. Read the comment docs.

LGTM


MQTT authentication not working

Authentication fails when setting MQTT_USERNAME and MQTT_PASSWORD in config, which is then passed to:
mqtt.setAuth(MQTT_USERNAME, MQTT_PASSWORD);
Editing the library to set the username and password directly in src/OXRS_MQTT.cpp on line 172 works fine, so it's something to do with passing in the values.
I plan to fix this myself: just putting it here as an issue to remind myself later.

I have finally got this code running locally and am seeing the same thing - it is a problem with the way I am trying to use #ifdef with const char variables.

See https://github.com/SuperHouse/OXRS-SHA-StateMonitor-ESP32-FW/pull/9


add an upgrade status endpoint

we are using the same url as in the crowbar api.
that makes it much easier for the clients to continue polling the
upgrade status during the upgrade
depends on https://github.com/crowbar/crowbar-core/pull/772

 #crowbar-init add an upgrade status endpoint

The status endpoint is indeed there and crowbarctl upgrade status is using it.
But crowbar-init fails with uninitialized constant Crowbar::Lock. When I include the lock library, it fails because the lock library is using Rails.logger ...

Any change in reviews here?

@rsalevsky added the crowbar lib directory to the $LOAD_PATH

 (3) S50P10 (S49P15) #11 Upgrade Admin Server: UI Story 2 - Service Integration

Approving after discussion - even though this is a bit ugly, we need to merge it ASAP.
Also, #toomanyreviewers.


Trouble with presentViewController

Hi,
I have a problem when I use a presentViewController..
Here's my case..
I have 2 centerViewControllers (let's say A and B).
From A, I can launch a presentViewController.
1st case :
When I launch the presentViewController from A, at first time, everything is perfect. It fills all the screen.
When I show B and back to A and launch the presentViewController, I have a 20px padding that appears at the bottom (Like if the y of the presentViewController was -20px)
2nd case :
When I show B then show A and launch the presentViewController, I have the 20px padding at the bottom  (Like if the y of the presentViewController was -20px)
Any help?
EDIT.
FYI. : the statusBarBehaviour property was set to .SlideAnimation
Seems to work when it sets to .ShowUnderlay

Can you make a demo project where you reproduce the issues and upload the zip here?

Or if you can reproduce it in main example project, make a screen capture.

Here's the file..
Take into account that I've changed your plist file (set View controller-based status bar appearance to YES)
and I've changed the statusBarBehaviour to SlideAnimation
From the "Center Controller 3", I've added a button "Press" that launches the Modal.
SideMenuController-master.zip

fixed in latest version


SONOFF Zigbee 3.0 USB Dongle Plus by ITead do support Auto BSL

The back-story is that several other Home Assistant community members have tested Sonoff provided uartLog.py script which seems to confirm "Auto BSL" is working even if it does not currently work with the current version of cc2538-bsl and llama-bsl.
https://community.home-assistant.io/t/itead-sonoff-zigbee-3-0-usb-dongle-plus-adapter-based-on-texas-instruments-cc2652p/340705/45
https://community.home-assistant.io/t/itead-sonoff-zigbee-3-0-usb-dongle-plus-adapter-based-on-texas-instruments-cc2652p/340705/96
Anyway, I have now also myself confirmed that running the attached uartLog.py script from Sonoff docx part as published in ITead's HOW TO FLASH FIRMWARE TO CC2652P instructions do indeed make ITead's Sonoff Zigbee 3.0 Plus Dongle automatically enter bootloader mode and after running that script just to get into BSL mode I could flash it directly using cc2538-bsl.py and llama-bsl.py scripts without having open the enclosure and pressing the BTL button which of course can be very convenient.
Copy of attached script from ITead/Sonoff: uartLog.zip
PS: JelmerT think probable reason why does not work in cc2538-bsl is extra delays in the script, see -> https://github.com/JelmerT/cc2538-bsl/issues/113
This is the code in their script that enters the bootloader:
def enterBoot(serialPort, delay=False):
    # delay is a workaround for bugs with the most common auto reset
    # circuit and Windows, if the EN pin on the dev board does not have
    # enough capacitance.
    # 如果开发板上EN管脚没有足够电容
    last_error = None

    # issue reset-to-bootloader:
    # RTS = either CH_PD/EN or nRESET (both active low = chip in reset
    # DTR = GPIO0 (active low = boot to flasher)
    #
    # DTR & RTS are active low signals,
    # ie True = pin @ 0V, False = pin @ VCC.
    setDTRState(serialPort, False)  # IO0=HIGH
    # setDTRState(serialPort, True)   # IO0=LOW
    setRTSState(serialPort, True)  # EN=LOW, chip in reset
    time.sleep(0.1)
    if delay:
        # Some chips are more likely to trigger the esp32r0
        # watchdog reset silicon bug if they're held with EN=LOW
        # for a longer period
        time.sleep(1.2)

    setDTRState(serialPort, True)  # IO0=LOW
    setRTSState(serialPort, False)  # EN=HIGH, chip out of reset
    if delay:
        # Sleep longer after reset.
        # This workaround only works on revision 0 ESP32 chips,
        # it exploits a silicon bug spurious watchdog reset.
        time.sleep(0.4)  # allow watchdog reset to occur
    time.sleep(1)

    setDTRState(serialPort, False)  # IO0=HIGH, done
    setRTSState(serialPort, False)
    time.sleep(1)

    return last_error

So it looks like both pins are active low, and RTS is connected to reset and DTR is connected to the bootloader pin. Which should be the same as in the standard configuration in this script.
So the only difference I'm seeing is the extra delays in their script.
Try adding some delays in the same spots, and see if that makes it work.

By the way, can probably assume this adapter uses DIO_13 for BSL Trigger Pin since it looks to take most hardware design references from TI LAUNCHXL-CC26xR1 or Slaesh's CC2652RB stick and Electrolama's zzh first-generation of CC2652 adapters?
...and if so it probably also uses DIO_7 for its LED (green?)?
https://github.com/Koenkk/Z-Stack-firmware/blob/master/coordinator/Z-Stack_3.x.0/bin/README.md

Adapter
TI Chip/Module Used
Firmware to Flash
BSL Trigger Pin (1)
Auto-BSL (2)
RF Switch Control Pins (3)
LED(s)
Hardware Flow Control

TI LAUNCHXL-CC26xR1
CC2652R
CC2652R_*.zip
DIO_13
No
N/A
DIO_6 (Red)DIO_7 (Green)
?

Electrolama zzh
CC2652R
CC2652R_*.zip
DIO_13
No
N/A
DIO_7 (Pink)
?

Slaesh's CC2652RB stick
CC2652RB
CC2652RB_*.zip
DIO_13
Yes
N/A
DIO_7 (Blue)
?

SONOFF Zigbee 3.0 USB Dongle Plus by ITead
CC2652P
CC1352P2_CC2652P_launchpad_*.zip
?
No
DIO_29: 20dBm PA
?
DIO13: TXDIO12:RCDIO19: CTSDIO18: RTS

As mentioned in https://github.com/JelmerT/cc2538-bsl/issues/113 if you can figure out how much delay needs to be added in cc2538-bsl you should be able to flash without needing uartlog.py.
Please submit a PR if you do, so we can support more HW in the cc2538-bsl script!

Hmm, but are you saying accepting PR #328 is dependent on the cc2538-bsl script supporting this feature for each dongle or not?
I thought that this README.md in this repo only listed the different hardware-enabled features for each specific dongle.
https://github.com/Koenkk/Z-Stack-firmware/blob/master/coordinator/Z-Stack_3.x.0/bin/README.md
At least as it stands now this README.md does not specify that "Auto-BSL" == cc2538-bsl script supported for dongle. In fact, the cc2538-bsl script is currently not mentioned in this repo at.
So what if "Auto-BSL" is already supported by the hardware and working in aother script other than the cc2538-bsl script?

FYI, JelmerT got it working however I believe that help may still wanted in testing firmware upgrade(s) with his pull request patch for ITead’s Sonoff CC2652P USB dongle with JelmerT cc2538-bsl tool using the new option --bootloader-sonoff-usb parameter.
https://github.com/JelmerT/cc2538-bsl/tree/feature/ITead_Sonoff_Zigbee-delay
Please report firmware upgrade issue with it or confirm it working under which operating system so this might get merged:
https://github.com/JelmerT/cc2538-bsl/pull/114
(Again there is a longer related troubleshooting discussion in https://github.com/JelmerT/cc2538-bsl/pull/114 with some more information on solution).

Closing as https://github.com/Koenkk/Z-Stack-firmware/pull/329 by @guozi7788 also contained this change setting Auto-BSL field to Yes value for the Sonoff Plus dongle now.


Improve find TelegramHandle command

Closes #143
Update the required prefix for the find TelegramHandle command to the correct prefix "h/" instead of "/h"

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Files with missing lines
Coverage Δ
Complexity Δ

...java/seedu/address/logic/commands/FindCommand.java
100.00% <ø> (ø)
7.00 <0.00> (+1.00)

.../seedu/address/logic/parser/FindCommandParser.java
100.00% <100.00%> (ø)
5.00 <0.00> (ø)


Example Chip(Chest): Sell Blocks

Let players take blocks from your chest by sending you Ether

Consolidated into just one chip: buysell-chest


Add Skiplink

Add a skiplink button component.
[] It should be a html a element
[] It should be imported into the layout component and placed before the header.
[] It should be hidden offscreen a few pixels above where it will normally be positioned but appear in the center of the header when tabbed to. Maybe a small transition could be added so it slides into place?
[] It should be styled to have black text, white background and a pink border (like a button).
[] It should move focus to the main element (also inside layout) when pressed - add an id to the main tag to do this.
[] Text should be: Skip to main content

Hi Emma, I really suck at CSS, this could be a good one to do little practice.
Can I be assigned to this as well?

Sure, feel free to give it a go and let me know if you need any help.

Hi Emma, is this what you were expecting to see?
https://user-images.githubusercontent.com/1682188/192809717-ddb184e4-1a20-49d2-894e-52c100e9b08f.mov

Yes! That’s pretty much what I envisioned 😀


CREATE DATA ORIGIN now gets its own section and index entry

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@nassibnassar I'm starting with a trivial PR to check that the process works correctly and that my commit is correctly signed. This fixes an error in the current documentation, that the CREATE DATA ORIGIN section did not get an index entry.

Well, that went pretty smoothly, once I'd got all the GPG stuff working.


Consider adding a packages_aggregates table

We're technically updating the package record but what we're really doing is aggregating up build data. So it's not the package per se that's changing.
A cleaner solution would perhaps be to move this aggregate field platform_compatibility to a separate table since we're using packages as a convenient place to stick it.
Maybe packages_aggregates. It could hold platform_compatibility and score_details. I'll add an issue to track.
Originally posted by @finestructure in https://github.com/SwiftPackageIndex/SwiftPackageIndex-Server/pull/3298#discussion_r1719584311

As an alternative we could add last_analyzed_at and last_ingested_at fields to make these dates explicit. One of the things I'd like to add to the portal (once it exists) is to surface these two dates to package authors and it feels better to have them be explicit rather than relying on updated_at.


Configuration Validation

Currently most of the configuration is validated during runtime. However it would be much better to make a validation of the whole configuration file during startup.
-> Implement a validation method for each configuration class and call them recursively.
-> If there is an error print the error and abort the starup of nelly.

Implemented with version 0.3


com.google.apps.framework.request.BadRequestException when running nlpanalytics with dialogflow

Describe the bug
I am trying to run intent analysis for my dialogflow project as described in https://medium.com/@floriantreml/tutorial-benchmark-your-chatbot-on-watson-dialogflow-wit-ai-and-more-92885b4fbd48
I was able to successfully import utterances, but when I'm running botium-cli nlpanalytics I am getting the following error:
  botium-core-BotDriver Build - Envs : { IS_BOTIUM_CONTAINER: true } +1ms
  botium-connector-PluginConnectorContainer-helper Botium plugin loaded from /usr/local/lib/node_modules/botium-cli/node_modules/botium-connector-dialogflow/index.js +0ms
  botium-connector-dialogflow Validate called +0ms
  botium-connector-BaseContainer _RunCustomHook onBuild finished +0ms
  botium-connector-dialogflow Build called +1ms
  botium-connector-dialogflow Clean called +10s
  botium-connector-BaseContainer _RunCustomHook onClean finished +10s
  botium-connector-BaseContainer Cleanup rimrafing temp dir /Users/vovan/intent_analysis/botiumwork/intent-test 20200702 191900 sy9Ir +0ms
K-Fold training for fold 1 failed: {"code":3,"message":"com.google.apps.framework.request.BadRequestException: NLUSettings.validate() failed because of the following reasons: Cannot change read-only field NLUSettings.defaultLanguage during an update","details":[]}

Expected behavior
No errors with benchmark summary.
Botium Flavour:
Botium CLI
Botium Version
bash-3.2$ botium-cli -V
0.0.69
Additional context
N/A

I cannot see the issue anymore after I added the following lines to the botium.json:
  "DIALOGFLOW_LANGUAGE_CODE": "en",
  "DIALOGFLOW_NLP_LANGUAGE_CODE": "en",

great that you was able to overcome this problem yourself!


XML tests can use a canned (i.e. non-legit) configuration

Even the XML handling tests need a SRS configuration to be supplied to the user. These tests, at least, should be able to manage with a canned config because they aren't trying to interact with an actual testing instance.

Has this already been covered to your satisfaction?


[Bug] Test bug-report from Jingsong Lee

Search before asking

[X] I searched in the issues and found nothing similar.

Flink version
1.15
Flink CDC version
2.1
Database and its version
Mysql
Minimal reproduce step
A special case
What did you expect to see?
Good issue template!
What did you see instead?
No template, I don't know how to use.
Anything else?
No response
Are you willing to submit a PR?

[X] I'm willing to submit a PR!

This looks very good!


[Design] - Rate Limit, Alerts, Monitoring, QA, Deployment Steps

CI

[ ] We need to run tests on every push

[ ] To be able to do this, we need to spawn a MySQL on CI
[ ] To be able to do this, we need to spawn a Redis server on CI
[ ] To be able to do this, we need migrations on at least a script to initialize the database

PR: https://github.com/HathorNetwork/hathor-wallet-service/pull/97
CD and IaaS
This is being worked on by @luislhl on https://github.com/HathorNetwork/ops-tools/issues/34
Database

We currently don't have migrations, that's necessary for automated deploys.
We can deploy the database from the last snapshot so we don't have to sync from genesis on every deploy

Snapshots
Snapshot daily
Monitoring
There are some critical events that we need to be alerted of:
Wallet Service

[ ] Error on balance calculation
[ ] Error on MySQL connection
[ ] Error on FullNode connection
[ ] Wether the wallet-service is being properly synced with the full-node in realtime

Daemon

[ ] A reorg is detected with more than 1000 blocks difference
[ ] More than X minutes/seconds without a new block from the connected fullnode
[ ] Websocket connection lost with the full-node after X retries
[ ] Error on Wallet Service connection

We already have migrations implemented, so a automated CD pipeline is already possible.
However, we need to discuss how to make everything work together, given that:

We were planning to not let the Database publicly accessible
By trying to access the database from Github Actions, we have no control over the source IP that this access comes from.

These two facts make it harder for us to trigger the migrations directly from Github Actions.
I have come up with some options to solve this problem:
Allow all possible Github Actions IPs on the database
This is possible, because they have an API that returns the updated IP ranges: https://api.github.com/meta
However, it is updated once a week: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#ip-addresses
We would need to have some automation that keeps updating our Security Group using their list.
However, I do not consider this a secure option, since anyone can run a Github Actions job.
Give Github Actions AWS permissions to update the Security Group
This is the one I prefer. I think it has a good benefit/cost ratio.
The idea is that we would get the IP of the current runner, then update the Security Group to allow access from it, run the migrations, and finally update the Security Group again removing the allowed IP.
The only security concern I see here is that of making sure no one we don't trust have write access to the repository where those secrets are set.
Since the IP would be allowed only during a short time span, I don't think this would open any significant breaches.
Use Self-hosted runners
https://docs.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners#about-self-hosted-runners
This is possibly the most secure option, since we would run our own runner, that would be used by our Action.
However, there is more work involved in making this work (I don't know how much yet), and more maintenance work to keep the machine updated.
This solves our problem because we could just allow our own runner's IP to access the database, without having to provide AWS secrets to the Action.
Create a new Lambda that runs migrations
In this option, Github Actions would still need to have AWS credentials that allowed it to call this Lambda.
Then, we would allow the Lambda to access the database.
One problem that may arise is if some migration in the future takes more than 15 minutes to run, which would hit the Lambda timeout. Depending on what the migration does, this is not so hard to happen in large tables. In Github Actions jobs, this limit is 6 hours.

Monitoring
Suggestions on how we could monitor the different events we are intending to monitor.
Wallet Service

Error on balance calculation
Error on MySQL connection
Error on FullNode connection

For those, we just need to make sure the Lambdas are exiting with an error status in all the cases, then we can create an Alert in AWS triggered by errors in the Lambdas.
The only problem I see with this is how we will make sure new Lambdas are also included in this monitoring.

Wether the wallet-service is being properly synced with the full-node in realtime

We have some options here that we should evaluate.
The first one is to simply create an alert in AWS to warn us in case the Lambda onNewTxRequest stops being called regularly, or in case it is erroring out. This would be easy to do and to send the alerts to Slack using AWS Chatbot + CloudWatch alerts.
Another option suggested by @andreabadesso is to export the highest height received by the WalletService to Prometheus, and then have an alert to compare it against the height in our FullNodes.
This is a good suggestion, since it could also catch cases where the sync is happening, but is just not fast enough. However, it brings additional complexity that we would need to solve.
We would need a way to expose the metrics to Prometheus, which in a Lambda environment is a little more challenging. The alternatives would be:
- Have a /metrics endpoint in API Gateway, which queries directly in MySQL for the height. This query is currently taking circa 1.5s to run. If we need to add other metrics, they would add other queries.
- Somehow share state between different Lambdas. Maybe in S3 or other way. Then we would make one Lambda update the file on specific events, like new blocks or transactions, and the other would just return this in the /metrics endpoint. But this doesn't feel like a good solution.
- Install a Prometheus PushGateway, and make the Lambdas push metrics there. This seems the most future-proof solution. However, since we are in the process of migration to a new Grafana/Prometheus installation, ideally we would install this PushGateway in this new installation. But our main full-nodes are on the old one, and the WalletService full-node is not monitored yet.
Daemon

A reorg is detected with more than 1000 blocks difference
This is an event that, as far as I understand, we are only capable of detecting inside the Daemon logic. So the Daemon needs a way to warn some other tool when this happens.

Again, some options.

Just use logs with specific markers (like including an [ALERT] string), and send them to CloudWatch. Then, we would be able to create alerts based on the log content.

The first problem here is that we are not sending the Kubernetes logs yet to CloudWatch.
The second problem is that we would be unable to send the log message together with the alert. To do this, additional effort would be needed: https://stackoverflow.com/questions/60171078/how-to-send-alert-based-on-log-message-on-cloudwatch/60171655?noredirect=1#comment120741778_60171655

Make the daemon send messages directly to AWS SNS, then route then to AWS Chatbot/Slack.

Make the daemon log the alerts to some file, then have another container read those logs, and for each entry, send an alert somewhere. This container could be built as a reusable container. Everytime we had a similar case, we would use it.

More than X minutes/seconds without a new block from the connected fullnode
Isn't this solved by the alert in WalletService synx with full-nodes?

Websocket connection lost with the full-node after X retries
This is similar to the reorg case. The daemon needs a way to push an alert somewhere warning about this.

Error on Wallet Service connection:
Isn't this solved by alerting on Lambda errors and timeouts?

Rate Limits
@andreabadesso and @pedroferreira1 have agreed that we could start with simple rate-limits on endpoints, and create another issue to discuss more complex strategies.
So, the idea for now is to just have them configured in API Gateway through Serverless.

Continuous Deployment
I agree with using AWS CodePipeline, it makes sense for me to use it for deployments and GithubActions for tasks that are better to stay publicly accessible
Monitoring

Whether the wallet-service is being properly synced with the full node in realtime

Our getLatestBlock query is now much faster after we refactored it to use the index it has on height, I think the /metrics endpoint would be the most recommended option although I agree that it might not be the most scalable one since we might want to monitor other metrics that could also be heavy on the database
Given that, I would also go with the PushGateway solution, I saw the potential drawbacks from using it but since our service is a lambda application, I think they don't apply here

A reorg is detected with more than 1000 blocks difference

I don't think we need to send the log message alongside the alert. If our mainnet reorgs more than 1000 blocks, we necessarily need to investigate it manually, so the alert will just be the trigger for that manual action, we can read the logs on CloudWatch

More than X minutes/seconds without a new block from the connected fullnode
Isn't this solved by the alert in WalletService sync with full-nodes?

Yes it is

Websocket connection lost with the full-node after X retries

I agree, we can just monitor the logs for alert strings

Error on Wallet Service connection
Isn't this solved by alerting on Lambda errors and timeouts?

Yes

Given that, I would also go with the PushGateway solution, I saw the potential drawbacks from using it but since our service is a lambda application, I think they don't apply here

@andreabadesso , just for clarity, you think we should start with the PushGateway solution?
Also, which potential drawbacks are you mentioning exactly? I didn't understand what are they and why they don't apply.

I don't think we need to send the log message alongside the alert. If our mainnet reorgs more than 1000 blocks, we necessarily need to investigate it manually, so the alert will just be the trigger for that manual action, we can read the logs on CloudWatch

Great, then alerts on logs seems like a good solution.

Task List
This is the task list so far, based on @andreabadesso 's and mine opinions
WalletService

[ ] Configure the rate-limits on endpoints through Serverless
[ ] Make sure we are either exiting the Lambda or logging a message containing specific strings when some of those events happen: (we can include other events we think of when doing this)

Error on balance calculation
Error on MySQL connection
Error on FullNode connection

Daemon

[ ] Make sure we are logging when the Daemon loses connection to the FullNode, and include an "ALERT" string on the log message
[ ] Make sure we are logging when a reorg greater than 1000 is detected, and include an "ALERT" string on the log message

Infra

[ ] Send all Kubernetes logs to CloudWatch
[ ] Configure CodePipeline to deploy the WalletService and Daemon
[ ] Create a Metric Filter and Alarm in CloudWatch to send alerts to Slack when the Daemon logs contains the string "ALERT"
[ ] Create an Alarm in CloudWatch for when either a Lambda exits or contains specific error string in the logs
[ ] Create an Alarm in CloudWatch for when the onNewTxRequest Lambda is not called in for too much time.

Under Discussion

We are still discussing if we will integrate WalletService with Prometheus from the start, or as a future improvement

Future Tasks
Infra

[ ] How to include the log message into Alerts?

Continuous Deployment

I agree about using the AWS Pipeline, I was just wondering about having our own runners. We currently have some private projects, which have limits on running actions and eventually we need to pay for github to continue running. If we think this will become common (having private projects that run github actions), then using our own runners to prevent having this extra cost with github could be a good solution.
I don't think this is relevant now because I don't see us having many private projects in the near future but I would like to know about the cost (monthly cost and configuration/initial time cost) to have our own runners.

In a first moment, use just AWS to monitor errors in Lambdas and invocations of onNewTxRequest to make sure the sync is working. This already provides us with a good level of reliability.

The daemon that calls onNewTxRequest, right? We might have problem receiving new blocks between the full node and the daemon and between the daemon and the wallet service, right? Maybe we should have a way to compare the data from the 3 services and check if they match by a delta difference (in the height I would say 1 is the limit).
I agree about using PushGateway but in the meantime we could use the prometheus that the full node already saves and create a prometheus for the daemon and create a periodic lambda task that compare the value in both prometheus height with the value in the wallet service db and alert any change > 1. Not sure if I understood correctly the alert need but I think this would be enough.

A reorg is detected with more than 1000 blocks difference

I think any reorg > 1 should be alerted. At least for now, we can adjust this later.

So, the idea for now is to just have them configured in API Gateway through Serverless.

Do we have an issue for that?

I don't think this is relevant now because I don't see us having many private projects in the near future but I would like to know about the cost (monthly cost and configuration/initial time cost) to have our own runners.

Can we create a new issue to make this estimation? I'm not sure this is trivial to do.
We would have to investigate the most cost-effective way of deploying those runners. EC2 is certainly not the best option for cost, since it would be idle for a long time. But I don't know in their runner supports Serverless deployments, or even deploying inside Kubernetes.

The daemon that calls onNewTxRequest, right? We might have problem receiving new blocks between the full node and the daemon and between the daemon and the wallet service, right? Maybe we should have a way to compare the data from the 3 services and check if they match by a delta difference (in the height I would say 1 is the limit).

My premise to try to simplify things was that checking that new blocks are arriving between the daemon <-> wallet service was enough to check if everything is working. If there is some problem between the full-node <-> daemon, daemon <-> wallet service will also be affected, and we would detect the problem.

I agree about using PushGateway but in the meantime we could use the prometheus that the full node already saves and create a prometheus for the daemon and create a periodic lambda task that compare the value in both prometheus height with the value in the wallet service db and alert any change > 1. Not sure if I understood correctly the alert need but I think this would be enough.

I think the point above applies here too. The daemon is an intermediary between the full-node and WalletService. If we are able to check if full-node and WalletService are in the same block, that's enough to guarantee that the daemon is also working. So I don't think we need the daemon to export Prometheus metrics.
But the Lambda idea seems good to me. It doesn't need to go on the DB, it would be enough to just get the full-node's metrics (or even call some API in the full-node to get the last block) and call the getLatestBlock Lambda in the WalletService, then compare them.

Can we create a new issue to make this estimation? I'm not sure this is trivial to do.

Yeah, sure. We can start with the AWS pipeline if it's easy to configure. I am not sure how many private projects we intend to have.

My premise to try to simplify things was that checking that new blocks are arriving between the daemon <-> wallet service was enough to check if everything is working. If there is some problem between the full-node <-> daemon, daemon <-> wallet service will also be affected, and we would detect the problem.

I think it makes sense. I was thinking about monitoring and already returning the point of failure. If we see a problem in the wallet service db we still need to debug if the problem was in the daemon <> full node or in the daemon <> wallet service. But I agree with you, for both situations it makes sense.

@msbrogli , including you here just so you receive a notification.

Questions:

Why does GitHub need access to the database?

In regards to "Give Github Actions AWS permissions to update the Security Group", I'm more concerned about the automated task that can give access to others than with the access itself.

I like the "Use Self-hosted runners" approach. It can run inside our VPS, which would reduce the permission issues in general.

I like the "Use AWS CodePipeline instead of Github Actions" as well. I'm not sure how complex it is to integrate both tho.

@luislhl I like your suggestion to use AWS CodePipeline. I'm not sure what would be the impact in our repositories and it seems an important piece of information to make a decision.

Monitoring: the ideas seem good to me.

Daemon: I agree that we should send the logs to the CloudWatch (or any other centralized log system).

Rate limits: But what will be the initial simple rate limits? I think we should have both a limit per IP and a global limit.

As far as I understood, the PushGateway would be used to push metrics to Prometheus, right? In this case, how would the metrics be collected? Would it be stored in an S3 file that would be used by PushGateway later? I didn't understand the whole picture.

In regards to reorgs, we will have alerts for them in our full nodes. So, maybe the wallet service could benefit from it (instead of creating its own solution). What do you think?

@msbrogli

Questions:

Why does GitHub need access to the database?

Depends on which option we choose to implement. In case we go with AWS CodePipeline, for example, it doesn't.
The initial example was just to illustrate the problem.

In regards to "Give Github Actions AWS permissions to update the Security Group", I'm more concerned about the automated task that can give access to others than with the access itself.

That's why we would have to be concerned about who has access to the repo. Only people with write access to the repo can trigger the pipeline.

I like the "Use Self-hosted runners" approach. It can run inside our VPS, which would reduce the permission issues in general.

Exactly, but it seems more complex than using AWS CodePipeline, which can also run inside the VPC.

I like the "Use AWS CodePipeline instead of Github Actions" as well. I'm not sure how complex it is to integrate both tho.

As far as I researched, CodePipeline can be triggered by actions in Github repos, like on new commits, releases, etc.

@luislhl I like your suggestion to use AWS CodePipeline. I'm not sure what would be the impact in our repositories and it seems an important piece of information to make a decision.

What kind of impact you are talking about exactly?
I think the only impact is that we would have to give it OAuth access to our Github accounts/repos. Like described in https://docs.aws.amazon.com/codepipeline/latest/userguide/connections-github.html

Rate limits: But what will be the initial simple rate limits? I think we should have both a limit per IP and a global limit.

Will describe them better. We have to check if API Gateway support limits per IP.

As far as I understood, the PushGateway would be used to push metrics to Prometheus, right? In this case, how would the metrics be collected? Would it be stored in an S3 file that would be used by PushGateway later? I didn't understand the whole picture.

The metrics would be pushed to PushGateway by Lambdas. PushGateway aggregates them, and makes the result available in a endpoint. Prometheus then pulls them from this endpoint.
It's basically an intermediary to allow pushing metrics instead of the usual pull-based approach in Prometheus.

In regards to reorgs, we will have alerts for them in our full nodes. So, maybe the wallet service could benefit from it (instead of creating its own solution). What do you think?

I need to understand better how will those alerts work in the full nodes. Also, @andreabadesso 's opinion is important here.

@luislhl

About why GitHub needs access, I meant the reason. But I guess I got it. Just to check: GitHub needs access to be able to run the migrations. Is that right?

Another idea is to use GitHub to trigger the execution of the migrations in the Kubernetes. But it might be even easier using the AWS CodePipeline.

About the impact in the repositories, I meant to move all actions to AWS CodePipeline. For instance, how would a project configure the AWS CodePipeline? Does it support to read the configuration file directly from the repository? Would we need to use TerraForm?

If API Gateway does not support throttling per IP, we can use it for the global limits. But I think we should have another intermediary to throttle per IP (e.g., nginx).

Can't Prometheus pull from S3?

We still need to develop the reorg alerts in the full node. But it is something that we will do anyway. It is very important to learn as soon as possible when a reorg occurs. But I agree we need to put @andreabadesso in this loop.

@msbrogli

About why GitHub needs access, I meant the reason. But I guess I got it. Just to check: GitHub needs access to be able to run the migrations. Is that right?

Yes, to run the migrations. But if we choose another solution, it's not needed.

Another idea is to use GitHub to trigger the execution of the migrations in the Kubernetes. But it might be even easier using the AWS CodePipeline.

This raises other concerns, like giving access on the Kubernetes API to GitHub actions. This API is currently blocked by IP, actually, for security reasons. We would need to expose it.

About the impact in the repositories, I meant to move all actions to AWS CodePipeline.

The idea is not to replace GitHub Actions entirely, just for deployment. I would let PR validation tasks and some other tasks (like building Docker images for hathor-core) still in Github Actions.

For instance, how would a project configure the AWS CodePipeline? Does it support to read the configuration file directly from the repository? Would we need to use TerraForm?

Those are good questions, I still don't have answer for them. But will try to figure them out.

Can't Prometheus pull from S3?

I don't think so, and a quick search didn't return anything showing this is possible. We would need a service that reads from S3 and exports metrics in an API. Maybe there is some project in the internet with such service that we would just have to deploy.

I have created a PR adding a .md file with the condensed information already discussed here.
https://github.com/HathorNetwork/hathor-wallet-service/pull/111
I think we should continue discussions there, so I'll be closing this issue.


solution.equalize syntax

in phreeqc I can write
 EQUILIBRIUM_PHASES
Goethite 0 1.04e-3

which states that Geothite should be in equilibrium but that at best only 1.04e-3 mol can be dissolved in order to attain equilibrium. I am unclear on how to achieve this with the solution.equalize method. Thx for any pointers!

Hi,
The solution.equalize method has the following parameters:
phases, to_si=[0], in_phase=[10], with_chemical=[None]
The in_phase parameter describes the amount of phase available.
To achieve your example above you thus need to write:
sol.equalize("Geothite", 0, 1.04e-3)
which will generate the following block of PHREEQC code
USE SOLUTION 0
EQUILIBRIUM PHASES 1
Geothite 0 0.00104
SAVE SOLUTION 0
END

When in doubt you can always enable debug mode using pp.ip.debug=True to see what PhreeqPython is sending to PHREEQC.

ah ok!. pp.ip.debug is a godsend, Thx!


Creating fresh WinUI Blank Package and running it throws exception.

Describe the bug
Repro steps:

Create a new WinUI Blank Package app in Visual Studio 2022 17.2
Build and run package app.

Exception reported straight away:
onecoreuap\windows\moderncore\inputv2\inputhost\outside\lifted\desktopwindowtoplevelhost.cpp(94)\Microsoft.UI.Input.dll!00007FFC366A7490: (caller: 00007FFC366A7781) FailFast(1) tid(5338) 8000FFFF Catastrophic failure
Unhandled exception at 0x00007FFCDE96FA32 (KernelBase.dll) in XXX.exe: Fatal program exit requested.
Steps to reproduce the bug

Create a new WinUI Blank Package app in Visual Studio 2022 17.2
Build and run package app.

Expected behavior
Expected to see a blank window ready to be worked on.
Screenshots
No response
NuGet package version
WinUI 3 - Project Reunion 0.8: 0.8.8
Windows app type

[ ] UWP
[ ] Win32

Device form factor
Desktop
Windows version
Windows 10 (21H2): Build 19044
Additional context
No response

WinUI 3 0.8.x is dead; switch to Windows App SDK 1.0.x or Windows App SDK 1.1 preview.

@olivegamestudio Do you still see this with the WindowsAppSDK 1.1? This is just a hunch, but can you make sure you have the runtime installed? It's called the "installer" here: https://docs.microsoft.com/en-us/windows/apps/windows-app-sdk/downloads


change password changes

What and why?
How to test?
Trello
https://trello.com/c/KZ7ZpmW3/1262-rops-ui-self-service-change-password

/deploy sinhaa


What is the difference between manually adding a user and a tag?

I expanded your project to include support for the database SQLite3. This is the server part. That is, I am sending data to the database from the emulator of the websocket. But at the same time I enter the data on the  tag in manual. I do not use the reader. Question: if the data on the card is driven in after the contact of the reader and the tag, then why do we need to manually enter the data on the card? How will a new record be tied to the tag during manual input?

esp-rfid does not enter any data on any card. There is no difference between manually adding a user or by scanning a tag, if you know the UID of the card you can add it manually.

Thank much!


workflows: improve performance of github CI

Bump actions/checkout to v2
Limit number of posts to be built to 10

Build time reduced from about 6 minutes to 2.20 minutes (rough average)

@00-matt See https://github.com/monero-project/monero-site/pull/892#discussion_r407413181. Personally i'm ok with living it as it is.


Allow users to login from micro site

Users should be able to login into OSM from within micro site

Requires implementation of OAuth sign in, which likely would involve expanding the microsite beyond its simple, front-end-only, Jekyll-based structure.


run_dqn demo fails with distributed training: ValueError: TrajectoryColumns cannot contain any None data references

Ubuntu 20.04
cuda-11.4
4 GPU / 4 CPU core node
Setup (from fresh VM):
> apt-get update && apt-get install -y --no-install-recommends \
  libgl1-mesa-glx libosmesa6 libglew-dev
> pip install --upgrade pip setuptools wheel
> git clone https://github.com/deepmind/acme.git acme_repo
> cd acme_repo
> pip install .[jax,tf,testing,envs]
> pip install --upgrade "jax[cuda]" -f https://storage.googleapis.com/jax-releases/jax_releases.html
> pip install ale-py

Your favorite method to install atari rooms into ale-py here
Then:
> cd examples/baselines/rl_discrete
> python run_dqn.py --run_distributed

Produces this error:
I0525 17:21:19.286271<PHONE_NUMBER>56864 terminal.py:91] [Actor] Actor Episodes = 1 | Actor Steps = 186 | Episode Length = 186 | Episode Return = -5.0 | Steps Per Second = 1.972
Node ThreadWorker(thread=<Thread(actor, stopped daemon<PHONE_NUMBER>99488)>, future=<Future at 0x7f181df5fa60 state=finished raised ValueError>) crashed:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/launchpad/launch/worker_manager.py", line 474, in _check_workers
    worker.future.result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 437, in result
    return self.__get_result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.8/dist-packages/launchpad/launch/worker_manager.py", line 250, in run_inner
    future.set_result(f())
  File "/usr/local/lib/python3.8/dist-packages/launchpad/nodes/python/node.py", line 75, in _construct_function
    return functools.partial(self._function, *args, **kwargs)()
  File "/usr/local/lib/python3.8/dist-packages/launchpad/nodes/courier/node.py", line 130, in run
    instance.run()
  File "/usr/local/lib/python3.8/dist-packages/acme/environment_loop.py", line 176, in run
    result = self.run_episode()
  File "/usr/local/lib/python3.8/dist-packages/acme/environment_loop.py", line 109, in run_episode
    self._actor.observe(action, next_timestep=timestep)
  File "/usr/local/lib/python3.8/dist-packages/acme/agents/jax/actors.py", line 94, in observe
    self._adder.add(
  File "/usr/local/lib/python3.8/dist-packages/acme/adders/reverb/transition.py", line 133, in add
    super().add(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/acme/adders/reverb/base.py", line 207, in add
    self._write()
  File "/usr/local/lib/python3.8/dist-packages/acme/adders/reverb/transition.py", line 169, in _write
    reward, discount = tree.map_structure(
I0525 17:21:20.443459<PHONE_NUMBER>84928 lp_utils.py:95] StepsLimiter: Reached 186 recorded steps
  File "/usr/local/lib/python3.8/dist-packages/tree/__init__.py", line 430, in map_structure
    [func(*args) for args in zip(*map(flatten, structures))])
  File "/usr/local/lib/python3.8/dist-packages/tree/__init__.py", line 430, in <listcomp>
    [func(*args) for args in zip(*map(flatten, structures))])
  File "/usr/local/lib/python3.8/dist-packages/acme/adders/reverb/transition.py", line 151, in <lambda>
    get_all_np = lambda x: x[self._first_idx:self._last_idx].numpy()
  File "/usr/local/lib/python3.8/dist-packages/reverb/trajectory_writer.py", line 604, in __getitem__
    return TrajectoryColumn(self._slice(val), path=path)
  File "/usr/local/lib/python3.8/dist-packages/reverb/trajectory_writer.py", line 629, in __init__
    raise ValueError('TrajectoryColumns cannot contain any None data '
ValueError: TrajectoryColumns cannot contain any None data references: TrajectoryColumn at path ('reward', slice(187, 188, None)) got [None].

Without the distributed flag, it works fine.

Digging a little, it appears that sometimes the next_timestep variable has None values:
https://github.com/deepmind/acme/blob/2871e3216d2ffc2bc0ffea8b6a0e3071897608b9/acme/agents/jax/actors.py#L95
TimeStep(step_type=<StepType.FIRST: 0>, reward=None, discount=None, observation=Somenonzeroarray)

I think I've tracked down where things are going wrong, at least in the environment loop:
https://github.com/deepmind/acme/blob/2871e3216d2ffc2bc0ffea8b6a0e3071897608b9/acme/environment_loop.py#L106
With the -run_distributed option, the environment step call sometimes returns a timestep with None reward and discount factor, as if it had called reset. I don't know enough about the backend to understand how to fix this.

I think I know the issue: the environment factory in that example (in fact all examples) returns the same instance of the same environment, so there's some sort of async problem with agents calling reset on the environment right after other agents called step. I switched the factory to something that generates new environment instances, and things appear to work fine.
Is this the correct solution to this?

@rdevon The environment_factory should create a new environment every time it is called.
Therefore, the examples rl_discrete/ are indeed all incorrect.


cupy do not suport two axis operation?

Description
A is a $m\times n\times n$ tensor
when i run np.all(np.isnan(A), axis=(1, 2)) or cp.all(cp.isnan(A), axis=1), there has no error.
but when i run A1 = cp.all(cp.isnan(A), axis=1); A2 = cp.all(cp.isnan(A1), axis=1) or A2 = cp.all(cp.isnan(A), axis=(1,2)), the error happens.
i add cl.exe D:\Software\VS2019IDE\VC\Tools\MSVC\14.29.30133\bin\Hostx64\x64 to windows system path, but it still doesnt work.
To Reproduce
D:\CTRec_Cupy\lib\site-packages\cupy\cuda\compiler.py:103: UserWarning: Failed to auto-detect cl.exe path: <class 'distutils.errors.DistutilsPlatformError'>: No Microsoft Visual C++ version found
  warnings.warn(f'Failed to auto-detect cl.exe path: {type(e)}: {e}')
Traceback (most recent call last):
  File "D:\CTRec_Cupy\lib\site-packages\cupy\cuda\compiler.py", line 64, in _run_cc
    log = subprocess.check_output(cmd, cwd=cwd, env=env,
  File "D:\Software\Python3.9_20231208\lib\subprocess.py", line 424, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "D:\Software\Python3.9_20231208\lib\subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['D:\\Software\\CUDA12.3\\bin\\nvcc.EXE', '-gencode=arch=compute_50,code=sm_50', '--ptx', '--std=c++11', '-ID:\\CTRec_Cupy\\lib\\site-packages\\cupy\\_core\\include', '-ID:\\CTRec_Cupy\\lib\\site-packages\\cupy\\_core\\include\\cupy\\_cuda\\cuda-12', '-ID:\\Software\\CUDA12.3\\include', '-ftz=true', 'C:\\Users\\14682\\AppData\\Local\\Temp\\tmp8mzyyiow\\preprocess.cu']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\CTRec_Cupy\lib\site-packages\cupy\cuda\compiler.py", line 366, in compile_using_nvcc
    _run_cc(cmd, root_dir, 'nvcc', log_stream)
  File "D:\CTRec_Cupy\lib\site-packages\cupy\cuda\compiler.py", line 80, in _run_cc
    raise NVCCException(msg)
cupy.cuda.compiler.NVCCException: `nvcc` command returns non-zero exit status.
command: ['D:\\Software\\CUDA12.3\\bin\\nvcc.EXE', '-gencode=arch=compute_50,code=sm_50', '--ptx', '--std=c++11', '-ID:\\CTRec_Cupy\\lib\\site-packages\\cupy\\_core\\include', '-ID:\\CTRec_Cupy\\lib\\site-packages\\cupy\\_core\\include\\cupy\\_cuda\\cuda-12', '-ID:\\Software\\CUDA12.3\\include', '-ftz=true', 'C:\\Users\\14682\\AppData\\Local\\Temp\\tmp8mzyyiow\\preprocess.cu']
return-code: 1
stdout/stderr:
nvcc fatal   : Cannot find compiler 'cl.exe' in PATH

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "xxx", line 49, in <module>
    vivek(detector_num_y, detector_num_z,
  File "xxx", line 537, in xxx
    t2 = cp.all(t1, axis=1)
  File "D:\CTRec_Cupy\lib\site-packages\cupy\_logic\truth.py", line 50, in all
    return a.all(axis=axis, out=out, keepdims=keepdims)
  File "cupy\_core\core.pyx", line 1169, in cupy._core.core._ndarray_base.all
  File "cupy\_core\core.pyx", line 1171, in cupy._core.core._ndarray_base.all
  File "cupy\_core\_routines_logic.pyx", line 8, in cupy._core._routines_logic._ndarray_all
  File "cupy\_core\_reduction.pyx", line 608, in cupy._core._reduction._SimpleReductionKernel.__call__
  File "cupy\_core\_reduction.pyx", line 364, in cupy._core._reduction._AbstractReductionKernel._call
  File "cupy\_core\_cub_reduction.pyx", line 701, in cupy._core._cub_reduction._try_to_call_cub_reduction
  File "cupy\_core\_cub_reduction.pyx", line 552, in cupy._core._cub_reduction._launch_cub
  File "cupy\_util.pyx", line 64, in cupy._util.memoize.decorator.ret
  File "cupy\_core\_cub_reduction.pyx", line 246, in cupy._core._cub_reduction._SimpleCubReductionKernel_get_cached_function
  File "cupy\_core\_cub_reduction.pyx", line 231, in cupy._core._cub_reduction._create_cub_reduction_function
  File "cupy\_core\core.pyx", line 2251, in cupy._core.core.compile_with_cache
  File "D:\CTRec_Cupy\lib\site-packages\cupy\cuda\compiler.py", line 496, in _compile_module_with_cache
    return _compile_with_cache_cuda(
  File "D:\CTRec_Cupy\lib\site-packages\cupy\cuda\compiler.py", line 539, in _compile_with_cache_cuda
    base = _preprocess('', options, arch, backend)
  File "D:\CTRec_Cupy\lib\site-packages\cupy\cuda\compiler.py", line 434, in _preprocess
    result = compile_using_nvcc(source, options, arch, 'preprocess.cu',
  File "D:\CTRec_Cupy\lib\site-packages\cupy\cuda\compiler.py", line 376, in compile_using_nvcc
    raise cex
cupy.cuda.compiler.CompileException: `nvcc` command returns non-zero exit status.
command: ['D:\\Software\\CUDA12.3\\bin\\nvcc.EXE', '-gencode=arch=compute_50,code=sm_50', '--ptx', '--std=c++11', '-ID:\\CTRec_Cupy\\lib\\site-packages\\cupy\\_core\\include', '-ID:\\CTRec_Cupy\\lib\\site-packages\\cupy\\_core\\include\\cupy\\_cuda\\cuda-12', '-ID:\\Software\\CUDA12.3\\include', '-ftz=true', 'C:\\Users\\14682\\AppData\\Local\\Temp\\tmp8mzyyiow\\preprocess.cu']
return-code: 1
stdout/stderr:
nvcc fatal   : Cannot find compiler 'cl.exe' in PATH

Installation
Source (pip install cupy)
Environment
nvcc -V

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Fri_Nov__3_17:51:05_Pacific_Daylight_Time_2023
Cuda compilation tools, release 12.3, V12.3.103
Build cuda_12.3.r12.3/compiler.33492891_0

python -c "import cupy; cupy.show_config()"

OS                           : Windows-10-10.0.19045-SP0
Python Version               : 3.9.10
CuPy Version                 : 12.3.0
CuPy Platform                : NVIDIA CUDA
NumPy Version                : 1.26.2
SciPy Version                : 1.11.4
Cython Build Version         : 0.29.36
Cython Runtime Version       : None
CUDA Root                    : D:\Software\CUDA12.3
nvcc PATH                    : D:\Software\CUDA12.3\bin\nvcc.EXE
CUDA Build Version           : 12020
CUDA Driver Version          : 12030
CUDA Runtime Version         : 12030
cuBLAS Version               : (available)
cuFFT Version                : 11012
cuRAND Version               : 10304
cuSOLVER Version             : (11, 5, 4)
cuSPARSE Version             : (available)
NVRTC Version                : (12, 3)
Thrust Version               : 200101
CUB Build Version            : 200101
Jitify Build Version         : 4a37de0
cuDNN Build Version          : (not loaded; try `import cupy.cuda.cudnn` first)
cuDNN Version                : (not loaded; try `import cupy.cuda.cudnn` first)
NCCL Build Version           : None
NCCL Runtime Version         : None
cuTENSOR Version             : None
cuSPARSELt Build Version     : None
Device 0 Name                : NVIDIA GeForce GTX 950M
Device 0 Compute Capability  : 50
Device 0 PCI Bus ID          : 0000:01:00.0

Additional Information
No response

okay, my bad, i add cl.exe to system path, and restart pycharm, then it works.
but i confused why this will happen? and i use cupy and numba.cuda well before.

Hmmm @kmaehashi has MSVC been a run-time requirement on Windows?

Hmmm @kmaehashi has MSVC been a run-time requirement on Windows?

It's no longer the case in CuPy v13 (#7869). The author looks using CuPy v12:

CuPy Version                 : 12.3.0

It's unclear why cl.exe detection failed, though.

The author looks using CuPy v12:

CuPy Version                 : 12.3.0

Good catch! @leslielee619 plz try to update your CuPy version to v13 and see if you still hit issues.


Add option to replace spaces in filename with UUID sep

Proposed change
Adds a new configuration option filename_replace_spaces that replaces spaces in the filename with uuid_sep when the title is used within the filename generation. This is done to avoid spaces in filenames, which often need to be escaped on unix-like systems. The option defaults to false to avoid any changes for existing users.

Type of change

[ ] Bugfix (non-breaking change which fixes an issue)
[x] New feature (thank you!)
[ ] Code quality improvements to existing code or addition of tests
[ ] Documentation update

Additional information

This PR fixes or closes issue: N/A
This PR is related to issue: N/A

Checklist

[x] I am running the latest version of the plugin.
[x] The code change is tested and works locally.
[x] There is no commented out code in this PR.
[x] The code has been formatted using Stylua (a .stylua.toml file is provided)
[x] The code has been checked with luacheck (a .luacheckrc file is provided)
[x] The README.md has been updated according to this change.
[x] The doc/telekasten.txt helpfile has been updated according to this change.

Good point! I'll