feat: enable pubsub via settings menu

Part of https://github.com/ipfs-shipyard/ipfs-desktop/issues/1647

Adds a menu preference option to enable pubsub which adds a flag to the ipfsd-ctl options for startup.
Question: should this prompt a restart, given that the setting won't take effect until ipfs has been been restarted?
Bug: I'm getting the enabled/disabled message logged twice when clicking on the menu item, not quite sure why :
$ npm start

><EMAIL_ADDRESS>start
> cross-env NODE_ENV=development electron .

2021-01-19T16:19:06.988Z info: [meta] logs can be found on /Users/andrewnesbitt/Library/Application Support/IPFS Desktop
2021-01-19T16:19:10.560Z info: [web ui] window ready
2021-01-19T16:19:10.560Z info: [tray] starting
2021-01-19T16:19:10.607Z info: [tray] started
2021-01-19T16:19:10.608Z info: [ipfsd] start daemon STARTED
2021-01-19T16:19:10.937Z info: [daemon] removing api file
2021-01-19T16:19:11.954Z info: [ipfsd] start daemon FINISHED 1.3460042808055879s
2021-01-19T16:19:11.957Z info: [launch on startup] unavailable during development
2021-01-19T16:19:11.972Z info: [pubsub] disabled
2021-01-19T16:19:11.975Z info: [ipfs on path] no action taken
2021-01-19T16:19:17.770Z info: [pubsub] enabled
2021-01-19T16:19:17.791Z info: [pubsub] enabled
2021-01-19T16:22:36.969Z info: [pubsub] disabled
2021-01-19T16:22:37.000Z info: [pubsub] disabled

@lidel, can you please have a look as you're able?

@lidel, can you please have a look as you're able?

What if we enabled pubsub by default 🙀 💞

What if we enabled pubsub by default 🙀 💞

@olizilla   if we do it here, we also should enable it in Brave.
Meanwhile, on the pubsub track there are open issues, would you pull the lever? :trollface:        :steam_locomotive: :train::train::train:
I'd rather wait for https://github.com/ipfs/go-ipfs/issues/5383 and https://github.com/ipfs/go-ipfs/issues/6621   :sweat_smile:

@olizilla   if we do it here, we also should enable it in Brave.
Meanwhile, on the pubsub track there are open issues, would you pull the lever? :trollface:        :steam_locomotive: :train::train::train:
I'd rather wait for https://github.com/ipfs/go-ipfs/issues/5383 and https://github.com/ipfs/go-ipfs/issues/6621   :sweat_smile:

Wow, thanks for the updates @lidel, I was only expecting some pointers, not just for you to finish the pr!
Tested on macOS, working as expected, restarting correctly and such 🎉

Wow, thanks for the updates @lidel, I was only expecting some pointers, not just for you to finish the pr!
Tested on macOS, working as expected, restarting correctly and such 🎉

@andrew I simplified it a bit (run out of coffee yesterday), mind checking again?
I want to ship this this or next week because of the bug we've  fixed in this PR :)

@andrew I simplified it a bit (run out of coffee yesterday), mind checking again?
I want to ship this this or next week because of the bug we've  fixed in this PR :)

Retested on macOS, still looking good 👍

Retested on macOS, still looking good 👍


docs: add copy button to spectron and devtron code blocks

See #4617

Thanks for catching this! I was able to reproduce this on Firefox. It looks like the Copy button is trying to display a tooltip, but it gets trapped inside the code box and overflows it. I changed some CSS properties for the pre and code tags to fix this so that the tooltip will display (overflow: visible), but the actual code will still be scrollable (overflow: auto) if the window gets very narrow.

Thanks for reviewing. That's unfortunate that it's still messed up. What's your test setup? I can't reproduce that visual on Firefox, Safari, or Chrome on MacOS. This is what it's looking like in Firefox for me when hovering on one of the copy buttons:

@vhashimotoo Thanks for reviewing the code in such detail. I just made the changes you suggested.


Error recovery options

Trying to parse JSON from an external source, I found that in case of malformed input the json decoder aborts with error, and further graph processing ends.
Is there any way to prevent such behaviour and at most stop processing current event but continue with the following? I tried to put JSON parsing to the end of handling so it should at least keep the main data in already saved entity, and just don't save parsed results into another one. But no luck.
From what I've found, try/catch still is not implemented in AS. Do you have any recommendations on how to handle this case? Or it is unsafe to use json parser in general if you aren't 100% sure the data are correct?

Just found that in the last release this resolved here: https://github.com/graphprotocol/graph-node/issues/1577.
But the question is generic about other error handling. So I don't close it.

Just tried json.try_fromBytes, no luck.
This works well (well-formed JSON data):
 let jo = json.fromBytes(bytes).toObject()

And this returns a standard error Mapping terminated before finishing to handle Ethereum event with no more info even with RUST_LOG: debug:
let result = json.try_fromBytes(bytes)

Error happens on exactly this line, everything below could be commented out, doesn't matter.

@osnwt We will need a full reproduction subgraph in order to debug that issue, please reach out on our Discord so we can more easily figure out what's going on there.

Sorry for being inactive, I am busy last days. I will try to prepare a reproducible example ASAP.

Closing due to inactivity, please reopen if still an issue.

@osnwt Same problem here, did you figure it out?


add note to test utils renderIntoDocument

Adding clarification because (as noted in the source) it's a slight misnomer and can be confusing.

Yikes, I just realized that ReactDOM.render returns null if you're rendering a function component 😱 That makes this method pretty unreliable for my use cases :-(

I'm going to close this, but we're likely going to revisit this in some fashion anyway. Besides, I suppose your own lib does a smashing job of this now :P


Improve error feedback

Currently if the user mis-configures something, only a small text is shown in the bottom status bar. It would be good if we could provide more obvious error messages to the user.

Can we consider this fixed by 2f8a753533f990f2543ffcde89fa1212e1322ac3? Or is there more to expect?
I am currently trying to implement an action which will provide the raw output for the current file similar to the Show Clang-Tidy Info action CLion offers. I am also considering doing this for --debug output so it is easier for people to figure out what is going on as well as reporting bugs upstream to Cppcheck. Won't hit a PR until I have finished the XML and several other fixes though.
Sorry I missed this ticket in my commit.

Yes, I think we can consider this fixed. Thanks.


Cant get this to work on product thumbnails

I have tried using your solution for image zooming for my product thumbnails.
https://store.shaka-surf.com/products/eco-medium-g5-surf-fins
But it's not working. Any ideas?

@domtaylor Sorry for the late reply.
are you using gatsby-remark-images in project? gatsby-remark-images

@JaeYeopHan no worries. I'm using both gatsby-remark-images, gatsby-transformer-ramark, gatsby-remark-images-medium-zoom in my project.

@domtaylor oh i see....

can i see the your project source code? (github link)
are you using webp format in gatsby-remart-images?

@JaeYeopHan just sent you an invite.
2. No. I will add that in gatsby-config.js now see if it helps

@domtaylor
check this line plz, https://github.com/domtaylor/gatsby-demo-store/blob/master/gatsby-config.js#L57
(you'r using webp format :) )
Currently, this plugin do not support webp image foramt. I'm sorry I can't help you.

@JaeYeopHan I just added this, before i wasn't and it still wasn't working.

@JaeYeopHan thanks for looking at the repo. I just added this line, because I thought it needed to be webp format, but before I wasn't and it was not working as well. So I think it's something else.


Add logos from impurepics

Adds the logo images and adds one to the doc main page and the README.
@alexarchambault I've used impure-logos-almond-1.svg here but you can just change the filename to play with the variants if you prefer another.

I may try with impure-logos-almond-0.svg, yeah. impure-logos-almond-1.svg feels more for stickers to me.

I prefer this way personnally. It shows off the ~smiling almond more.


Add required fields attributes

Some dashboard.promotions views were missing a fields attribute. For Django 1.8 compatibilty.
Fixes #1754

Thanks :smiley:

Ace! My first ever pull request and it got merged very quickly. Thanks @mvantellingen  :)


Add context menu to open a repository with an external tool (such as GitExtensions)

Currently, RepoZ allows opening external tools (VS Code, Windows Terminal, File Explorer) with the location of the selected repository. It would be nice if there was an option to open a custom tool. In my case, I'm using GitExtentions extensively and would love to be able to configure that as an application I could open from RepoZ context menu.

Hey Sean,
you are not alone. Patrik, Glenn & me were discussing this a few months ago in issue #114.
This is coming, I'm pretty sure. Please give me some time, I did not find a slot to work on this.

Let me close this in favor for issue #114.
Feel free to reopen if this is not what you expected.

@awaescher, yes, exactly that. The title of that issue doesn't do justice to the thread 🙂
Thank you for pointing to #114 and will up-vote it.

Thanks for the hint, I updated the title.


Invalid CSV Class File Error

Hi,
While evaluating the prebuilt model exactly as per the documentation, I am able to do everything as stated in the example here: https://deepforest.readthedocs.io/en/latest/Example.html
However, while computing the performance/ accuracy, I am getting following error:
There are 1 unique labels: ['Tree']
Disabling snapshot saving
ValueError                                Traceback (most recent call last)
 in
1 model.config["save_path"] = currentpath
----> 2 mAP = model.evaluate_generator(annotations=training_chips)
3 print("Mean Average Precision is: {:.3f}".format(mAP))
C:\ProgramData\Anaconda3\lib\site-packages\deepforest\deepforest.py in evaluate_generator(self, annotations, comet_experiment, iou_threshold, max_detections)
228             image_max_side=args.image_max_side,
229             config=args.config,
--> 230             shuffle_groups=False,
231         )
232
C:\ProgramData\Anaconda3\lib\site-packages\keras_retinanet\preprocessing\csv_generator.py in init(self, csv_data_file, csv_class_file, base_dir, **kwargs)
145                 self.classes = _read_classes(csv.reader(file, delimiter=','))
146         except ValueError as e:
--> 147             raise_from(ValueError('invalid CSV class file: {}: {}'.format(csv_class_file, e)), None)
148
149         self.labels = {}
C:\ProgramData\Anaconda3\lib\site-packages\six.py in raise_from(value, from_value)
ValueError: invalid CSV class file: mypath\classes.csv: line 2: format should be 'class_name,class_id'
Any guideline is appreciated. Is it a version or compatibility issue?
Thanks.

Thanks for the report.
I have not seen this error before, its happening upstream of DeepForest in keras-retinanet. Did you create your own classes.csv file, or using the one automatically generated from DeepForest?
I am not able to reproduce immediately, but I will investigate. I see you are on windows. Can you specify full python version and drop the entire code in the issue?
Python 3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 13:42:17)
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> from matplotlib import pyplot as plt
>>> from deepforest import deepforest
Using TensorFlow backend.
>>> from deepforest import utilities
>>> from deepforest import get_data
>>> from deepforest import preprocess
>>>
>>> #convert hand annotations from xml into retinanet format
... YELL_xml = get_data("2019_YELL_2_541000_4977000_image_crop.xml")
>>> annotation = utilities.xml_to_annotations(YELL_xml)
>>> annotation.head()
                                   image_path  xmin  ymin  xmax  ymax label
0  2019_YELL_2_541000_4977000_image_crop.tiff  1012   161  1041   196  Tree
1  2019_YELL_2_541000_4977000_image_crop.tiff   106    92   139   137  Tree
2  2019_YELL_2_541000_4977000_image_crop.tiff   558   688   601   723  Tree
3  2019_YELL_2_541000_4977000_image_crop.tiff    18   378    53   422  Tree
4  2019_YELL_2_541000_4977000_image_crop.tiff     2   456    36   509  Tree
>>>
>>> #Write converted dataframe to file alongside the provided example images
... eval_file = "eval_example.csv"
>>> annotation.to_csv(eval_file, index=False)
>>> YELL_test = get_data("2019_YELL_2_541000_4977000_image_crop.tiff")
>>>
>>> #Crop and save images in current directory
... crop_dir = os.getcwd()
>>> cropped_annotations= preprocess.split_raster(path_to_raster=YELL_test,
...                                  annotations_file=eval_file,
...                                  base_dir=crop_dir,
...                                  patch_size=400,
...                                  patch_overlap=0.05)
>>> #View output
... cropped_annotations.head()
                                     image_path  xmin  ymin  xmax  ymax label
1   2019_YELL_2_541000_4977000_image_crop_0.png   106    92   139   137  Tree
3   2019_YELL_2_541000_4977000_image_crop_0.png    18   378    53   400  Tree
30  2019_YELL_2_541000_4977000_image_crop_0.png   273     1   335    52  Tree
31  2019_YELL_2_541000_4977000_image_crop_0.png   333     1   389    37  Tree
32  2019_YELL_2_541000_4977000_image_crop_0.png   128   299   158   337  Tree
>>>
>>> #Write window annotations file without a header row, same location as the "base_dir" above.
... eval_annotations_file= "cropped_example.csv"
>>> cropped_annotations.to_csv(eval_annotations_file,index=False, header=None)
>>> test_model = deepforest.deepforest()
Reading config file: deepforest_config.yml
A blank deepforest object created. To perform prediction, either train or load an existing model.
>>> test_model.use_release()
Model from DeepForest release https://github.com/weecology/DeepForest/releases/tag/v0.2.1 was already downloaded. Loading model from file.
Loading pre-built model: https://github.com/weecology/DeepForest/releases/tag/v0.2.1
WARNING:tensorflow:From /Users/ben/miniconda3/envs/DeepForest/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:4070: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.

2020-04-06 07:03:44.398861: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
tracking <tf.Variable 'Variable:0' shape=(9, 4) dtype=float32> anchors
tracking <tf.Variable 'Variable_1:0' shape=(9, 4) dtype=float32> anchors
tracking <tf.Variable 'Variable_2:0' shape=(9, 4) dtype=float32> anchors
tracking <tf.Variable 'Variable_3:0' shape=(9, 4) dtype=float32> anchors
tracking <tf.Variable 'Variable_4:0' shape=(9, 4) dtype=float32> anchors
WARNING:tensorflow:From /Users/ben/Documents/DeepForest/keras_retinanet/backend/tensorflow_backend.py:104: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
>>>
>>> test_model.config["save_path"] = "."
>>> mAP = test_model.evaluate_generator(annotations=eval_annotations_file)
There are 1 unique labels: ['Tree']
Disabling snapshot saving
Running network: N/A% (0 of 12) |                                                                                | Elapsed Time: 0:00:00 ETA:  --:--:--WARNING:tensorflow:From /Users/ben/miniconda3/envs/DeepForest/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:422: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.

Running network: 100% (12 of 12) |#############################################################################################| Elapsed Time: 0:00:22 Time:  0:00:22
Parsing annotations: 100% (12 of 12) |#########################################################################################| Elapsed Time: 0:00:00 Time:  0:00:00
431 instances of class Tree with average precision: 0.5670
mAP using the weighted average of precisions among classes: 0.5670
mAP: 0.5670
>>> print("Mean Average Precision is: {:.3f}".format(mAP))
Mean Average Precision is: 0.567

Thanks for the report.
I have not seen this error before, its happening upstream of DeepForest in keras-retinanet. Did you create your own classes.csv file, or using the one automatically generated from DeepForest?
I am not able to reproduce immediately, but I will investigate. I see you are on windows. Can you specify full python version and drop the entire code in the issue?
Python 3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 13:42:17)
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> from matplotlib import pyplot as plt
>>> from deepforest import deepforest
Using TensorFlow backend.
>>> from deepforest import utilities
>>> from deepforest import get_data
>>> from deepforest import preprocess
>>>
>>> #convert hand annotations from xml into retinanet format
... YELL_xml = get_data("2019_YELL_2_541000_4977000_image_crop.xml")
>>> annotation = utilities.xml_to_annotations(YELL_xml)
>>> annotation.head()
                                   image_path  xmin  ymin  xmax  ymax label
0  2019_YELL_2_541000_4977000_image_crop.tiff  1012   161  1041   196  Tree
1  2019_YELL_2_541000_4977000_image_crop.tiff   106    92   139   137  Tree
2  2019_YELL_2_541000_4977000_image_crop.tiff   558   688   601   723  Tree
3  2019_YELL_2_541000_4977000_image_crop.tiff    18   378    53   422  Tree
4  2019_YELL_2_541000_4977000_image_crop.tiff     2   456    36   509  Tree
>>>
>>> #Write converted dataframe to file alongside the provided example images
... eval_file = "eval_example.csv"
>>> annotation.to_csv(eval_file, index=False)
>>> YELL_test = get_data("2019_YELL_2_541000_4977000_image_crop.tiff")
>>>
>>> #Crop and save images in current directory
... crop_dir = os.getcwd()
>>> cropped_annotations= preprocess.split_raster(path_to_raster=YELL_test,
...                                  annotations_file=eval_file,
...                                  base_dir=crop_dir,
...                                  patch_size=400,
...                                  patch_overlap=0.05)
>>> #View output
... cropped_annotations.head()
                                     image_path  xmin  ymin  xmax  ymax label
1   2019_YELL_2_541000_4977000_image_crop_0.png   106    92   139   137  Tree
3   2019_YELL_2_541000_4977000_image_crop_0.png    18   378    53   400  Tree
30  2019_YELL_2_541000_4977000_image_crop_0.png   273     1   335    52  Tree
31  2019_YELL_2_541000_4977000_image_crop_0.png   333     1   389    37  Tree
32  2019_YELL_2_541000_4977000_image_crop_0.png   128   299   158   337  Tree
>>>
>>> #Write window annotations file without a header row, same location as the "base_dir" above.
... eval_annotations_file= "cropped_example.csv"
>>> cropped_annotations.to_csv(eval_annotations_file,index=False, header=None)
>>> test_model = deepforest.deepforest()
Reading config file: deepforest_config.yml
A blank deepforest object created. To perform prediction, either train or load an existing model.
>>> test_model.use_release()
Model from DeepForest release https://github.com/weecology/DeepForest/releases/tag/v0.2.1 was already downloaded. Loading model from file.
Loading pre-built model: https://github.com/weecology/DeepForest/releases/tag/v0.2.1
WARNING:tensorflow:From /Users/ben/miniconda3/envs/DeepForest/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:4070: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.

2020-04-06 07:03:44.398861: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
tracking <tf.Variable 'Variable:0' shape=(9, 4) dtype=float32> anchors
tracking <tf.Variable 'Variable_1:0' shape=(9, 4) dtype=float32> anchors
tracking <tf.Variable 'Variable_2:0' shape=(9, 4) dtype=float32> anchors
tracking <tf.Variable 'Variable_3:0' shape=(9, 4) dtype=float32> anchors
tracking <tf.Variable 'Variable_4:0' shape=(9, 4) dtype=float32> anchors
WARNING:tensorflow:From /Users/ben/Documents/DeepForest/keras_retinanet/backend/tensorflow_backend.py:104: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
>>>
>>> test_model.config["save_path"] = "."
>>> mAP = test_model.evaluate_generator(annotations=eval_annotations_file)
There are 1 unique labels: ['Tree']
Disabling snapshot saving
Running network: N/A% (0 of 12) |                                                                                | Elapsed Time: 0:00:00 ETA:  --:--:--WARNING:tensorflow:From /Users/ben/miniconda3/envs/DeepForest/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:422: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.

Running network: 100% (12 of 12) |#############################################################################################| Elapsed Time: 0:00:22 Time:  0:00:22
Parsing annotations: 100% (12 of 12) |#########################################################################################| Elapsed Time: 0:00:00 Time:  0:00:00
431 instances of class Tree with average precision: 0.5670
mAP using the weighted average of precisions among classes: 0.5670
mAP: 0.5670
>>> print("Mean Average Precision is: {:.3f}".format(mAP))
Mean Average Precision is: 0.567

Thank you for the quick response. Meanwhile, I struggled for a few hours and was finally able to resolve it after line by line debugging of code. Here is the issue so you can address it appropriately. In my case the culprit was the classes.csv file which is automatically created by the test_model.evaluate_generator class which calls create_classes(annotations_file) from utilities.py. The reason is certainly Macintosh (CR) where it should be a Windows (CRLF). I guess a carriage return is added while creating the csv on windows systems. I am not sure but perhaps it was UTF-8 issue too. So I changed line 177 in utilities.py to
with open(classes_path,'w',  newline='\n', encoding='utf-8')
and it worked.
Best regards.

Thanks for your work on this. I will test it on mac and linux. Can you give
details of your windows and python build? DeepForest runs on azure
pipelines with a windows build for pypi, so this should have been caught in
tests if it was widespread. I'd like to add a dedicated test once I figure
out what build is causing this.
On Mon, Apr 6, 2020 at 11:04 AM aasem<EMAIL_ADDRESS>wrote:

Thanks for the report.
I have not seen this error before, its happening upstream of DeepForest in
keras-retinanet. Did you create your own classes.csv file, or using the one
automatically generated from DeepForest?
I am not able to reproduce immediately, but I will investigate. I see you
are on windows. Can you specify full python version and drop the entire
code in the issue?
Python 3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 13:42:17)
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import os
from matplotlib import pyplot as plt
from deepforest import deepforest
Using TensorFlow backend.
from deepforest import utilities
from deepforest import get_data
from deepforest import preprocess
#convert hand annotations from xml into retinanet format
... YELL_xml = get_data("2019_YELL_2_541000_4977000_image_crop.xml")
annotation = utilities.xml_to_annotations(YELL_xml)
annotation.head()
image_path  xmin  ymin  xmax  ymax label
0  2019_YELL_2_541000_4977000_image_crop.tiff  1012   161  1041   196  Tree
1  2019_YELL_2_541000_4977000_image_crop.tiff   106    92   139   137  Tree
2  2019_YELL_2_541000_4977000_image_crop.tiff   558   688   601   723  Tree
3  2019_YELL_2_541000_4977000_image_crop.tiff    18   378    53   422  Tree
4  2019_YELL_2_541000_4977000_image_crop.tiff     2   456    36   509  Tree
#Write converted dataframe to file alongside the provided example images
... eval_file = "eval_example.csv"
annotation.to_csv(eval_file, index=False)
YELL_test = get_data("2019_YELL_2_541000_4977000_image_crop.tiff")
#Crop and save images in current directory
... crop_dir = os.getcwd()
cropped_annotations= preprocess.split_raster(path_to_raster=YELL_test,
...                                  annotations_file=eval_file,
...                                  base_dir=crop_dir,
...                                  patch_size=400,
...                                  patch_overlap=0.05)
#View output
... cropped_annotations.head()
image_path  xmin  ymin  xmax  ymax label
1   2019_YELL_2_541000_4977000_image_crop_0.png   106    92   139   137  Tree
3   2019_YELL_2_541000_4977000_image_crop_0.png    18   378    53   400  Tree
30  2019_YELL_2_541000_4977000_image_crop_0.png   273     1   335    52  Tree
31  2019_YELL_2_541000_4977000_image_crop_0.png   333     1   389    37  Tree
32  2019_YELL_2_541000_4977000_image_crop_0.png   128   299   158   337  Tree
#Write window annotations file without a header row, same location as the "base_dir" above.
... eval_annotations_file= "cropped_example.csv"
cropped_annotations.to_csv(eval_annotations_file,index=False, header=None)
test_model = deepforest.deepforest()
Reading config file: deepforest_config.yml
A blank deepforest object created. To perform prediction, either train or load an existing model.
test_model.use_release()
Model from DeepForest release https://github.com/weecology/DeepForest/releases/tag/v0.2.1 was already downloaded. Loading model from file.
Loading pre-built model: https://github.com/weecology/DeepForest/releases/tag/v0.2.1
WARNING:tensorflow:From /Users/ben/miniconda3/envs/DeepForest/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:4070: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.

2020-04-06 07:03:44.398861: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
tracking <tf.Variable 'Variable:0' shape=(9, 4) dtype=float32> anchors
tracking <tf.Variable 'Variable_1:0' shape=(9, 4) dtype=float32> anchors
tracking <tf.Variable 'Variable_2:0' shape=(9, 4) dtype=float32> anchors
tracking <tf.Variable 'Variable_3:0' shape=(9, 4) dtype=float32> anchors
tracking <tf.Variable 'Variable_4:0' shape=(9, 4) dtype=float32> anchors
WARNING:tensorflow:From /Users/ben/Documents/DeepForest/keras_retinanet/backend/tensorflow_backend.py:104: add_dispatch_support..wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where

test_model.config["save_path"] = "."
mAP = test_model.evaluate_generator(annotations=eval_annotations_file)
There are 1 unique labels: ['Tree']
Disabling snapshot saving
Running network: N/A% (0 of 12) |                                                                                | Elapsed Time: 0:00:00 ETA:  --:--:--WARNING:tensorflow:From /Users/ben/miniconda3/envs/DeepForest/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:422: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.

Running network: 100% (12 of 12) |#############################################################################################| Elapsed Time: 0:00:22 Time:  0:00:22
Parsing annotations: 100% (12 of 12) |#########################################################################################| Elapsed Time: 0:00:00 Time:  0:00:00
431 instances of class Tree with average precision: 0.5670
mAP using the weighted average of precisions among classes: 0.5670
mAP: 0.5670

print("Mean Average Precision is: {:.3f}".format(mAP))
Mean Average Precision is: 0.567

Thank you for the quick response. Meanwhile, I struggled for a few hours
and was finally able to resolve it after line by line debugging of code.
Here is the issue so you can address it appropriately. In my case the
culprit was the classes.csv file which is automatically created by the
test_model.evaluate_generator class which calls
create_classes(annotations_file) from utilities.py. The reason is
certainly Macintosh (CR) where it should be a Windows (CRLF). I guess
a carriage return is added while creating the csv on windows systems. I am
not sure but perhaps it was UTF-8 issue too. So I changed line 177 in
utilities.py to
with open(classes_path,'w', newline='\n', encoding='utf-8')
and it worked.
Best regards.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/weecology/DeepForest/issues/101#issuecomment-609949750,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJHBLDRZRKON67A374L2VDRLIKSNANCNFSM4MCAVZJQ
.

--
Ben Weinstein, Ph.D.
Postdoctoral Fellow
University of Florida
http://benweinstein.weebly.com/

Here is the code to potentially reproduce
pip install DeepForest

import os
from matplotlib import pyplot as plt
from deepforest import deepforest
from deepforest import utilities
from deepforest import get_data
from deepforest import preprocess

#convert hand annotations from xml into retinanet format
YELL_xml = get_data("2019_YELL_2_541000_4977000_image_crop.xml")
annotation = utilities.xml_to_annotations(YELL_xml)
annotation.head()

#Write converted dataframe to file alongside the provided example images
eval_file = "eval_example.csv"
annotation.to_csv(eval_file, index=False)

#Find data on path
YELL_test = get_data("2019_YELL_2_541000_4977000_image_crop.tiff")

#Crop and save images in current directory
crop_dir = os.getcwd()
cropped_annotations= preprocess.split_raster(path_to_raster=YELL_test,
                                 annotations_file=eval_file,
                                 base_dir=crop_dir,
                                 patch_size=400,
                                 patch_overlap=0.05)
#View output
cropped_annotations.head()

#Write window annotations file without a header row, same location as the "base_dir" above.
eval_annotations_file= "cropped_example.csv"
cropped_annotations.to_csv(eval_annotations_file,index=False, header=None)

test_model = deepforest.deepforest()
test_model.use_release()

test_model.config["save_path"] = "."
mAP = test_model.evaluate_generator(annotations=eval_annotations_file)
print("Mean Average Precision is: {:.3f}".format(mAP))

Cannot reproduce on azure instance labeled
  pool: {vmImage: 'vs2017-win2016'}

@henrykironde  @ha0ye @juniperlsimonis do any of you have more up to date windows builds that could confirm the above snippet?

I found a windows version that errors.
(test) bweinstein@test C:\Users\bweinstein>python
Python 3.6.10 |Anaconda, Inc.| (default, Mar 23 2020, 17:58:33) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> from matplotlib import pyplot as plt
>>> from deepforest import deepforest
Using TensorFlow backend.
>>> from deepforest import utilities
>>> from deepforest import get_data
>>> from deepforest import preprocess
>>>
>>> #convert hand annotations from xml into retinanet format
... YELL_xml = get_data("2019_YELL_2_541000_4977000_image_crop.xml")
>>> annotation = utilities.xml_to_annotations(YELL_xml)
>>> annotation.head()
                                   image_path  xmin  ymin  xmax  ymax label
0  2019_YELL_2_541000_4977000_image_crop.tiff  1012   161  1041   196  Tree
1  2019_YELL_2_541000_4977000_image_crop.tiff   106    92   139   137  Tree
2  2019_YELL_2_541000_4977000_image_crop.tiff   558   688   601   723  Tree
3  2019_YELL_2_541000_4977000_image_crop.tiff    18   378    53   422  Tree
4  2019_YELL_2_541000_4977000_image_crop.tiff     2   456    36   509  Tree
>>>
>>> #Write converted dataframe to file alongside the provided example images
... eval_file = "eval_example.csv"
>>> annotation.to_csv(eval_file, index=False)
>>>
>>> #Find data on path
... YELL_test = get_data("2019_YELL_2_541000_4977000_image_crop.tiff")
>>>
>>> #Crop and save images in current directory
... crop_dir = os.getcwd()
>>> cropped_annotations= preprocess.split_raster(path_to_raster=YELL_test,
...                                  annotations_file=eval_file,
...                                  base_dir=crop_dir,
...                                  patch_size=400,
...                                  patch_overlap=0.05)
>>> #View output
... cropped_annotations.head()
                                     image_path  xmin  ymin  xmax  ymax label
1   2019_YELL_2_541000_4977000_image_crop_0.jpg   106    92   139   137  Tree
3   2019_YELL_2_541000_4977000_image_crop_0.jpg    18   378    53   400  Tree
30  2019_YELL_2_541000_4977000_image_crop_0.jpg   273     1   335    52  Tree
31  2019_YELL_2_541000_4977000_image_crop_0.jpg   333     1   389    37  Tree
32  2019_YELL_2_541000_4977000_image_crop_0.jpg   128   299   158   337  Tree
>>>
>>> #Write window annotations file without a header row, same location as the "base_dir" above.
... eval_annotations_file= "cropped_example.csv"
>>> cropped_annotations.to_csv(eval_annotations_file,index=False, header=None)
>>>
>>> test_model = deepforest.deepforest()
Reading config file: C:\Miniconda\envs\test\lib\site-packages\deepforest\data\deepforest_config.yml
A blank deepforest object created. To perform prediction, either train or load an existing model.
>>> test_model.use_release()
Downloading model from DeepForest release v0.2.1, see https://github.com/weecology/DeepForest/releases/tag/v0.2.1 for details
finetuned_weights_20191126_170710.h5: 146MB [00:03, 39.8MB/s]
Model was downloaded and saved to C:\Miniconda\envs\test\lib\site-packages\deepforest\data/NEON.h5
Loading pre-built model: https://github.com/weecology/DeepForest/releases/tag/v0.2.1
WARNING:tensorflow:From C:\Miniconda\envs\test\lib\site-packages\keras\backend\tensorflow_backend.py:4070: The name tf.nn.max_pool is deprecated. Please use tf.nn.max
_pool2d instead.

2020-04-06 18:49:09.468966: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: A
VX2
tracking <tf.Variable 'Variable:0' shape=(9, 4) dtype=float32> anchors
tracking <tf.Variable 'Variable_1:0' shape=(9, 4) dtype=float32> anchors
tracking <tf.Variable 'Variable_2:0' shape=(9, 4) dtype=float32> anchors
tracking <tf.Variable 'Variable_3:0' shape=(9, 4) dtype=float32> anchors
tracking <tf.Variable 'Variable_4:0' shape=(9, 4) dtype=float32> anchors
WARNING:tensorflow:From C:\Miniconda\envs\test\lib\site-packages\keras_retinanet\backend\tensorflow_backend.py:104: add_dispatch_support.<locals>.wrapper (from tensor
flow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
>>>
>>> test_model.config["save_path"] = "."
>>> mAP = test_model.evaluate_generator(annotations=eval_annotations_file)
There are 1 unique labels: ['Tree']
Disabling snapshot saving
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Miniconda\envs\test\lib\site-packages\deepforest\deepforest.py", line 230, in evaluate_generator
    shuffle_groups=False,
  File "C:\Miniconda\envs\test\lib\site-packages\keras_retinanet\preprocessing\csv_generator.py", line 147, in __init__
    raise_from(ValueError('invalid CSV class file: {}: {}'.format(csv_class_file, e)), None)
  File "<string>", line 3, in raise_from
ValueError: invalid CSV class file: classes.csv: line 2: format should be 'class_name,class_id'
>>> print("Mean Average Precision is: {:.3f}".format(mAP))

https://azuremarketplace.microsoft.com/en-us/marketplace/apps/microsoft-dsvm.dsvm-win-2019?tab=Overview

Fixed on azure. Passes tests
(test) bweinstein@test C:\Users\bweinstein\DeepForest>pytest tests
======================================================================== test session starts ========================================================================
platform win32 -- Python 3.6.10, pytest-5.4.1, py-1.8.1, pluggy-0.13.1
rootdir: C:\Users\bweinstein\DeepForest, inifile: setup.cfg
collected 26 items

tests\test_data.py .                                                                                [  3%]
tests\test_deepforest.py .........
...

pushing to pypi in next release. thanks @aasem, it should be cleaned up by the evening once azure pipelines run.


Error 500 when using middleware

Hi there,
I've seemingly found another issue, that may or may not be part of your domain, or possibly Google's. I figured I'd raise your awareness of it as well: https://github.com/firebase/firebase-tools/issues/6009
Seems that adding your middleware.ts somehow makes any deploys to Firebase hosting result in a status 500. The app runs fine locally.
If you can see any obvious reasons as to why, please do let me know, otherwise you can close this and we can see what the Google team come up with.
I've tested this across multiple different apps with different configurations and the error always occurs. There are no additional hints in the Google logs that could help us figure out what's going on unfortunately.

Hi,
thank you for responding so quickly. I have a feeling that troubleshooting may not be needed: https://github.com/FirebaseExtended/firebase-framework-tools/pull/83
I'll edit this comment and close the issue once the next version of Firebase tools is out, so we can verify this works now.


Rename SchemaType to DataType

Currently, we use SchemaType for types of columns in DDLs.
However, this kind of type is named data_type in the GoogleSQL document (here).
where data_type is:
    { scalar_type | array_type | proto_type_name }

Then, we need to rename SchemaType to DataType to avoid this inconsistency.

I am not sure it is better than now.
The documentation of query syntax and DDL syntax is separated, so they are written without concerning of confusion between them.
With SchemaType and Type, it's clear that the former is a type used in schemas.
However, with DataType and Type, that distinction becomes less clear and could lead to confusion.
ZetaSQL and Cloud Spanner Emulator don't use any name with data_type or DataType for SchemaType equivalent.
ZetaSQL: ASTColumnSchema
Cloud Spanner Emulator: Type (FYI: Cloud Spanner Emulator don't use ZetaSQL for DDL)
I think SchemaType is a good enough name without causing any confusion. If necessary, we can add a doc comment saying that the type corresponds to the data_type in the DDL reference.

Ok. I understandd.


Incompatibility issues with JaCoCo

From headcrashing on February 21, 2013 09:22:20
There seems to be an incompatibility issue between powermock and JaCoCo, possibly related to JDK 7.
Attached is a small maven project which can be executed using "mvn clean verify" to showcase the problem.
Depending on which type of classloading and which type of rule implementation is used, the result is different:

powermock-module-junit4-rule + powermock-classloading-xstream:

tests succeed (hence a.b.c.d1.A.getC() IS covered -- see test code)

target/site/jacoco/index.html a.b.c.d1.A.getC() says 0%

powermock-module-junit4-rule + powermock-classloading-objenesis

one test fails

java.lang.ArrayIndexOutOfBoundsException: 5 at a.b.c.d1.WhenNewTest.should MockConstructor (WhenNewTest.java)

powermock-module-junit4-rule-agent + powermock-classloading-objenesis

both tests fail

Cannot subclass final class a.b.c.d1.B
Expecting a stackmap frame at branch target 21 in method a.b.c.d2.H./La/b/c/d2/G;)V at offset 11

powermock-module-junit4-rule-agent + powermock-classloading-xstream

both tests fail

Cannot subclass final class a.b.c.d1.B
Expecting a stackmap frame at branch target 21 in method a.b.c.d2.H.(La/b/c/d2/G;)V at offset 11

Both, powermock and JaCoCo are really great tools and we appreciate the hard work of the authors, but unless both work fine together we really have the problem that we are totally stuck with our QA. Hence we really beg for a working solution! :-)
Attachment: powermockito-jacoco-failure.zip
Original issue: http://code.google.com/p/powermock/issues/detail?id=432

From headcrashing on February 22, 2013 10:46:29
Seems to be realted to #402.

From<EMAIL_ADDRESS>on April 22, 2014 05:39:02
This tracker item was not touched for over one year. Meanwhile Javassist and JaCoCo provide support for Java 7, and Oracle published Java 8. It would be great if one of the PowerMock authors could post some roadmap when PowerMock will work together with JaCoCo on Java 7. :-)


Coverage report run in virtual environment via PDM

Description
Coverage report run in virtual environment via PDM
Type of change

[ ] Refactor
[ ] New feature
[ ] Bug fix
[ ] CVE fix
[ ] Optimization
[ ] Documentation Update
[x] Configuration Update
[ ] Bump-up dependent library

/approve
/lgtm


For Retrievable Tasks, only block until "obj" is not null

Is your feature request related to a problem? Please describe.
Currently Retrievable Tasks block until the execute() method returns. However, this can be problematic. For example, what if I would like to return the value back to the caller but still have further processing to do inside of execute()?
Describe the solution you'd like
Either an expanded contract for RetrievableTask.java with two separate (or overloaded) execute() methods OR an extension of RetrievableTask with only this variation of the execute() method.
Describe alternatives you've considered
Writing an additional semaphore, but isn't the point of the library to avoid users doing this?

Could have a boolean param in the RetrievableTask constructor w/ a private internal class that returns back an ObjectBlockingRetrievableTask versus a MethodBlockingRetrievableTask (default)

This issue is superseded by  #15

This ticket will remain open as a reference #15 but all further comments are closed.

Ticket Status
This ticket's status has been updated to: CLOSED FOR COMMENTS
Remarks
Please refer to the previous comment.

Development in issue-2 has made major improvements. There are now two types of RetrievableTasks:

ObjectBlockingRetrievableTask
MethodBlockingRetrievableTasks
In addition, the "obj" has been set to private, and now the only way to set its value is via a setVal() method which has been mandated (via the abstract modifier).

Development is now shifting towards writing the unit and functional tests necessary before this can be rolled out into a development release.


Escape translations script in the code blocks

See -> https://github.com/Manvel/cmints/issues/99
this is the ticket to address website part of the implementation.

Implemented in https://github.com/Manvel/cmints-website/issues/61


Always return the backlog quota for a namespace in rest response

Motivation
There is no easy way to get the backlog for namespace if it uses the default. This change fills the backlog quota for a namespace even if it has not been set. The cluster default is used in that case.
Modifications
Changed the get namespace endpoint to hydrate the backlog quota.
Result
The namespace returned from the admin rest api will always have a backlog quota.

rerun integration tests

rerun cpp tests


output with text is not sorted.

json output is sorted and always same.
But output with text is not sorted.
vaku folder read my-path/ -T -o text

In the API the only sorted output is anything that returns a []string (pathList, folderSearch). With the CLI the json output is sorted when I marshall the API output into JSON, but with text output I just loop over the results and print them (here). I agree it would be nice for CLI results to be sorted by default, but it's not something I'm going to add. Feel free to submit a PR and I'll be happy to review. Thanks!

I'm working on a version 2 of vaku that will include a cli option for sorting output. SHould be ready in ~1 month but timeline could change.

Version 2 was just release and sorting is now turned on by default. Thanks for filing this issue 😄


Fix code example in testing guide

Fix a duplicated variable definition and missing one in a testing guide code example.

Thanks!


Windows: executables built with dbg and fastbuild compilation modes crash on startup with failed absl assertion

Description:
Building envoy-static.exe and many (I don't believe it is all, but have not narrowed it down yet) test executables on Windows with fastbuild or debug compilatin modes causes a crash on startup due to a failed assertion:
Assertion failed: false && "*next_sample >= 1", file external/com_google_absl/absl/container/internal/hashtablez_sampler.cc, line 202

Repro steps:
Build //source/exe/envoy-static or tests with -c dbg or -c fastbuild on Windows

Currently in CI etc. we are only building opt mode builds

Looks like an https://github.com/abseil/abseil-cpp issue.

I'm told it is fixed upstream.  We need to update to latest abseil.
Relevant change:
https://github.com/abseil/abseil-cpp/commit/62f05b1f57ad660e9c09e02ce7d591dcc4d0ca08#diff-d39294eb126ec74b09a131e2ef19918d

thanks @antoniovicente good call and good find 👍 we can test out bumping abseil and ensure it fixes the issue

Did it work?  We should bump up abseil version on the main repo.

Yes it did, however we ran into some issues with linking other foreign cc dependencies (luajit and moonjit) when using dbg mode. We are locally testing a PR to disable them for the time being as they are only used in extensions so we can have a change set that enables dbg builds in one shot


:bookmark: Version 4.13.0

Description

Related Issue

How Has This Been Tested

Types of changes

[ ] Docs change / refactoring / dependency upgrade
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Requires a change to the official Guide documentation.

why are tests failing?

nevermind, server side issue


Integration Test - Release - 2019/02/28 17:55:52

Auto create for integration test. IssueCommandFlow

#pri1

Just type '#backlog'

#pri2

#backlog

#please-close

@PRMergerTest3 - This issue has in the backlog for 80 days. In 10 days, it will be automatically closed.
For more information about the process by which we handle documentation feedback, see "How we respond to Azure technical documentation feedback" in our public contributor guide.

The feedback you logged for Integration Test - Release - 2019/02/28 17:55:52 has been in our backlog for 90 days. While we try to respond individually to as many feedback items as possible, we are not able to respond to all of them. Since this item has been in the backlog for 90 days, we’re closing it.
For more information about the process by which we handle documentation feedback, see "How we respond to Azure technical documentation feedback" in our public contributor guide.


Can’t check logs in sunstone

/!\ To report a security issue please follow this procedure:
[https://github.com/OpenNebula/one/wiki/Vulnerability-Management-Process]
Description
When the log have message with apostrophe (es_ES) no show log
To Reproduce
you can add the line with accents in /var/log/one/$id.log (https://forum.opennebula.io/t/cant-check-logs/9002)
Expected behavior
show log without error
Details

Affected Component: [e.g. Sunstone, Scheduler or Storage]
Hypervisor: [e.g. KVM]
Version: [e.g. 5.4.12 or development]

Additional context
Add any other context about the problem here.

Progress Status

[ ] Branch created
[ ] Code committed to development branch
[ ] Testing - QA
[ ] Documentation
[ ] Release notes - resolved issues, compatibility, known issues
[ ] Code committed to upstream release/hotfix branches
[ ] Documentation committed to upstream release/hotfix branches

Closing it.
Reason: Can't reproduce it


Support all the switches?

Hi there, thanks for the fantastic work!
I'd like to use lein-zprint (rather than a standalone binary) so that it can be run automatically with a git hook or in CI without needing a separate installation step. That way we can enforce formatting without worrying about everyone's personal setup, and they can install a binary only if desired.
Unfortunately the commandline flags don't seem to be supported:
14:55 $ lein zprint -lfsw *.clj
java.lang.Exception: Unknown switch '-lfsw'
 at leiningen.zprint$get_switch.invokeStatic (zprint.clj:116)
 ...

The -c flag would also be nice to have. :)

Yes, the leiningen binary and the uberjar (and standalone binaries) are different pieces of code.  They both use the basic zprint library, but are otherwise different (though similar) pieces of code.  The lein zprint code predates the uberjar code by several years.
The current lien zprint code and reporting is more or less like -lw in the prebuilt binaries.
What specifically are you wanting that I might be able to add to lien zprint?  It sounds like -c.
I'm torn between trying to extend lein zprint to process a few more switches that you need and adding some kind of call-out to the uberjar code that is already there.  Depends I think on what you want.  Things like -s and -f are probably pretty easy to add to lein zprint.  Possibly -c as well.
I'm open to doing something to get you what you want, once I know in more detail what it is.
Thanks!

Thanks so much for the reply. I'd specifically like to replace cljfmt in the following lines in our makefiles:
# Runs style checker
static/fmt-check:
  @ lein cljfmt check

# Runs style checker and fixes issues
static/fmt-fix:
  @ lein cljfmt fix

I think to replace that first command I'd need lein zprint -c or some other way to do a dry run.

I'll look at adding a -c switch to lein zprint.  It doesn't look too bad, but you never know...

I have the basics implemented, but I'm wondering how you interpret the output?  For the pre-built binaries, I emit the number of incorrectly formatted files as the exit status.  Thus, it exits successfully when they are all formatted correctly.
I can't figure out how to do that in leiningen.  I looked at cljfmt, and it looks like it output messages to stderr if some files are not formatted correctly, otherwise the messages come out to stdout.
What do you want done?
Here is what it does now, all to stdout:
projects/zprint % lein zprint -c proj.clj
Processing file: proj.clj
File: proj.clj was incorrectly formatted
1 file formatted incorrectly
√ projects/zprint % lein zprint proj.clj
Processing file: proj.clj
√ projects/zprint % lein zprint -c proj.clj
Processing file: proj.clj
All files formatted correctly.

I can’t believe you’ve already implemented this. Thank you! I would suggest following the example of cljfmt so that it will work as a drop-in replacement in our (and other’s) CI pipelines.
This is a side question/observation, but it seems that the only way to override user config (or even to use a separate config file) is to add a directive in their personal project.clj. The problem with this is that people are resistant to change, and it creates friction by requiring everyone to add a personal config file.
I’m just returning to Clojure after a few years in the NPM ecosystem, and the standard pattern there (i.e. with Prettier) is for config files to have “depth-first” priority by default so that formatting just works in different projects while remaining transparent to the user. Maybe setting ‘:search-config true’ in project.clj could accomplish this while preserving backward compatibility?

I'm confused by your side question.  I hear that you want to override user config, but to do that everyone has to add a personal config file?  That doesn't make any sense to me, so I'm not getting something here. I would like to fix the problem, but I'm still up in the air as to what the problem is.
I can say that, with recent versions of lein-zprint, you can indeed set :search-config? true in the :zprint key in your project.clj.  So that if there is a .zprintrc or .zprint.edn file in the project, it will get looked at.
I guess I'm confused as to what zprint configuration you want to have take precedence?  The user's personal ~/.zprintrc has the least precedence.  Next is any local .zprintrc or .zprint.edn files, and then the :zprint key in the project.clj, and then any options from the actual line containing lein zprint.  The :zprint key in the project.clj file is examined for :search-config? true before the other configuration options it contains are used.
But all of that may or may not be interesting.  I'd love to learn the details of what isn't working for you, if you wouldn't mind explaining in more detail.  Thanks!

with recent versions of lein-zprint, you can indeed set :search-config? true in the :zprint key in your project.clj

The user's personal ~/.zprintrc has the least precedence. Next is any local .zprintrc or .zprint.edn files, and then the :zprint key in the project.clj, and then any options from the actual line containing lein zprint. The :zprint key in the project.clj file is examined for :search-config? true before the other configuration options it contains are used.

This is exactly the behavior I had wanted/expected. Thanks for clarifying. I had been following the configuration section of the readme, but now I see that you describe using :search-config? true in project.clj further down the page. I'll give it a try to be sure, but it sounds good.

In release 1.2.4 you now have the -c switch.  It also exits with non-zero exit-status if any of them do not "check" correctly.  Try lein zprint :help for a bit more information.

Perfect, thanks!

I wish it were perfect.  Unfortunately, I had a dependency on a alpha version of zprint in the 1.2.4 version of lein-zprint.  I have just fixed it so that now there is a <IP_ADDRESS> version of lein-zprint, which should actually work.  Sorry about that!

No worries, I've had to finish up a project before coming back to this anyway. I'll let you know how it goes. 😄

Well, I finally was given some time to work on tech debt and got this into our project. It works perfectly as a drop-in replacement for clj-fmt and everyone seems happy. I've added it as a pre-commit hook to make sure everyone uses it, and for myself I use the cli version to format on save with Vim.
Thanks again!


Visually distinguish integer and text columns

It would be useful if I could tell from looking at the table page if a column was a integer or a text (or a float I guess?). This is particularly important for knowing if it safe to sort by that column.

Not sure the best way to do this. Here's an example using very subtle colour changes - but I'm worried it's not at all discoverable:

That's with #4c0077 for floats and #666 for integers.

I like the gray. The purple is a bit confusing.
I could have both integers and floats show up as gray?

Turns out 1.0 stored in a float column already displays like that:
echo '[{"foo": 1.0, "bar": 1}, {"foo": 2, "bar": 2}]' | sqlite-utils insert /tmp/db.db t -

I'm going to make all integers AND floats show up as #666666.

type-pk is a more accurate name than type-link - since people might assume that type-link refers to columns containing a link or a URL or similar.

Live demo: https://latest.datasette.io/fixtures/sortable

I don't like how the column headers themselves are no longer black in mobile view.

Deployed here: https://covid-19.datasettes.com/covid/economist_excess_deaths - and it helped me spot a bug! The floating point numbers there were being treated as strings:


Add Kubernetes deploy events

Add logzio-k8s-events sub chart version 0.0.3:

Sends Kubernetes deploy events logs.

Before merging please test next pages(we use data from configs frolder):

https://app-staging-us-east-1.internal.logz.io/#/dashboard/send-your-data/agent/new

https://app-staging-us-east-1.internal.logz.io/#/dashboard/integrations/collectors


x/sys/windows: uintptr(unsafe.Pointer) calls outside of syscalls

Documentation states:

(4) Conversion of a Pointer to a uintptr when calling syscall.Syscall.
The Syscall functions in package syscall pass their uintptr arguments directly to the operating system, which then may, depending on the details of the call, reinterpret some of them as pointers. That is, the system call implementation is implicitly converting certain arguments back from uintptr to pointer.
If a pointer argument must be converted to uintptr for use as an argument, that conversion must appear in the call expression itself

However, there are calls within /x/sys/windows that do not follow this requirement. For an example, this call is casting an unsafe.Pointer to uintptr then is then passed to here
The generated calls in windows/zsyscall_windows.go should likely accept unsafe.Pointer arguments instead of uintptr.
Examples
// from windows/svc/service.go
handle, err := windows.RegisterServiceCtrlHandlerEx(windows.StringToUTF16Ptr(theService.name), ctlHandlerCallback, uintptr(unsafe.Pointer(&theService)))

// from windows/zsyscall_windows.go
func RegisterServiceCtrlHandlerEx(serviceName *uint16, handlerProc uintptr, context uintptr) (handle Handle, err error) {
	r0, _, e1 := syscall.Syscall(procRegisterServiceCtrlHandlerExW.Addr(), 3, uintptr(unsafe.Pointer(serviceName)), uintptr(handlerProc), uintptr(context))
	handle = Handle(r0)
	if handle == 0 {
		err = errnoErr(e1)
	}
	return
}

CC @golang/runtime, @alexbrainman.

CC @golang/windows
I agree that this looks like a bug in x/sys/windows.  I don't know how hard it would be to fix.

@ianlancetaylor
In this instance, you have the global theService for the context that's being passed in and converted. That context ends up here.  You could ignore this context (and the conversion that could cause errors) and just use the global theService directly in this function.

This is a duplicated of #59687. IMO @alexbrainman CL supersedes the one submitted in #59687.

@ianlancetaylor
In this instance, you have the global theService for the context that's being passed in and converted. That context ends up here. You could ignore this context (and the conversion that could cause errors) and just use the global theService directly in this function.

I implemented @mguy-huntress suggestion in
https://go-review.googlesource.com/c/sys/+/591475
Hopefully it is good enough.
Alex


No package 'libsecret-1' found

$ npx glitch-deploy
Package libsecret-1 was not found in the pkg-config search path.
Perhaps you should add the directory containing `libsecret-1.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libsecret-1' found
gyp: Call to 'pkg-config --libs-only-l libsecret-1' returned exit status 1 while in binding.gyp. while trying to load binding.gyp
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/home/travis/.nvm/versions/node/v8.3.0/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:336:16)
gyp ERR! stack     at emitTwo (events.js:125:13)
gyp ERR! stack     at ChildProcess.emit (events.js:213:7)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:200:12)
gyp ERR! System Linux 4.15.0-1028-gcp
gyp ERR! command "/home/travis/.nvm/versions/node/v8.3.0/bin/node" "/home/travis/.nvm/versions/node/v8.3.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /home/travis/.npm/_npx/5194/lib/node_modules/glitch-deploy/node_modules/keytar
gyp ERR! node -v v8.3.0
gyp ERR! node-gyp -v v3.6.2
gyp ERR! not ok
(node:5194) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: Failed to launch chrome!
/home/travis/.npm/_npx/5194/lib/node_modules/glitch-deploy/node_modules/puppeteer/.local-chromium/linux-515411/chrome-linux/chrome: error while loading shared libraries: libgconf-2.so.4: cannot open shared object file: No such file or directory
TROUBLESHOOTING: https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md
(node:5194) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

You need to install libsecret, here is the command to install it on ubuntu:
sudo apt-get install libsecret-1-dev


Fix support for new DSP game version with new config entries

This should fix #17, as well as implementing #16

Added more config entries for new mechanisms implied in DSP new version.


[WIP] feature/rework-of-communities-api

OPENED community type renamed to OPEN.
Reworked community API is used to interact with the server.
'members' field removed from CommunityResponse.
Tests adjusted.

Codecov Report

Merging #60 into master will decrease coverage by 0.38%.
The diff coverage is 66.07%.

@@            Coverage Diff             @@
##           master      #60      +/-   ##
==========================================
- Coverage   72.64%   72.25%   -0.39%
==========================================
  Files          66       67       +1
  Lines        1627     1651      +24
  Branches      114      116       +2
==========================================
+ Hits         1182     1193      +11
- Misses        413      426      +13
  Partials       32       32

Impacted Files
Coverage Δ

src/app/projects/projects-new.component.ts
82.75% <ø> (-0.58%)
:arrow_down:

src/app/communities/community-type.enum.ts
100% <100%> (ø)
:arrow_up:

src/app/communities/community-role.enum.ts
100% <100%> (ø)

...d-communities/recommended-communities.component.ts
80% <100%> (ø)
:arrow_up:

src/app/communities/communities-new.component.ts
73.33% <100%> (ø)
:arrow_up:

src/app/communities/communities-edit.component.ts
72.17% <100%> (ø)
:arrow_up:

src/app/communities/communities.component.ts
67.18% <44.44%> (-3.79%)
:arrow_down:

src/app/communities/communities.service.ts
84.61% <64.7%> (-15.39%)
:arrow_down:

src/app/community-view/community-view.component.ts
72.3% <65.21%> (-2.24%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f53ffd7...fd3928d. Read the comment docs.


Adding Sphinx docs build check to Travis.

Also adding Vision usage doc to the main index since without it the Sphinx build fails.

Got an LGTM from @tseaver over chat


cannot create a tensor larger than 2GB

When I run python main.py --datasource=omniglot --metatrain_iterations=40000 --meta_batch_size=16 --update_batch_size=5 --num_classes=20 --update_lr=0.1 --num_updates=5 --logdir=logs/omniglot20way5shot/, it raised a ValueError: Cannot create a tensor proto whose content is larger than 2GB. I'd like to ask how to train the model on omniglot, 20-way, 5-shot?

For the omniglot experiments, I trained on 1-shot and evaluated on 5-shot.
To evaluate on 5-shot using a model trained on 1-shot, use the flags --train=False --update_batch_size=5 --train_update_batch_size=1
If you want to train a model on 5-shot, you can also reduce the meta-batch size so that it fits in memory.

OK, I see. Thank you very much! I reduced the meta_batch_size to 1 and num_updates to 1, but it still raised the same error. Also, I did these experiments several times and I got different results. It seems that there are some random factors, such as initialization of the network. How much of these random factors will have an effect on the results?

Generally, I have found the experiments to be fairly repeatable at
convergence (though learning speed at the beginning of training may vary).
What GPU are you using? You need a GPU with sufficient memory. I used a
Titan X, which has 12 GB.
On Jul 18, 2017 6:57 PM, "zfengwei"<EMAIL_ADDRESS>wrote:

OK, I see. Thank you very much! I reduced the meta_batch_size to 1 and
num_updates to 1, but it still raised the same error. Also, I did these
experiments several times and I got different results. It seems that there
are some random factors, such as initialization of the network. How much of
these random factors will have an effect on the results?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/cbfinn/maml/issues/3#issuecomment-316249842, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABMlAS-LQzAOFs3JvrbU5bXJYiuDw2Seks5sPWKDgaJpZM4Oa2vs
.

I also use a GPU with 12GB. It seems that this is the restriction of tensorflow. It cannot create a tensor larger than 2GB.

Ok. Is this the input tensor that the error is referring to? I don't think
that you should be hitting 2 GB with the input tensor with a meta-batch
size of 1. I would estimate that for 20-way classification, a meta-batch
size of 1, 5-shot, it should be 20*(5+15)(8080*3)*32 bits = 0.03 GB.
[The 5+15 is 5 images for calculating the gradient step and 15 images for
the post-update evaluation.]
Can you provide more information on the error?
On Tue, Jul 18, 2017 at 8:57 PM, zfengwei<EMAIL_ADDRESS>wrote:

I also use a GPU with 12GB. It seems that this is the restriction of
tensorflow. It cannot create a tensor larger than 2GB.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/cbfinn/maml/issues/3#issuecomment-316265862, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABMlATRtNZw22PQGYDSlsKQNqgcA2jCOks5sPX6hgaJpZM4Oa2vs
.

If you want to try decreasing the 15 (to as low as 1), you change this line:
https://github.com/cbfinn/maml/blob/master/main.py#L249
On Tue, Jul 18, 2017 at 9:09 PM, Chelsea Finn<EMAIL_ADDRESS>wrote:

Ok. Is this the input tensor that the error is referring to? I don't think
that you should be hitting 2 GB with the input tensor with a meta-batch
size of 1. I would estimate that for 20-way classification, a meta-batch
size of 1, 5-shot, it should be 20*(5+15)(8080*3)*32 bits = 0.03 GB.
[The 5+15 is 5 images for calculating the gradient step and 15 images for
the post-update evaluation.]
Can you provide more information on the error?
On Tue, Jul 18, 2017 at 8:57 PM, zfengwei<EMAIL_ADDRESS>wrote:

I also use a GPU with 12GB. It seems that this is the restriction of
tensorflow. It cannot create a tensor larger than 2GB.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/cbfinn/maml/issues/3#issuecomment-316265862, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABMlATRtNZw22PQGYDSlsKQNqgcA2jCOks5sPX6hgaJpZM4Oa2vs
.

Thank you for your reply! The error is as follows:
Generating filenames Traceback (most recent call last):   File "main.py", line 350, in <module>     main()   File "main.py", line 272, in main     image_tensor, label_tensor = data_generator.make_data_tensor()   File "/home/maml/data_generator.py", line 103, in make_data_tensor     filename_queue = tf.train.string_input_producer(tf.convert_to_tensor(all_filenames), shuffle=False)   File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 676, in convert_to_tensor     as_ref=False)   File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 741, in internal_convert_to_tensor     ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)   File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/constant_op.py", line 113, in _constant_tensor_conversion_function     return constant(v, dtype=dtype, name=name)   File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/constant_op.py", line 106, in constant     attrs={"value": tensor_value, "dtype": dtype_value}, name=name).outputs[0]   File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2506, in create_op     original_op=self._default_original_op, op_def=op_def)   File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1219, in __init__     "Cannot create a tensor proto whose content is larger than 2GB.") ValueError: Cannot create a tensor proto whose content is larger than 2GB.

Thank you for your reply! I think the error is caused by this line:
https://github.com/cbfinn/maml/blob/master/data_generator.py#L86
I reduced the num_total_batches to 100000 and I can train the model on 5-shot.

Ah, okay. Thanks for letting me know!
On Jul 19, 2017 12:35 AM, "zfengwei"<EMAIL_ADDRESS>wrote:

Thank you for your reply! I think the error is caused by this line:
https://github.com/cbfinn/maml/blob/master/data_generator.py#L86
http://url
I reduced the num_total_batches to 100000 and I can train the model on
5-shot.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/cbfinn/maml/issues/3#issuecomment-316298678, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABMlAe8Gkt63sWnIcPMyXJxmOLXHF8H2ks5sPbHFgaJpZM4Oa2vs
.


Parse APKs in parallel

It would be great if Flank could parse APKs (and retrieve smart flank data for them) in parallel. In my testing pipeline, I test many APKs in one flank invocation. It's gotten to a point where parsing APKs is actually taking longer than executing them on FTL. This is, in part, because every APK is parsed serially.
Describe the solution
It would be great if flank could parse the APKs in parallel

Hey @asadsalman
We already do it in parallel, please check code snippet

Closing it.


svm_train() in svmutil.py is not thread-safe

To be more exact, svm_set_print_string_function() in svm.cpp is not thread-safe, as it modifies a global function pointer svm_print_string. If you call svm_train() in multiple Python threads with '-q' option, it will try to convert print_null() to a C-style function pointer and assign it to svm_print_string in order to suppress the output.
However, the function pointer generated each time seems to point to different addresses, which makes it possible to corrupt svm_print_string and crash the program.
I suggest to make the printing function pointer independent in different function calls. A global print option is neither thread-safe nor practical.

There are a few other places where libsvm isn't thread safe.. Some are
because of historical reasons. We may handle some important ones in the
near future, though the plan is uncertain yet..
On 2020-04-30 19:03, wyykak wrote:

To be more exact, svm_set_print_string_function() in svm.cpp is not
thread-safe, as it modifies a global function pointer
svm_print_string. If you call svm_train() in multiple Python
threads with '-q' option, it will try to convert print_null() to a
C-style function pointer and assign it to svm_print_string in order
to suppress the output.
However, the function pointer generated each time seems to point to
different addresses, which makes it possible to corrupt
svm_print_string and crash the program.
I suggest to make the printing function pointer independent in
different function calls. A global print option is neither thread-safe
nor practical.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub [1], or unsubscribe
[2]. [ { "@context": "http://schema.org", "@type": "EmailMessage",
"potentialAction": { "@type": "ViewAction", "target":
"https://github.com/cjlin1/libsvm/issues/167", "url":
"https://github.com/cjlin1/libsvm/issues/167", "name": "View Issue" },
"description": "View this Issue on GitHub", "publisher": { "@type":
"Organization", "name": "GitHub", "url": "https://github.com" } } ]
Links:
[1] https://github.com/cjlin1/libsvm/issues/167
[2]
https://github.com/notifications/unsubscribe-auth/ABI3BHR7IMVMNRPIPF4IARLRPFLH7ANCNFSM4MVNGHBA

Thanks for your reply！
In fact I have tried to use libsvm together with multiprocessing in Python, but SVM models, as C structs, cannot be serialized and tranferred between processes.
Currently libsvm only supports saving models as ascii files. Maybe you could add an interface to save models as strings? Therefore multiprocessing will be a possible way to parallelize libsvm.


CI: Flutter Web is already available thru stable channel, removing CI step for activating it

:scroll: Description
CI: Flutter Web is already available thru the stable channel, removing CI step for activating it
:bulb: Motivation and Context
just a small improvement
:green_heart: How did you test it?
:pencil: Checklist

[X] I reviewed submitted code
[ ] I added tests to verify changes
[ ] I updated the docs if needed
[X] All tests passing
[X] No breaking changes

:crystal_ball: Next steps

Codecov Report

Merging #354 (e39e35d) into main (82ef2ab) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #354   +/-   ##
=======================================
  Coverage   90.00%   90.00%
=======================================
  Files           5        5
  Lines         170      170
=======================================
  Hits          153      153
  Misses         17       17

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 82ef2ab...e39e35d. Read the comment docs.


NuScenes Tracking: GT and Prediction track matching

I successfully ran tracking for the nuscenes dataset. I want to re-identify the tracked objects also by the original Instance_token that exists in nuscenes instead of the numerical tracking_id (Sort of matching between predicted and GT tracks/objects).
Could a distance comparison work in this case (similar to the tracking evaluation in nuscenes-devkit) or is it better to perform a bipartite matching between GT and Prediction ?

maybe distance based matching? Currently it seems to have a lot false positive boxes if we use low score threshold.


The timeout configuration of MapReduce does not work

env info：
jdk：1.8
redis：5.0.6
redission: 3.11.5
There are two data processing services here. When I use MapReduce to process data, the timeout (30 s) is set. However, when one of the services in the process of reducing data goes down unexpectedly, there is still no response from the map after the timeout, Does the data processed by the down service need to be reallocated to other machines? What is the specific meaning of this timeout? Is there something wrong with my use?
The following pictures are related information:
This is the normal data processing service:

This is a service that has been down unexpectedly:

This is the configuration code information:

This is the now time:

@mrniko   thanks

Default taskRetryInterval is 5 minutes, but for map-reduce it's just cancelled: https://github.com/redisson/redisson/blob/ba31cf09018f05ffa54f017e1027e45f59624ff2/redisson/src/main/java/org/redisson/mapreduce/CoordinatorTask.java#L109
This logic should be reconsidered.

Default taskRetryInterval is 5 minutes, but for map-reduce it's just cancelled: https://github.com/redisson/redisson/blob/ba31cf09018f05ffa54f017e1027e45f59624ff2/redisson/src/main/java/org/redisson/mapreduce/CoordinatorTask.java#L109
This logic should be reconsidered.

Default taskRetryInterval is 5 minutes, but for map-reduce it's just cancelled after timeout:
https://github.com/redisson/redisson/blob/ba31cf09018f05ffa54f017e1027e45f59624ff2/redisson/src/main/java/org/redisson/mapreduce/CoordinatorTask.java#L109
This logic should be reconsidered.

OK, thank you for your information. Is there any good practice to compensate MapReduce for timeout or abnormal node calculation?

Default taskRetryInterval is 5 minutes, but for map-reduce it's just cancelled after timeout:
https://github.com/redisson/redisson/blob/ba31cf09018f05ffa54f017e1027e45f59624ff2/redisson/src/main/java/org/redisson/mapreduce/CoordinatorTask.java#L109
This logic should be reconsidered.

OK, thank you for your information. Is there any good practice to compensate MapReduce for timeout or abnormal node calculation?

Fixed. Thanks for report!


Revert "use maven shade"

Reverts sharding-sphere/sharding-sphere#1605

Use maven shade cannot compile with IDEA, need to release shaded jar to maven centre repo.
So we decide if user find guava conflict, should shade by themselves.


fix: [Nested Tree View Styling] in [FluentDialog]

🐛 Bug Report
The root items of a TreeView do not align when nested within a FluentDialog
💻 Repro or Code Sample

<FluentDialog>
    <FluentTreeView>
        <FluentTreeItem Text="Root item 1">
            <FluentTreeItem Text="Flowers">
                <FluentTreeItem Disabled="true" Text="Daisy" />
                <FluentTreeItem Text="Sunflower" />
                <FluentTreeItem Text="Rose" />
            </FluentTreeItem>
            <FluentTreeItem Text="Nested item 2" />
            <FluentTreeItem Text="Nested item 3" />
        </FluentTreeItem>
        <FluentTreeItem Text="Root item 2" />
        <FluentTreeItem Text="Root item 3" />
        <FluentTreeItem Text="Root item 4">
            <FluentDivider></FluentDivider>
            <FluentTreeItem Text="Flowers">
                <FluentTreeItem Disabled="true" Text="Daisy" />
                <FluentTreeItem Text="Sunflower" />
                <FluentTreeItem Text="Rose" />
            </FluentTreeItem>
            <FluentTreeItem Text="Nested item 2" />
            <FluentTreeItem Text="Nested item 3" />
        </FluentTreeItem>
        <FluentTreeItem Text="Root item 5 - Leaf Erikson" />
    </FluentTreeView>
</FluentDialog>

🤔 Expected Behavior
Should look like the TreeView when not rendered within a FluentDialog

😯 Current Behavior
Align the items with no children with the items with children

💁 Possible Solution
Could manually update the styling
🔦 Context
Would like to embed a treeview within a dialog control.
🌍 Your Environment

OS & Device: [e.g. MacOS, iOS, Windows, Linux] on [iPhone 7, PC]
Browser [e.g. Microsoft Edge, Google Chrome, Apple Safari, Mozilla FireFox]
.NET and Fluent UI Blazor library Version [e.g. 8.0.3 and 4.7.1]

Thanks for the update! Hopefully can get resolved in the future :)

Closing this as it is not solvable from our side (browser issue).


fix(strategy): update swpr strategy subgraph endpoints [swapr]

Summary
Our Swapr strategy is outdated. This PR updates the subgraph endpoints to the correct ones.

@luzzif can approve this?

@adamazad Can more address to examples.json? few test cases are failing

Closing as stale, please reopen if needed


Deeply nested keys are not working

This library supports usage of keys with one level of nesting, like editor.title, or main.message, but once I try to go one more level deep, the library stops recognizing keys. Examples are dbe.title.open_database or dbe.title.open_folder.

Sorry, invalid issue, the cause was macro not picking up changes in localization files during incremental compilation


Keyerror: polygon

Hi,
Uniswap-python supports the Polygon version of Uniswap, right?
I get keyerror: polygon when I try this test example:
from uniswap import Uniswap
address = "0xxxx"
private_key ="xxxx"
version = 2
provider = "https://polygon-mainnet.g.alchemy.com/v2/Nxxxxxx"
uniswap = Uniswap(address=address, private_key=private_key, version=version, provider=provider)
eth = "0x0000000000000000000000000000000000000000"
bat = "0x0D8775F648430679A709E98d2b0Cb6250d2887EF"
dai = "0x6B175474E89094C44Da98b954EedeAC495271d0F"
uniswap.get_price_input(eth, dai, 1018)
uniswap.get_price_output(eth, dai, 1_000 * 1018)

Just in case
WETH contract @ polygon is 0x7ceB23fD6bC0adD59E62ac25578270cFf1b9f619
DAI contract @ poygon is 0x8f3Cf7ad23Cd3CaDbD9735AFf958023239c6A063
In addition, you should consider the opportunity to read the manual more carefully as qty parameter values are quite disputable in this snippet.

But I think I got those qty parameters from the manual. This could be something about the alchemy rpc. Testing... I will update if I find the cause.

I tested with the public polygon rpc instead (polygon-rpc.com) but I got the same error message:
File "/home/pi/pyuniswap", line 9, in
uniswap = Uniswap(address=address, private_key=private_key, version=version, provider=provider)
File "/home/pi/.local/lib/python3.9/site-packages/uniswap/uniswap.py", line 135, in init
router_contract_addr = _router_contract_addresses_v2[self.netname]
KeyError: 'polygon'

provider = "https://..."

polygon = Web3(Web3.HTTPProvider(provider , request_kwargs={"timeout": 60}))
polygon.middleware_onion.inject(geth_poa_middleware, layer=0)

uniswap = Uniswap(address=address, private_key=private_key, version=version, provider=polygon)

Tried adding the line you suggested which I recognize from Python web3 but I still get the same error:
"/home/pi/pyuniswap", line 18, in
uniswap = Uniswap(address=address, private_key=private_key, version=version, provider=provider)
File "/home/pi/.local/lib/python3.9/site-packages/uniswap/uniswap.py", line 135, in init
router_contract_addr = _router_contract_addresses_v2[self.netname]
KeyError: 'polygon'

This command:
unipy price WETH DAI
Gives a similar error:
keyerror:PROVIDER

Seems like it could be an issue with the Polygon rpc providers after all. I tested my alchemy one, didn't work, then I tested the public big one and it also didn't work. However, going back again I tested with infura this time and it worked. I will continue trying to get this to work with Polygon later.

@bitcoinmeetups Sorry, you can fix this by setting router_contract_addr, and maybe also factory_contract_addr (see https://uniswap-python.com/api.html#uniswap-class).
Is an official fork of Uniswap v2 even deployed to Polygon? Are we missing an entry here?
https://github.com/uniswap-python/uniswap-python/blob/2ee0e3b0710f0606834be5e88837eb7e901099c7/uniswap/constants.py#L31-L57


Not applying power of to parenthetical multiplied expressions

>260(e^0.25)
BinOp(
    Star,
    Constant(
        260.0
    ),
    Pow(
        Constant(
            2.718281828459045
        ),
        Constant(
            0.25
        )
    )
)
333.84660833881276
>260(e)^0.25
BinOp(
    Star,
    Constant(
        260.0
    ),
    Constant(
        2.718281828459045
    )
)
706.7532753993518

I believe this bug was fixed in a later commit than the 1.2.0 release I was using to find this.


Does this work with native Vulkan applications?

The README mentions:

Under normal conditions, this ReShade shader requires Lilium's DXVK fork, which is able to force games to use scRGB (RGBA16F) textures as internal render targets and output, this allows for 3 things:

As I understand it, does this mean this shader can work with native Vulkan applications? I'm interested in using it with Godot 4's Vulkan renderer, which internally renders in HDR (RGBA16F[^1]) but currently doesn't support HDR output.
[^1]: This is for the Forward+ rendering method. The Mobile rendering method uses RGB10A2 instead to minimize bandwidth usage, but most high-end desktop games are expected to use Forward+ (and you can force it with a CLI argument if a game defaults to Mobile).

Yes, if you force a native Vulkan app to output on a RGBA16F buffer and set linear sRGB as color space (scRGB), which AFAIK are supported on both Linux and Windows, you will get an HDR window capable of displaying colors above 80 nits (in other words, colors beyond 0-1).
Theoretically on Windows, a window and swapchain created by Vulkan wouldn't be to output an HDR image, though both the Nvidia and AMD drivers AFAIK create a DX12 swapchain on top of the Vulkan one to enable HDR compatibility both when in windowed or fullscreen mode.
So basically what you can do is keep the previous SDR rendering pipeline, render in sRGB with gamma applied (both on the 3D rendering and the UI layer), and then convert the image back to linear space just before copying it in the swapchain buffer.
If you apply the "AutoHDR" part of my shader, you will be able to extract some fake HDR brightnesses from it, based on user selected values.
As long as the intermediary buffers have a high bit depth, none of this shouldn't result in any additional banding.

Theoretically on Windows, a window and swapchain created by Vulkan wouldn't be to output an HDR image, though both the Nvidia and AMD drivers AFAIK create a DX12 swapchain on top of the Vulkan one to enable HDR compatibility both when in windowed or fullscreen mode.

NVIDIA defaults to DXGI indeed, though you can force Vulkan in the NVIDIA Control Panel. I don't remember hearing about AMD doing this though.

You might also want to check out this PR for Dolphin that added HDR support and did something similar to what I mentioned above:
https://github.com/dolphin-emu/dolphin/pull/11850


Event with a ZonedDateTime field that was stored in Axon 3.2 can't be read in 3.3

Using the AxonFramework with a jdbc event store in postgres:

Raise an event with a ZonedDateTime field in version 3.2.
Upgrade to 3.3 and reset a tracking processor that listens to that event.

The following exception will occur:
com.thoughtworks.xstream.converters.ConversionException: Invalid reference
---- Debugging information ----
message             : Invalid reference
reference           : ../../startTime
referenced-type     : java.time.ZonedDateTime
referenceable       : false
...
version             : 1.4.10
-------------------------------

In Axon 3.2 the date format is:
    <startTime resolves-to="java.time.Ser">
        <byte>6</byte>
        <int>2018</int>
        <byte>8</byte>
        <byte>3</byte>
        <byte>7</byte>
        <byte>-21</byte>
        <byte>0</byte>
        <byte>7</byte>
        <string>UTC</string>
    </startTime>

In Axon 3.3 it is:
    <startTime>2018-08-03T06:15:00Z[UTC]</startTime>

Hi @boldpawn,
this might have to do with java time support in XStream. Axon 3.2 had a dependency on 1.4.9, as of Axon 3.2.1, it is updated to 1.4.10.
According to their documentation, it should be backwards compatible (see http://x-stream.github.io/changes.html).
Perhaps explicitly including 1.4.9 resolves the problem. If so, I would file an issue with XStream that the backwardscompatibility isn't working as expected.

Hi Allard,
Using Axon 3.3.2 with the 1.4.9 version of xstream does indeed do the trick. We are now able to read events stored with Axon 3.2.
So it appears that xstream is not backwards compatible for ZonedDateTime
Marcel Schröer


Add support for aiobotocore

Closes #5
Added Async functions for begin subsegment and capture decorator.
Also fixed typo from my last PR
The yield from syntax is not valid syntax in py27 so was causing issues when I tried to add the async functions in the recorder module. Even tried some evil generator hacks but then that didn't work when py3 tried to use the functions.
Instead the recorder is subclassed to provide some py3 compatible async functions. When the SDK is imported in py3, the async compatible recorder class will be chosen.
The get_new_stubbed_recorder function was also updated to match the above logic. All the test cases were modified to explicitly set the context, as I was having issues where botocore was using an AsyncContext and then getting in a mess.
Updated the docs to mention if one is using aioboto3/aiobotocore that they should use the AsyncContext and provided a snippet.
Tox seems to be running ok on py27,34,35,36.

I also updated the code that loads in json to use pkg_resources so then the module can also be used in egg/zip form without being extracted.
Had to do a from __future__ import absolute_import in ext/boto_utils.py as importing the botocore.exceptions module was failing in py27 as botocore folder was in the same directory.

One last thing is that I like the new structure of ext/resources/botocore_para_whitelist.json. Could we rename it to aws_para_whitelist.json? It's more obvious as more and more files added under resources/ folder. Also botocore causes some confusion on if that json file is effective to boto3 or aiobotocore etc.

Sure, yeah exactly.

Should be good

Oh one thing I forgot is that feel free to add capture_async and aiohttp middleware support to the repo landing page. To be more specific the README.md Quick Start section.

Sure, I'll push some more changes tomorrow.

Added capture async example and a slimmed down example of adding middleware to aiohttp