Request of evaluation sample indices and mesh results

Thank you for your great paper!
As I delved into your research, I came across the section where you mentioned the selection of 50 shapes from ABC, with 20 from each category of ShapeNet for evaluation.
I am eager to learn more about the specific samples you have chosen for evaluation, and I was wondering if you could kindly provide me with the index numbers or any additional details regarding these selected shapes.
Furthermore, I am keen to gain a better understanding of your work and would greatly appreciate it if you could share the mesh qualitative results as presented in your paper.

Hi, Please check our latest version on CVPR website.
https://openaccess.thecvf.com/content/CVPR2022/html/Yu_CAPRI-Net_Learning_Compact_CAD_Shapes_With_Adaptive_Primitive_Assembly_CVPR_2022_paper.html
We used 1000 shapes from ABC and 1300 shapes from ShapeNet.
The processed data are given in the data link on the readme.

Here are three examples I just tried after running my codes:
https://drive.google.com/drive/folders/1SZZLqQ6maMLpDVxIzWtEUNad0SqWrW1V?usp=share_link
You can try my codes to get more results.

Thank you for your kind reply. Could you also provide the evaluation metrics (CD, ECD, NC, LFD) implementations?

You can find them on the BSP-Net evaluation section:
https://github.com/czq142857/BSP-NET-original


pmap causes segfault for large arrays

Description
The following pretty minimal example demonstrates a segfault that I am experiencing:
import jax
from jax import lax
from jax import numpy as np

from numpy import random

def test_foo():

    nt = 360
    ns = 80

    # with smaller tensor sizes all works
    # nt = 200

    Ms = np.array(random.randn(nt, nt, ns, ns))
    v2 = np.array(random.randn(nt, ns, ns))

    @jax.jit
    def einsum_for_single_t(Mst):
        return np.einsum(
            "ukl,umn,uln->km",
            Mst,
            Mst,
            v2,
        ).flatten()

    @jax.jit
    def compute_batch(t_range):
        def debug_print(t_range):
            print(f"compute_batch({t_range})")
        jax.debug.callback(debug_print, t_range)

        ret = lax.map(einsum_for_single_t, Ms[t_range])
        # ret = np.array([einsum_for_single_t(Mst) for Mst in Ms[t_range]])
        # ret = np.einsum(
        #     "tukl,tumn,uln->km",
        #     Ms,
        #     Ms,
        #     v2,
        # ).flatten()
        return ret

    print("now computing on", jax.devices())

    num_devices = len(jax.devices())
    t_range = np.array(np.array_split(np.array(range(2 * num_devices)), num_devices))
    print("t_range", t_range)

    map_result = lax.map(compute_batch, t_range)
    print(" map_result", map_result)
    pmap_result = jax.pmap(compute_batch, devices=jax.devices())(t_range) # SEGFAULT
    print("pmap_result", pmap_result)
    assert np.allclose(map_result, pmap_result)

if __name__ == "__main__":
    test_foo()

The program can be run with pytest or plain python.
Sample output when running on 2 V100 GPUs with trace:
now computing on [StreamExecutorGpuDevice(id=0, process_index=0, slice_index=0), StreamExecutorGpuDevice(id=1, process_index=0, slice_index=0)]
t_range [[0 1]
 [2 3]]
compute_batch([0 1])
compute_batch([2 3])
 map_result [[[ -292.6665    709.2823    784.7988  ...  -829.2205    289.9011
    -313.3165 ]
  [ 2115.9807   -243.4448  -2148.2627  ...  1514.6876   1626.8052
    1423.9783 ]]

 [[-1073.2985  -1070.3408   1277.1445  ...   230.26642   544.117
     424.26282]
  [ 3718.9797  -2192.378    -477.6187  ... -1207.0231  -1002.5527
     611.35144]]]
Fatal Python error: Segmentation fault

Current thread 0x000014f77f2f7740 (most recent call first):
  File "/.../lib/python3.8/site-packages/jax/_src/dispatch.py", line 1014 in backend_compile
  File "/.../lib/python3.8/site-packages/jax/_src/profiler.py", line 314 in wrapper
  File "/.../lib/python3.8/site-packages/jax/_src/dispatch.py", line 1079 in compile_or_get_cached
  File "/.../lib/python3.8/site-packages/jax/interpreters/pxla.py", line 1673 in from_hlo
  File "/.../lib/python3.8/site-packages/jax/interpreters/pxla.py", line 1512 in _compile_unloaded
  File "/.../lib/python3.8/site-packages/jax/interpreters/pxla.py", line 1535 in compile
  File "/.../lib/python3.8/site-packages/jax/_src/profiler.py", line 314 in wrapper
  File "/.../lib/python3.8/site-packages/jax/interpreters/pxla.py", line 1248 in parallel_callable
  File "/.../lib/python3.8/site-packages/jax/linear_util.py", line 303 in memoized_fun
  File "/.../lib/python3.8/site-packages/jax/interpreters/pxla.py", line 974 in xla_pmap_impl_lazy
  File "/.../lib/python3.8/site-packages/jax/_src/api.py", line 2253 in cache_miss
  File "/.../lib/python3.8/site-packages/jax/_src/traceback_util.py", line 162 in reraise_with_filtered_traceback
  File "/.../test_jax_segfault2.py", line 59 in test_foo
[...]
Segmentation fault      (core dumped)

As this shows the version with map works and then the pmap version segfaults. But I would like to use pmap to parallelize over multiple GPUs. For smaller array sizes this works and both GPUs are used. I am deliberately breaking up the einsum via lax.map(einsum_for_single_t, Ms[t_range]) to avoid that einsum intermediates no longer fit in memory. Using the commented alternative list comprehension leads to the same result.
The segfault happens also on a single GPU and the same happens on A100 GPU(s).
Commenting in the @jax.jits doesn't change things.
For smaller array sizes (comment in the nt = 200) everything works.
In CPU mode everything works.
What jax/jaxlib version are you using?
jax 0.4.1, jaxlib 0.4.1+cuda11.cudnn86
Which accelerator(s) are you using?
GPU
Additional system info
Python 3.8.12, Linux
NVIDIA GPU info
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 465.19.01    Driver Version: 465.19.01    CUDA Version: 11.3     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA Tesla V1...  On   | 00000000:02:00.0 Off |                    0 |
| N/A   33C    P0    25W / 250W |      0MiB / 32510MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  NVIDIA Tesla V1...  On   | 00000000:41:00.0 Off |                    0 |
| N/A   33C    P0    25W / 250W |      0MiB / 32510MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

I think #13923 is related.
Downgrading version might also fix your issue.
Also, your nvidia driver is technically too old for your jax installation (i believe).

Thanks! With v0.3.10 I get the same segfault.

did you fix this @cvjjm? I'm having the same issue

I tracked the crash down to an integer overflow in XLA. openxla/xla#6803 should fix.
However: you do not want to do what you are doing (capturing a large array lexically, here Ms). Large array constants are slow and lead to gigantic programs. You should pass large arrays as arguments instead. Make sense?
Hope that helps!


Write a C program to convert feet to centimeters

Description
Write a C program to convert feet to centimeters
1 foot = 30.48 centimeters
Input  : 5
Output : 152.4

How to contribute

Save the solution in program/ConvertFeetToCentimeters.c file
Add ConvertFeetToCentimeters.c file in convert-feet-to-centimeters folder

@harshraj8843 Could you assign me this issue?

!assign

Hey @william-monroy,
This issue has been assigned to you for more than 15 days.
We are unassigning you from this issue.
If you are still interested in contributing to this issue, please get auto-assigned on the issue by commenting !assign again.
Thanks for your contributions :)

!assign


fix: correct sample links in example admin application

Motivation
The current links configured for the sample interface within the admin application are incorrect and go to a 404. This PR corrects the casing in the URL for the docs folder.
Checklist

[ ] Code follows coding conventions held in this repo
[ ] Automated tests have been added
[ ] Tests are passing
[ ] Docs have been updated (if applicable)
[ ] Temporary settings (e.g. variables used during development and testing) have been reverted to defaults

How to test
If manual testing is required, what are the steps?

This was noticed in the KBank demo site, so ideally KBank would be updated with this fix too.


Use keyboard styling for hotkeys in tooltips

We have nice key styling in the shortcuts screen:

But not for tooltips:

Apparently these used to have key styling? See #1495. In any case, we should make these consistent.

There, this is better.


[Android] All changes for next release

Updated Android library.

current build tools
minSdk 23
targetSdk 33
added support for new Permissions in manifest
migrated to AndroidX
migrated to last version of rxandroidble2 library
migrated to RxJava2
usage of new Java language features
small refactor
created small SampleApp module (as a part of refactor process)

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Fix filestore and a bug in recursive stage/unstage

I took the ad-hdf5 branch (#208) for more exercise at XPD tonight. I discovered some small but important bugs -- see commits for details.
Automated tests are coming tomorrow.

I should note (and celebrate!) that we were able to retrieve an image array through filestore at XPD. We had no problems with IOC instability while using TIFF. (The HDF5 plugin is another story -- on AD 2.2 at CSX it was robust, but at XPD it crashed the IOC yet again.) But the TIFF plugin integration seems to be ready for prime time.


Change XAPI database

There is a setting to update the XAPI database however, this setting is not properly used across the whole project.
We need to refactor so, changing the database schema will not break it all.

We should at least confirm we can change all of the databases and things still work: ASPECTS_XAPI_DATABASE, ASPECTS_EVENT_SINK_DATABASE, ASPECTS_VECTOR_DATABASE. What do you think about making the table names and mv names hard coded? I'm having trouble seeing a use case for changing them and it's a lot of configuration to carry around. What do you think @SoryRawyer ?

I see just one use case to override the database name, and it's just for developing purposes. Not sure what a table name override can offer that cannot be accomplished with other extensions

(and I definitely use the database changing behavior in dev and testing a fair amount, I think it's very valuable to support)

As per my testing, this is already working!


How to use Camera to detect obstacle in Apollo5.0

System information

OS Platform and Distribution: Linux Ubuntu 16.04
Apollo version (3.5, 5.0, 5.5, 6.0): 5.0

Steps to reproduce the issue:
In Dreamview, the default setting for the Perception module is to only use Lidar. How to only use Camera for obstacle detection. I have tried to use 'mainboard -d /apollo/modules/perception/production/dag/dag_streaming_perception_camera.dag' to use the Camera. However, the PerceptionObstacle module delay in Dreamview keeps red, and the car cannot detect the obstacle. Also, I tried to create a new '.dag' file that contains the contents of both 'dag_streaming_perception.dag' and 'dag_streaming_perception_camera.dag'. After launching this file, the module delay becomes green, and the car can detect the obstacle. However, I'm not sure if the Camera works this time.
Therefore, if there is a way only to use Camera for obstacle detection?

It is recommended to use the latest version for testing. the ref doc

@YCC-324 Hi,
I looked at the code of apollo 5.0 and roughly located where the problem occurred. Start perception.dag separately and the final fusion result will be published to the /apollo/perception/obstacles channel.https://github.com/ApolloAuto/apollo/blob/r5.0.0/modules/perception/production/conf/perception/fusion/fusion_component_conf.pb.txt#L5.
use 'mainboard -d /apollo/modules/perception/production/dag/dag_streaming_perception_camera.dag' to use the Camera,the result will be published to the /perception/obstacles channel.
You should modify the channel in the above configuration file to /apollo/perception/obstacles.

Thank you. I think this is one of the reasons. But after changing it, the obstacle is still not detected. Did you succeed?

It is recommended to use the latest version for testing. the ref doc

Thanks, the Camera can truly work in the newest version. But recently, we have to perform an experiment on Apollo 5.0. So, I wonder if the Camera can work well in this version too.

It is recommended to use the latest version for testing. the ref doc

Hello @daohu527,  could you or Apollo team please fix the link to images in document ?
The link is broken then images can not be shown, for example :


[BUG] - Image Rotates after clicking image via camera on lower Android versions

Lib Version [4.3.1]

Describe the bug
The image taken via camera rotates (-90 degrees) after being captured. I've tried this with and without intent chooser. I've tried in on an emulator because that's where we do tests.
Happens on API level 26. Android 8 (Oreo). Not happening on Android 11. Might be happening on other API levels too. I haven't tested
To Reproduce
Steps to reproduce the behavior:

Go to main screen of sample app.
Click on the 3rd button "Sample calling cropimage"
Now in the next screen click on the 2nd button "Calling library directly with URI".
Choose camera.
Click an image
Notice after capturing the image, it is rotated by -90 degrees.

Expected behavior
On clicking an image via camera the image shouldn't rotate.
Media

Smartphone (please complete the following information):

Device: Android Studio Emulator with Pixel 2 configuration
OS: Android 8 (Oreo) Api Level 26

Additional context
I might be a little busy but if I get time I will try to look into it and open a PR. I think this is what @Jeyaseelan2528 meant in issue #292

Hey @Devenom1 thanks for the detailed PR, helps a lot.
I try to reproduce, but could not, so maybe is a specific case of brand/os. Will try with the simulator you use later.
but from your video and steps

Notice after capturing the image, it is rotated by -90 degrees.

This may not be the library, but the camera itself. Could you try to open the camera app and take a picture so we check if will rotate too? Because taking the picture before we accept is the device camera working, not the lib 🤔

Hey @Canato thanks for the quick response as always.
Here is a video from the default camera app on the android studio emulator. It doesn't rotate with the default camera app.

I remember facing this issue before when I made a local copy of the Arthur Hub Library. I had fixed this issue on that but now I've deleted that and I'm too lazy to check my changes on github. It was a mess. I switched to CanHub library completely.

Could you please try the latest release, 4.4.0 and reopen this issue if it still persists?


Dataverse collection-level metrics

Overview of the Feature Request
Have a similar button and page for metrics in dataverse collection pages other than the installation root with the metrics for the collection and its subcolletions.
This could be split in two steps :

[ ]  Have Dataverse metrics support target collections
[ ] Add the metrics button in every dataverse collection page

What inspired the request?
Dataverse collection administrator need for visibility and reporting on their collection
Any open or closed issues related to this feature request?
None known, but will be related to the addition of Dataverse metrics in the SPA
Screenshots

The URL of the button should be modified to specify the collection

And the button should appear in subcollection with the appropriate link
Community involvement
We will probably contribute the first part for handling the collection aliases but are waiting on progress of the SPA to determine when we will do the UI implementation

I missed the dvSelect parameter :zipper_mouth_face: , so I'm closing this issue and UI part will be directly in the appropriate repo.


disable device modes

Hello,
is there a built in solution to disable some device modes? In my case I want to disable the "Large Monitor" and "Widescreen Monitor" sizes completly, so that in all cases "computer" is the widest container/grid etc. size.
Couldn't find anything about it in the docu.
Thanks for your help... :)

None that I know of. You'll have to code it yourself.
Also, in the future, consider posting usage questions on ProjectTalk so we can keep github only for bug reports / proposals.


Doc site

In depth docs in docs/ should be hosted.

[ ] ~Text attributes~
[ ] ~Line modifiers~
[ ] ~Embeds~
[ ] ~boustro_starter components~
[ ] ~Theming boustro~
[ ] ~Theming custom components~
[ ] ~Serialization~

Docs are hosted at jjagg.dev/boustro and there's stubs for articles still to write.


waifu-diffusion with AMD

Is it possible to change the model to waifu-diffusion? I did not find guides to work with amd cards.

I was able to use the waifu-diffusion model with the following code.
pipe = StableDiffusionPipeline.from_pretrained("hakurei/waifu-diffusion", torch_dtype=torch.float16)

pipe.safety_checker = lambda images, **kwargs: (images, False)

prompt = "1girl, \
blue eyes, \
cat ears, \
full body, \
looking at viewer, \
school uniform, \
simple background, \
silver long hair, \
under rim eyewear, \
white background"

image = pipe(prompt, height=512, width=512, num_inference_steps=50, guidance_scale=7.5, eta=0.0, execution_provider="DmlExecutionProvider")["sample"][0]
image.save("Dml_waifu1.png")

Generate image. (The image is corrupted because I did not adjust the prompts, etc.

Thanks for the answer.
Just recently I found a build that launches webui (allows you to use a negative prompt) but without using the GPU. It's a pity that the CPU is slow in this process. Now I'm looking for the possibility to launch webui using rx5700 (it is somehow used in dml_onnx).
Maybe someone already has similar developments?


Polymer components installed through bower

I am using polymer components in angular2.
when i install any component say paper-button with bower in command line,the version of component installed with bower is different from the one which is used in polymer demo.
The problem here is that the versions of components(say paper-button and paper-spinner) installed from bower needs to have different versions of polymer.
Please update the bower so we can easily install all components from bower that are up to date and needs the same version of polymer.

Bower updates automatically from tags, so as soon as we tag an element on github, it is available in bower.
I suspect your issue is that you are mixing elements from the Polymer org and the PolymerElements org.
If you are using Polymer 1, use repos from PolymerElements org. If you are using Polymer 0.5, use repos from Polymer org.
Sorry that this is confusing, but we are trying to still support users of Polymer 0.5, and if we removed the older repos it would break those uses.

ok thanks :+1:


add empty aws-roles empty list

add empty aws-roles empty list same style as products empty list

Now the Public Cloud Users Tab has a message if no users are added to the User Group.

The button allows to add a new User to the User Group.


[uk] Linear algebra

Hi @shervinea , this is finally done ;)

@grendach - care to review this one?

Hi @ror6ax, whenever you get the time, please feel free to incorporate @grendach's suggestions as you see fit. That way, we'll be able to move forward with the merge of all your work to the main repo.


How remove the unused code when detected automatically ?

I am not a ruby guy but can you suggest any changes that could help to remove the non used code automatically when read ?

There are too many false positives :) I suggest not doing this automatically


What is the intent of the "status" device? (question more than an issue)

G'day,
I am curious as to what the intent of the status device is?
Do you have an example?
Regards,
Christopher

Just a simple way to be able to share information from a device

Just a simple way to be able to share information from a device


KAFKA-7109: Close cached fetch sessions in the broker on consumer close

Previously, the consumer's incremental fetch sessions would time out once the consumer was gone.

@cmccabe please review. No easy way to test this as far as I'm aware, integration tests should at least verify it doesn't break anything.

cc @cmccabe

Thanks for this, @stanislavkozlovski.  Looks good.
We need a test, though.  Take a look at clients/src/test/java/org/apache/kafka/clients/consumer/internals/FetcherTest.java  You should be able to write something that will verify that closing the Fetcher does the expected thing.

LGTM.  Thanks, @stanislavkozlovski.

Retest this please

Retest this please

Retest this please

@cmccabe I've addressed the latest changes and rebased to trunk

Retest this please

It may be worth reviving this, it requires a rebase.

@stanislavkozlovski , encountered similar errors and I agree we should close the incremental fetch sessions when consumer closed. Are you able to complete this PR? If not, I can co-author with you to help complete this PR to merge this good improvement. Thank you.

@stanislavkozlovski , I'm going to take over this ticket tomorrow if you don't mind. Thank you.

@showuon yes, please feel free. apologies for being slow to respond and missing the previous messages about reviving this. I will go ahead and close this PR now

Thank you!


Install Error on MacOS

Hello There,
im having an error installing System Verilog on my mac.
When running the command:
brew install yosys graphviz icarus-verilog gnu-sed dos2unix
the following error appears:
Warning: No available formula with the name "yosys".
==> Searching for similarly named formulae and casks...
Error: No formulae or casks found for yosys.
What do I do in this case?
Thanks!

The formula yosys is still available on Homebrew: https://formulae.brew.sh/formula/yosys.
I'm also able to install it locally without any errors.
Which version of macOS are you using?

I think there's something wrong with your brew installation.
You can run brew doctor to diagnose possible issues.
You can also check if all default taps are enabled using brew taps:
homebrew/cask
homebrew/core
homebrew/services

If nothing works, you could try to reinstall Homebrew.

That's definitely what you should do, yeah


splits test_tournament.py into separate files for each class under test

...and tournament files

These greatly improve the organisation of everything. Thanks!


Hashtable not adding all elements to the table

Describe the Issue
Hashtable may not be resizing correctly which may be causing the table to not add each element after a certain size. (Issue may be due to memory constraints.)
Library Name and Version:
Name: Hashtable
Version: 1.0.8
Environment:

Platform/OS: [e.g., Windows, macOS, Linux] Windows
Arduino IDE version (if applicable): NA
PlatformIO version (if applicable): NA
Development board (if applicable): Simulated Arduino Uno

Library Source:

[x] GitHub
[x] PlatformIO Registry

Expected Behavior
List should resize properly and add each element into the table properly
Actual Behavior
Table gets to about 26 elements before throwing an error. (Using a forloop)
To Reproduce
Steps to reproduce the behavior:

Include library '...'
Initialize the library '...'
See error

Error Message & Code Snippet
If applicable, add the error message and the piece of code that causes the issue.
// Your code that triggers the error
Screenshots
If applicable, add screenshots to help explain your problem.

Additional Context
Add any other context about the problem here.

<!-- End of the issue report template. -->

Issue seems to be a memory constraint for the Uno. When using a Simulated Mega, the table resizes perfectly and has no problems. All elements get added with no problems. Cap on uno seems to be about 86 elements.


Create a histogram pane

Is your feature request related to a problem? Please describe.
No.
Describe the solution you'd like
A pane should be available which allows for viewing an audio histogram with draggable boxes representing captions.

[ ] A scrubbing timeline should be used to scrub through the video

Describe alternatives you've considered
None.
Additional context
Example from the Youtube subtitles UI:

Errors with updated Readme. When running Setup.bat, incompatibility error pops up.
Pop up window:

Terminal output:

Traceback (most recent call last):   File "Setup.py", line 15, in <module>     VulkanRequirements.Validate()   File "E:\Github\Repos\subtend\scripts\SetupVulkan.py", line 18, in Validate     if (not cls.CheckVulkanSDK()):   File "E:\Github\Repos\subtend\scripts\SetupVulkan.py", line 30, in CheckVulkanSDK     cls.__InstallVulkanSDK()   File "E:\Github\Repos\subtend\scripts\SetupVulkan.py", line 57, in __InstallVulkanSDK     os.startfile(os.path.abspath(vulkanPath)) OSError: [WinError 216] This version of %1 is not compatible with the version of Windows you're running. Check your computer's system information and then contact the software publisher: 'E:\\Github\\Repos\\subtend\\Subtend\\vendor\\VulkanSDK\\VulkanSDK-<IP_ADDRESS>-Installer.exe'


Implement a more robust search algorithm

Search UX is currently a little rough. I have observed users in production struggling with it. It seems that they have a hard time finding their bank, and often have trouble even knowing how their bank is spelled. It would be great to implement a full text index and allow fuzzy matching to help in this case.

Hi @ianpatton
sorry I heard about this but was looking for the issue in the unified-search repo.
in case you haven't noticed, the search behavior is implemented here https://github.com/Universal-Connect-Project/unified-search
about the issue itself. could you clarify if this is just generally speaking or did you have a specific example that I can test and find some unexpected behavior ( so that it's a bug instead of enhancement)
about fuzzy matching, when exploring the data and designing the indexing approaches. I found there are actually a lot of banks with similar names, I tried to balance between flexibility and accuracy, so the current algorithm is allowing matching with partial words, middle words but it doesn't allow matching with mis-spelled words, not too fuzzy
if you have any examples that we can test against, it'll help us finding out what to improve
one example I can think of is to add "short form" matching , e.g.: matching "Federal credit union" by searching "fcu" , this doesn't existing in the current implementation

Here's a quick example. The user types out quite a lot before they select the bank

This one is probably a better example, showing misspelling and not knowing if they should use short or long form

the first example looks "normal" to me,  by typing this much, it matches what user meant.

by typing one letter less, it displays something else but still matched the term, just need to keep typing to narrow it down.
it doesn't mean it's so good, we do have a "issue" that the institution list needs to be cleaned up, so far there are multiple entries that look the same. this is not an easy task, to filter out and leave the best, MX is having a team for this.
the second example does explain something, my understanding is that the user only remembered the short form of the bank, he it wasn't able to find with short form. then he started to struggle to find the correct spell. (I'd try to google it if I were.. )
I think fuzzy match is still a topic that needs careful discussion. though shortform matching sounds like a worthy idea to add, and it's not hard to add, just when splitting the search terms, grab the first letters and make up a word as one more term to build the index,
cc @tphalp


How to use it with non square resolutions?

i wanted to try to run the model with non-square resolution but it always ends up throwing certain tensor error "The size of tensor a (240) must match the size of tensor b (134) at non-singleton dimension 4".
I was trying to run it for 19801080 image, I tried adjust resolution to make it 19801072 to make it divisible by 16 but it still didn't work.
how can I resolve it???

You could try to use the latest released demo.ipynb to adapt the size of reference image


Consider completely serializing access to SQLite

Creating a transaction should acquire a single-token semaphore. It seems like SQLite is really just too unreliable to call from multiple threads, even with retries and other mitigations.

https://github.com/io7m-com/darco/issues/33

Done.


null blocks

Sometimes there are null values in the block lookups
Eg. https://api.dcrdata.org/block/range/146880/146882
and https://api.dcrdata.org/block/146881
It's odd because in my local dev environment I'm getting a return value for block 146881
{
  "height": 146881,
  "size": 74128,
  "hash": "000000000000005c4ea75265ed0d5e45f56cc9b10161322851ca9593a48a4691",
  "diff": 11034562.58528116,
  "sdiff": 58.00850994,
  "time":<PHONE_NUMBER>,
  "ticket_pool": {
    "size": 44155,
    "value": 1998415.10031227,
    "valavg": 45.25908957790216
  }
}

But I have other blocks that come back as null, eg http://localhost:7777/api/block/146443
but for that same block, the production server has values https://api.dcrdata.org/block/146443
So I'm thinking there may be an issue with indexing?

There are a couple of things that are likely to be causing this, and most of them need to be fixed to be sure.
First, I have several processes running on the public web server, including two versions of dcrdata that fight for resources, so it seems to be common when mempool is going crazy with ticket purchases to have this sort of thing happen (same block you pointed to):
2017-07-01 04:37:14.311 [DBG] MEMP: Vote in new block triggering mempool data collection
2017-07-01 04:37:15.693 [INF] DCRD: Block height 146881 connected
2017-07-01 04:37:33.936 [DBG] MEMP: mempoolDataCollector.Collect() completed in 19.591016827s
2017-07-01 04:37:35.698 [ERR] DCRD: Block data collection TIMEOUT after 20 seconds.
2017-07-01 04:37:36.120 [DBG] DCRD: blockDataCollector.Collect() completed in 20.42313802s

The blocks before and after that one did not take nearly that long, so it looks like having the mempool data collection even be able to run first is a bad idea since it can be an order of magnitude slower with several hundred tickets in mempool, and having two dcrdata instances hammering the same dcrd is worse.
Second, and related to the first actually, is if blocks roll in quickly, requests pile up and because of some limitations in the dcrd RPCs, it was a bit hard to collect data for a specific block rather than the latest block.  So it's easy to completely miss a block (and overload dcrd).  The latest PR, merged June 30 was intended to address this to some extent.
These probably don't explain why your block 146443 is null though.
Third, reorganizations are only partly handled.  :cry:  I've worked a lot on getting it handled enough not to crash dcrdata, but I still need to it to fix up new blocks added when switching a side chain to become the new main chain.  It's handled in the internal stake database (the ffldb_stake folder) but the sqlite db needs to be fixed up still.  This isn't terribly hard, but I've prioritized other stuff.

Just submitted PR https://github.com/dcrdata/dcrdata/pull/55 that should address "third" in my post above.

I also snuck in commit https://github.com/dcrdata/dcrdata/commit/0cc7437aac99d908ae4c16606292dfefb1b6645d to address "first" in the above post.  It's a band-aid, but it should help for now.

Merged https://github.com/dcrdata/dcrdata/pull/55, so I think this can be closed.
After syncing master, be sure to glide install again before build, then rm -r ffldb_stake dcrdata.sqlt.db to start fresh.


Feature/better hatch build system

There's a little bit of todos for future work, but this seems to cover all the cases so far and works well enough.
Right now it just does auto-detection, but want to allow people to explicitly list the contexts/subkernels in the pyproject instead, but figure that can be a future update.

This looks good to me just looking through the code. What's the best way to test it outside this repo for a new project/context?

Note:
Between the question above and now, I fixed a couple bugs that was preventing the package to build from github and Docker, and I went ahead and did some refactoring in beaker_kernel to make class names and using the "base" clasess easier and more intuitive.
To build a new context, you would first start from a new/existing python project that uses the hatch builder.
In the source files, you will want to define at least a new context class based off of (the newly renamed) BeakerContext class. You will probably want to define an agent as well, but it's possible to use the default (BeakerAgent) for basic interactions.
The new context class MUST have a slug defined as a class attribute.
It will probably have init and setup methods defines as well, along with any tools, etc.
A basic context file would look something like this:
my_project/src/my_project/context.py
from beaker_kernel import BeakerContext
from .agent import MyAgent

class MyContext(BeakerContext):

    enabled_subkernels = ["python3"]

    SLUG = "my_context"

    def __init__(self, beaker_kernel: "BeakerKernel", config: Dict[str, Any]):
        super().__init__(beaker_kernel, MyAgent, config)

    async def setup(self, context_info=None, parent_header=None):
        super().setup(context_info, parent_header)

With the context existing in a .py file in the src tree for the package, it will be discovered by the auto-discovery in the builder.
To build the package so that the context is properly registered, you can updated your pyproject.toml as follows:
dependencies = [
    "beaker_kernel@git+https://github.com/jataware/beaker-kernel.git@feature/better-hatch-build-system#egg=beaker_kernel",
]

[tool.hatch.build]
require-runtime-dependencies = true

[tool.hatch.build.hooks.beaker]

The require-runtime-dependencies is required because it ensures that beaker_kernel is installed during build time so we can check if the classes in the src tree are subclasses of the BeakerContext class defined therein.
I would love to make that be inherent for the plugin, but so far haven't figured out how to do so as yet.
Right now, there are no options for tool.hatch.build.hooks.beaker, but there will probably be some in the future as described above.
Let me know if you have any questions.
@brandomr

I'm getting this error:
.931   × Preparing editable metadata (pyproject.toml) did not run successfully.
1.931   │ exit code: 1
1.931   ╰─> [21 lines of output]
1.931       Traceback (most recent call last):
1.931         File "/usr/local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 167, in prepare_metadata_for_build_editable
1.931           hook = backend.prepare_metadata_for_build_editable
1.931       AttributeError: module 'hatchling.build' has no attribute 'prepare_metadata_for_build_editable'
1.931
1.931       During handling of the above exception, another exception occurred:
1.931
1.931       Traceback (most recent call last):
1.931         File "/usr/local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
1.931           main()
1.931         File "/usr/local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
1.931           json_out['return_val'] = hook(**hook_input['kwargs'])
1.931         File "/usr/local/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 176, in prepare_metadata_for_build_editable
1.931           whl_basename = build_hook(metadata_directory, config_settings)
1.931         File "/tmp/pip-build-env-6xjxursf/overlay/lib/python3.10/site-packages/hatchling/build.py", line 83, in build_editable
1.931           return os.path.basename(next(builder.build(directory=wheel_directory, versions=['editable'])))
1.931         File "/tmp/pip-build-env-6xjxursf/overlay/lib/python3.10/site-packages/hatchling/builders/plugin/interface.py", line 116, in build
1.931           configured_build_hooks = self.get_build_hooks(directory)
1.931         File "/tmp/pip-build-env-6xjxursf/overlay/lib/python3.10/site-packages/hatchling/builders/plugin/interface.py", line 386, in get_build_hooks
1.931           raise UnknownPluginError(message)
1.931       hatchling.plugin.exceptions.UnknownPluginError: Unknown build hook: beaker
1.931       [end of output]

Could you share a full pyproject.toml that worked for you? I've got:
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "test"
dynamic = ["version"]
description = ''
requires-python = ">=3.10"
license = "MIT"
keywords = []
authors = [
  { name = "Brandon Rose", email =<EMAIL_ADDRESS>},
]

[tool.hatch.version]
path = "src/test/__about__.py"

dependencies = [
    "beaker_kernel@git+https://github.com/jataware/beaker-kernel.git@feature/better-hatch-build-system#egg=beaker_kernel",
]

[tool.hatch.metadata]
allow-direct-references = true

[tool.hatch.build]
require-runtime-dependencies = true

[tool.hatch.build.hooks.beaker]

Ok, so I created a new project and basically copied your pyproject there exactly and the package built just fine out of the box:
matt@mavis:~/development/beaker/beaker-buildtest$ hatch build
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── sdist ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
dist/beaker_buildtest-0.0.1.tar.gz
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── wheel ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Found the following contexts:
  'my_context': TestContext in package beaker_buildtest.context

dist/beaker_buildtest-0.0.1-py3-none-any.whl
matt@mavis:~/development/beaker/beaker-buildtest$ unzip -l dist/beaker_buildtest-0.0.1-py3-none-any.whl
Archive:  dist/beaker_buildtest-0.0.1-py3-none-any.whl
  Length      Date    Time    Name
---------  ---------- -----   ----
      129  2020-02-02 00:00   beaker_buildtest/__about__.py
      107  2020-02-02 00:00   beaker_buildtest/__init__.py
      136  2020-02-02 00:00   beaker_buildtest/agent.py
      467  2020-02-02 00:00   beaker_buildtest/context.py
       98  2020-02-02 00:00   beaker_buildtest-0.0.1.data/data/share/beaker/contexts/my_context.json
     1585  2020-02-02 00:00   beaker_buildtest-0.0.1.dist-info/METADATA
       87  2020-02-02 00:00   beaker_buildtest-0.0.1.dist-info/WHEEL
     1099  2020-02-02 00:00   beaker_buildtest-0.0.1.dist-info/licenses/LICENSE.txt
      801  2020-02-02 00:00   beaker_buildtest-0.0.1.dist-info/RECORD
---------                     -------
     4509                     9 files

Full set of files here:
beaker-example.tar.gz
What I think is happening is that I think that your hatch/pip is installing a cached version of beaker_kernel instead of fetching from git.
Let's try clearing your caches and such and see if that fixes it with your existing setup:
pip cache purge
hatch env prune
Then try building again normally.

Getting back to this since I'm trying to create a new context with it. I tried to run your example context in Docker and get a circular import error:
jupyter-1  | │ /jupyter/src/beaker_buildtest/context.py:2 in <module>                       │
jupyter-1  | │                                                                              │
jupyter-1  | │    1 from typing import Dict, Any                                            │
jupyter-1  | │ ❱  2 from beaker_kernel import BeakerContext, BeakerKernel                   │
jupyter-1  | │    3                                                                         │
jupyter-1  | │    4 from .agent import TestAgent                                            │
jupyter-1  | │    5                                                                         │
jupyter-1  | │                                                                              │
jupyter-1  | │ ╭────── locals ──────╮                                                       │
jupyter-1  | │ │  Any = typing.Any  │                                                       │
jupyter-1  | │ │ Dict = typing.Dict │                                                       │
jupyter-1  | │ ╰────────────────────╯                                                       │
jupyter-1  | ╰──────────────────────────────────────────────────────────────────────────────╯
jupyter-1  | ImportError: cannot import name 'BeakerContext' from partially initialized
jupyter-1  | module 'beaker_kernel' (most likely due to a circular import)
jupyter-1  | (/usr/local/lib/python3.10/site-packages/beaker_kernel/__init__.py)
jupyter-1 exited with code 1

any ideas?


[UVG] Some minor fixes, add translation strings

Required

[ ] The pull request title clearly contains the name of the sheet I am editing.
[ ] The pull request title clearly states the type of change I am submitting (New Sheet/New Feature/Bugfix/etc.).
[ ] The pull request makes changes to files in only one sub-folder.
[ ] The pull request does not contain changes to any json files in the translations folder (translation.json is permitted)

Changes / Description
Fixed broken buttons, some CSS changes and add translations strings

Character Sheet Info Roll20 Internal Use only.


docs(tracer): new ignore_endpoint feature

Issue #, if available: #910
Description of changes:

Documents new ignore_endpoint feature in Tracer.
from aws_lambda_powertools import Tracer

tracer = Tracer()
# ignore all calls to `ec2.amazon.com`
tracer.ignore_endpoint(hostname="ec2.amazon.com")
# ignore calls to `*.sensitive.com/password` and  `*.sensitive.com/credit-card`
tracer.ignore_endpoint(hostname="*.sensitive.com", urls=["/password", "/credit-card"])

def ec2_api_calls():
    return "suppress_api_responses"

@tracer.capture_lambda_handler
def handler(event, context):
    for x in long_list:
        ec2_api_calls()

Checklist

[x] Meet tenets criteria
[ ] Update tests
[x] Update docs
[x] PR title follows conventional commit semantics

Breaking change checklist

RFC issue #:

[ ] Migration process documented
[ ] Implement warnings (if it can live side by side)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

View rendered docs/core/tracer.md

Codecov Report

Merging #931 (9ae6ed2) into develop (be15e3c) will increase coverage by 0.00%.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop     #931   +/-   ##
========================================
  Coverage    99.96%   99.96%
========================================
  Files          118      118
  Lines         5281     5291   +10
  Branches       606      608    +2
========================================
+ Hits          5279     5289   +10
  Partials         2        2

Impacted Files
Coverage Δ

aws_lambda_powertools/logging/logger.py
100.00% <0.00%> (ø)

aws_lambda_powertools/logging/formatter.py
100.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update be15e3c...9ae6ed2. Read the comment docs.


Split loggers and log providers/observers (ILogger has too many responsibilities)

I think it is problematic that the ILogger interface is used for creating log entries and for writing them to some output. For that reason, I suggest that providers (e.g. Console, EventLog, ...) should not inherit ILogger. Instead, we should move towards an Observable-like system.
I will try my best to explain the benefits and possible future improvements, so please take a seat for the long story! :)
This is the current dependency structure:

LoggerFactory creates and manages Logger instances - one per categoryName. They are responsible for creating log entries.
LoggerFactory has one ILoggerProvider per provider
Every Logger instance stores a list of ILogger instances - one per provider. They are responsible for writing log entries to outputs.
Whenever a new provider is added, LoggerFactory adds the ILogger instance from the provider to every Logger-instance.
Each Logger communicates directly with each provider-ILogger.

I think that this solution has a few disadvantages:

Providers are forced to have ILogger instances per logger category. Requiring different Logger instances is a concept of log entry creation and is useful there. Why should there be many instances of e.g. an EventLog-provider?
This also makes it harder for providers to batch log entries. If you want to send e.g. 50 log entries at once to some output system, you would either have to batch 50 per category or you would have to merge them somehow first.
the direct relationship between Logger and the provider-ILoggers makes it harder to apply global patterns like filtering because you always have to broadcast everything to every Logger in the LoggerFactory.
Providers are forced to implement the same contract. If we would e.g. have global filtering, IsEnabled would only be interesting for the creation-side. Providers would not need the same method because they would not even receive the entry.
Mixing the concepts makes this really hard to understand. Just look at the mix of ILogger and Logger here.

In my opinion, the logging system should work more like the observable pattern. ILogger instances should be responsible for creating log entries and they should send them to some central coordinator. This coordinator knows about subscribed providers and forwards the entries to each of them.
what I would like to see
Ideally, I would completely split log creation and forwarding. This would begin with how providers are set up. I think it is weird, that you can add providers to a LoggerFactory because the factory is used in many locations. Why should you be able to do something like this:
public SomeClass {
  public SomeClass(ILoggerFactory loggerFactory) {
    loggerFactory.AddConsole();
  }
}

I think it should only be possible to call CreateLogger here. I think this would come closest to what the name "factory" implies.
Ideally, providers would be observers - e.g. ILogObserver. There would be a corresponding LogObservable which acts as the described central coordinator. Only this class would need the AddObserver (aka Subscribe, aka AddProvider) method. So when you would create the application, you would configure LogObservable instead of LoggerFactory. This would look like this:
var logObservable = new LogObservable()
  .AddEventLog()
  .AddConsole();

var loggerFactory = new LoggerFactory(logObservable);
// both will be added to DI as singleton

// There would be no AddProvider here
var logger = loggerFactory.CreateLogger<Program>();

Ideally, the LogObservable and LogObserver would actually implement IObservable and IObserver to allow us to use Rx components. However I'm not sure if this would play well with other required methods like BeginScope.
baby steps
Since this would be a quite big change, I'm proposing a compromise and a first step with this pull request:

the meaning of ILoggerProvider will be changed (instead of renamig it to LogObserver). This interface will represent everything that is required for providers to write entries to some output. It will no longer have CreateLogger, instead it will have methods like Log and BeginScope. This will be a breaking change, however it only affects people who built their own providers.
Providers will no longer have to inherit ILogger because everything they need is on ILoggerProvider.
Providers will get the categoryName as part of their methods. This means, there may only be one instance of every provider. If the provider needs to have different instances, it can still do so based on this parameter.
The LoggerFactory will still be responsible for adding providers to make sure there is no breaking change for client applications.
The LoggerFactory and Logger instances will no longer maintain a direct relationship with providers. Instead, they will just forward everything to the new LogObservable class. This results in very easy to understand code in Logger and LoggerFactory
The new LogObservable contains most of the code from Logger and LoggerFactory. It allows us to have one central location that talks to the providers.

This code contains a POC of how this would work. Please note the following:

the existing providers still contain *LoggerProvider and *Logger classes. The *Logger class would not be necessary but I kept them to keep changes to a minimum.
the providers still contain code for filtering. This could be moved to the LogObservable class.
some unit tests fail - I just made sure everything compiles for now.

so don't merge it yet :-) :-)
I think that this solution would also make global filtering #398 easier. I think it would remove the need to create a separate LoggerFactory. I will try to come up with a good solution.
I'm looking forward to your feedback!

Hi @cwe1ss, I'm your friendly neighborhood .NET Foundation Pull Request Bot (You can call me DNFBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
The agreement was validated by .NET Foundation and real humans are currently evaluating your PR.
TTYL, DNFBOT;

I'm like 97% sure we won't take an architectural change this big, this late in the game. This not only affects us but the serilog and nlog implementations (which are already filterable and fully featured). At this point, it just seems like a non-starter IMO.

Not sure if it's too late. It's a fact that rc2 will have breaking changes.

thanks for taking the time to read through this, David! I'm sorry for taking up so much of your time!
I understand that. Maybe this PR is just helpful to raise awareness for how this library is scoped and what it can/can't/shouldn't do. I wouldn't be offended if this PR doesn't go my way.
Most of my initial post tries to explain an ideal scenario which I'm sure shouldn't be fully implemented now. However, I tried to come up with a minimal-impact (but yes, still critical) solution. This would be the "management summary":
A new central observable-like component will send log entries to all providers. Providers will inherit only ILoggerProvider, an observer-like modified version of ILogger. This simplifies provider implementations (they only need one instance) and allows for more flexibility.
I don't think that breaking existing providers is a big issue because there already are other breaking changes for RC2 and this change wouldn't be very difficult. I'd be happy to send PRs.
Serilog and NLog are great products and I enjoy using them. However, people will try to have as few dependencies as possible, so you can't prevent people from making this library their first class logger solution. They will eventually build direct providers for most outputs. For that reason, I don't think that the architecture of this library should lack important extensibility points.
@nblumhardt, @304NotModified What's your opinion about this?

Too bad it's also unlogic that consolelogger won't implement ilogger. The name of consolelogger is now also not correct.
FYI they called for that reason "targets"  in NLog, so we have ilogger and consoletarget in NLog.

I agree, I kept that the same for now just to keep the changes minimal. They should not be called *Logger because they are not the same as instances that create log entries. (even with the current system IMO)
As I've said in my initial post, I also think that the name LoggerProvider is misleading. Something like *Sink, *Target, *Observer, *... would be better IMO.

I made further enhancements and I also added a POC for filtering. The changes to the core classes are now actually pretty small, however it's still a huge-impact PR. [it looks a lot less dramatic when you look at it locally instead of here on GitHub :-) ]
Here's the summary of changes in this PR. Stuff from my initial post is included here, so this summary is complete.
Log creation and log writing has been separated - introducing ILogSink
ILogger is now only used for creating log entries. The complete contract for providers is in the current ILoggerProvider interface. However, since this name no longer makes sense (it doesn't "provide loggers"), I've renamed it to ILogSink. Of course, this can change if anyone has better ideas.
The Abstractions-package no longer has any knowledge about providers/sinks
ILogSink was moved to Microsoft.Extensions.Logging. This makes it even clearer, that M.E.L.Abstractions is used for creating log entries and M.E.L for processing log entries.
IMO this also makes clear that M.E.L should be seen as a competitor to Serilog, NLog, etc and not as a supplier because it contains routing logic and has its own providers/sinks.
@nblumhardt @304NotModified With this change, it would now easily be possible for Serilog, NLog, etc to completely circumvent Microsoft.Extensions.Logging. Instead of implementing a provider/sink, you could just create your own ILoggerFactory and ILogger and have people inject it through IWebHostBuilder.UseLoggerFactory. This way, people who are using your frameworks would no longer have to use Microsoft.Extensions.Logging (although other libraries like Hosting will still bring in this DLL)!
I think this alone is a huge win. It's really unnecessary to send all logs through M.E.L if all of them should end up in Serilog, NLog, etc anyway. Reducing dependencies FTW! :)
Breaking changes for sinks/providers

Sinks must reference Microsoft.Extensions.Logging instead of Microsoft.Extensions.Logging.Abstractions (ILogSink is there)
Sinks must only implement ILogSink instead of ILoggerProvider and ILogger.
Sinks no longer have to maintain loggers per category. The ILogSink contains the categoryName in each method.
the Add*-methods (e.g. AddEventLog) have to target LoggerFactory instead of ILoggerFactory because AddProvider/AddSink no longer exists on the ILoggerFactory interface.

This is the contract of ILogSink:
public interface ILogSink: IDisposable {
  void Log<TState>(categoryName, logLevel, eventId, state, exception, formatter);
  bool IsEnabled(categoryName, logLevel);
  IDisposable BeginScope(categoryName, state);
}

ILogSinkProvider / LogSinkProvider
To make sure, not each category Logger in M.E.L has to maintain a list of sinks, there is a component called ILogSinkProvider which maintains sink-specific properties. It maintains a list of sinks and a filter object (see below). This component takes over the provider-specific logic of LoggerFactory. However, LoggerFactory still has AddSink for backwards-compatibility. (people can now still use e.g. AddEventLog on LoggerFactory if they want to)
M.E.L sink implementations
Console/Debug/EventLog/TraceSource have been renamed and changed to only implement ILogSink.
Filtering POC
Filtering is now built into the library. ILogSinkProvider can maintain a ILogFilter object which has a method bool IsEnabled(ILogSink sink, string categoryName, LogLevel logLevel).
M.E.L/Filtering contains an implementation of ILogFilter which contains many parts of #398. It's a zero-overhead solution if you don't want to use it (at least I think so).
There's not yet a Configuration-based implementation, but this could easily be added. The sinks also still have their own implementation.
.............
This code is still pretty much untested and some tests fail. I will continue to work on this, if you decide that we should proceed with it.

Not sure if it's too late. It's a fact that rc2 will have breaking changes.

It is too late. We're not going to take a change this big to the design of logging at this point. RC2 does have breaking changes but that's no excuse to make more.
On top of that, I'm not even sure I like the proposed changes. Especially the fact that custom sinks now have to use Logging instead of abstractions. I also don't understand how the application wires up the sink since it doesn't control creating the logger factory.

I added [Prototype] to the title to make sure it's clear that I don't expect this to be merged like this. I think it's just easier to discuss stuff if there's some actual code. It's also important that the main reason why this PR is so big is that it shows multiple things. I probably shouldn't have done that.

RC2 does have breaking changes but that's no excuse to make more.
In case of providers/sinks I think it actually is. There are probably less than 20 provider implementations in the world. Having the ILogSink interface would be a win for each of them. Maybe we get some feedback from those developers during the next days.

@davidfowl what to you think about the core idea of ILogSink? how about not renaming providers to sinks and just changing ILoggerProvider to contain the methods from my ILogSink interface? The naming wouldn't be great but it's still acceptable IMO. As you can see in the Debug and EventLog provider, this hugely simplifies provider creation.
Logger would then just keep a list of ILoggerProvider instead of ILogger if you don't want to introduce the proposed ILogSinkProvider container. This would be a really small change and wouldn't have any breaking changes for application developers.

Regarding moving ILoggerProvider away from the Abstractions package:
People here have said many times that we should use Serilog, NLog, etc because they have more features. So why should we be required to use Microsoft.Extensions.Logging at all? It seems highly unlikely that anyone will want to combine providers from M.E.L with sinks from Serilog because Serilog completely covers the M.E.L providers.
IMO, "providers" should only be used for actual outputs like console, files, sql, etc. Logging frameworks like Serilog, NLog, etc should be seen at the same layer as Microsoft.Extensions.Logging and should therefore be implemented as a custom ILoggerFactory. This would completely remove one layer.
But this is where ILoggerProvider and ILoggerFactory.AddProvider start to get problematic. These are concepts of Microsoft.Extensions.Logging's implementation. Serilog, NLog, etc don't need them. That's why I wanted to move it away!
However, I noticed that my idea of putting AddEventLog, AddConsole on LoggerFactory doesn't work. It must be an extension method to ILoggerFactory, because that's what's injected into Startup. So we would need a different solution. Maybe ILoggerFactory.AddProvider can be an extension method in Microsoft.Extensions.Logging.

I also don't understand how the application wires up the sink since it doesn't control creating the logger factory.

Now that we have IWebHostBuilder.UseLoggerFactory, we can actually create the logger factory. People just have to move their logging setup into the Main method. You don't have DI there but logging should be configured before DI anyway IMO.

On top of that, I'm not even sure I like the proposed changes. Especially the fact that custom sinks now have to use Logging instead of abstractions.

We could also introduce another package called Microsoft.Extensions.Logging.ProviderAbstractions which contains the ILoggerProvider-interface. This just seemed like an overkill to me.
The important thing for me was, that the interface shouldn't be in the regular Abstractions package. See previous post for the reasons.

I'm closing this as it contains too many things. Also, filtering has now been merged into dev. I hope I didn't annoy you with this PR... I'm pretty new to this open source thing.
Maybe I will create a separate and smaller PR that shows just the changes to ILoggerProvider. Would still be a breaking change though.


Update Argilla start page

[ ] Change link to docs from Rubrix to Argilla documentation
[ ] Add a tab with a code snippet for Feedback datasets

Code snippet for feedback datasets:
# install datasets library with pip install datasets
import argilla as rg
from datasets import load_dataset

# load an Argilla Feedback dataset from the Hugging Face hub
dataset = rg.FeedbackDataset.from_huggingface("argilla/oasst_response_quality", split="train")

# push the dataset to Argilla
dataset.push_to_argilla("oasst_response_quality")


Remove taint usage on Ruby 2.7+

What was the end-user problem that led to this PR?
Ruby 2.7 is deprecating taint support.
See https://bugs.ruby-lang.org/issues/16131 for details.
What was your diagnosis of the problem?
Bundler still uses untaint in a few places, which will cause deprecation warnings on Ruby 2.7.
What is your fix for the problem, implemented in this PR?
s/\.untaint/.tap{|x| x.untaint if RUBY_VERSION < "2.7" }/g
Why did you choose this fix out of the possible options?
It seemed simplest.

@bundlerbot r+


Dump Database not functioning

Description
Attempting to dump a database using DBeaver, clicking start results in the dump window flickering, unresponsive and closing the whole program results in this error :
DBeaver Version
Community <IP_ADDRESS>405051837
Operating System
Windows 11
Database and driver
MySQL 5.7
Steps to reproduce
Connect to DB
Highlight DB and select 'dump database' from tools
Select the DB and tables to dump and click 'start'
Additional context
going via SSH tunnel

Using different clients result in the same outcome

Please, provide the log files. You can find instructions here https://dbeaver.com/docs/wiki/Log-files/.

It's been a while since no update here. If the issue is still actual please let me know and provide additional information.


Remove unused partial in zone specs

Within the zone specs, the allow(ServerRole).to receive(:seed_data).and_return(@csv) line breaks strict partial verification since ServerRole doesn't implement a seed_data method. So, it isn't actually doing anything, and the specs pass without it.
While I was here I also updated the outer describe block to RSpec.describe to be future compliant with rspec4.
Similar to https://github.com/ManageIQ/manageiq/pull/18730

@miq-bot add_label test, refactoring


Update README.md

Added periods and colons where appropriate

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: shlavocky sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


Iterpretation of and candidates selection according to rank and/or confidence

I'm a little bit confused regarding selection of the best candidate from multiple ranked ligand poses. I see for instance these result files after a DiffDock run with default settings and have a couple of questions:
rank1.sdf
rank1_confidence0.34.sdf
rank2_confidence0.24.sdf
rank3_confidence0.22.sdf
rank4_confidence0.20.sdf
rank5_confidence0.17.sdf
rank6_confidence0.15.sdf
rank7_confidence0.07.sdf
rank8_confidence-0.04.sdf
rank9_confidence-0.04.sdf
rank10_confidence-0.33.sdf

Obviously, the confidence also take negative values (rank8-10), is that true?
Are the smaller or larger confidence associated with higher result confidence?
Does the rank (no. 1-10) have higher priority than the confidence value or vice versa regarding the selection of favorable candidates?

Strangely, I sometimes observe confidence values that monotonically increase from rank1 to rank 10  (with rank1 confidence close to 0, above example) and sometimes the opposite, monotonically decreasing (with rank1 confidence close to 1). But never (so far) a unsorted (non-monotonic) relation between confidence and rank. Isn't that possible or kind of a bug?

Is it fine to have multiple confidence for a particular rank like below
rank2_confidence-0.59.sdf
rank2_confidence-0.65.sdf
rank2_confidence-0.68.sdf


consul 0.8.0 and later failing to cluster

Hi,
I've just tried the stable/consul chart with ImageTag set to 0.8.0 (.1 and .2) and consul fails to form a cluster when deployed on a minikube test system.
The following errors are thrown by consul in the logs:
2017/05/12 17:54:15 [ERR] memberlist: Failed push/pull merge: Member 'youthful-heron-consul-2' has conflicting node ID '78788416-6a6d-1bf9-9d20-06f082e658f3' with this agent's ID from=<IP_ADDRESS>:37822
2017/05/12 17:54:15 [ERR] memberlist: Failed push/pull merge: Member 'youthful-heron-consul-1' has conflicting node ID '78788416-6a6d-1bf9-9d20-06f082e658f3' with this agent's ID from=<IP_ADDRESS>:35528

I'll do some research into why this might be and update this issue with any further info.

This is discussed more in hashicorp/consul#2877, looks like the -disable-host-node-id option was added in hashicorp/consul#2904.
I've gone ahead and modified a local copy of the chart, and can confirm this works with >=0.8.0.
The node-id is written to the persistent disk, meaning the node-id will survive restarts of a pod:
❯ kubectl get pods
NAME                    READY     STATUS    RESTARTS   AGE
hazy-whippet-consul-0   1/1       Running   0          2m
hazy-whippet-consul-1   1/1       Running   0          1h
hazy-whippet-consul-2   1/1       Running   0          1h
❯ kubectl exec -it hazy-whippet-consul-0 /bin/sh -- -c 'cat /var/lib/consul/node-id'
cc9f2e75-0b90-2533-40df-680826ca7b71
❯ kubectl delete pod hazy-whippet-consul-0
pod "hazy-whippet-consul-0" deleted
❯ kubectl exec -it hazy-whippet-consul-0 /bin/sh -- -c 'cat /var/lib/consul/node-id'
cc9f2e75-0b90-2533-40df-680826ca7b71

oh hell, never mind... it is now converging (.  it wasn't before but now it is... hmmm... that is comforting :/

This should be resolved. Closing


[issue-331] Make connector Generated ID consistent among checkpoints

Signed-off-by: Brian Zhou<EMAIL_ADDRESS>Change log description

Make connector Generated ID consistent among checkpoints

Purpose of the change
Fixes #331
What the code does

Removed the stream cuts info in generateUid function because checkpoint restore will change the starting streamcut
Changed unit tests for generateUid

How to verify it
./gradlew clean build passes.
Target to master, should cherry-pick to all 0.7 branch

@fpj can you please add me to the owner of this repo and remove the codecov status as a requirement? This PR removed some lines so the coverage drops, it is now blocking the merge.

The report is complaining about the absence of coverage for some of the code lines in this file:
src/main/java/io/pravega/connectors/flink/serialization/JsonSerializer.java
Perhaps the test cases you are modifying are inducing this code coverage reduction unintentionally. Have a look, please.


docs: Deprecate the underlying send_reliable_submission

High Level Overview of Change
Title says it all.
Context of Change
submit_and_wait is the newer version of that function with more features & future support.
Note: I moved the internals of send_reliable_submission into a private helper because submit_and_wait relies on that functionality, but should not trigger deprecation warnings each time it's called. (Basically separating the "public face" which is deprecated, from the helper)
Type of Change

[X] Documentation Updates

Test Plan

CI Passes (and does NOT mention the deprecation - I checked beforehand to verify the deprecation was working, but don't want to clog the logs going forward)

Why deprecate it and not just delete it? It's an easy enough migration for people using it who are upgrading to 2.0.

The thought process was that this is the most commonly used function, and so we were worried that people wouldn't update if it was changed.
That being said - it is just a search + replace, so I see it being fine either way.

Closing since we've decided to remove send_reliable_submission instead of deprecating. Will open a separate PR and link here afterwards.


feat: implement Item.first()

Implement static method .first() which should return the first instance of an item.

Closed by 38716b5f5bdea6aa83976475e7cc72d829f09ed5.


[QUESTION] Get collection size or how can calculate the collection size Version 5

Hello,
give a function how can read the collection size out of the box?
or
How can the calculate this?
In the version 4 i have claculate this about the singel documents with "GetBytesCount(true)" but this is not working on liteDB 5.
I switch liteDB 4.1.4 to 5.0.8.
Have you an Idee? Thank you

@foxshooter If you want to read the total size occupied by a collection, you can do it with the following command:
int bytes = db.Execute("select SUM(*.usedBytes) from $dump where pageType = 'Data' and collection = 'collection'").Single()["usedBytes"];

It's a bit convoluted, but it works fine. Of couse, you need to change the collection name in the query. We'll consider adding a property to the LiteCollection that gets this count.

@lbnascimento Thank you.
Can you explain why pageType='Data' and which Type can use?

@foxshooter If you leave out the pageType='Data', this will also include Index pages.

@lbnascimento okay,
Thank you!
You helped me a lot.


Editing Task content focus keyboard problem

When editing a task, only when clicking on the title textfield will the keyboard show up. Otherwise, even if you keep clicking in the content textfield, the keyboard stays hidden.

Thanks @Bodaymat , the issues is already fixed and it will shows keyboard either you click on title, body, or one of them twice

Thanks @Bodaymat , the issues is already fixed and it will shows keyboard either you click on title, body, or one of them twice


fix: fix user relationship test

Changes

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#616  👈
#615
#596
#583
#582
#581
#580
#578
#573
#572
#571
#570
#566
#565
#579
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @MasterPtato and the rest of your teammates on  Graphite


WARNING: The MNEMONIC variable is not set. Defaulting to a blank string.

When I am starting the docker-compose then the mnemonic does not picked from .env file as mentioned in comment in docker compose file.
  # This env variable comes from the .env file in the directory in which the
  # docker-compose command is run. Here it will be ./packages/sidetree-ledger-ethereum

and due to that every time, when the container is being restarted a new set of keys are being assigned.
I am starting the container, just by right click on the docker-compose in vscode by selecting "Compose up" option.

hmm, seems like we probably need to separate the docker compose files so that running a local node is easier.

I ll let it open so that you can close it, when it done in future releases

I think I've seen this when starting up a server. Want to see if it's happening, and why it's happening, because the seed should be defined in the configuration file.

Taking some notes, it looks like this error can be reproduced by running the test for the cas-ipfs module.
><EMAIL_ADDRESS>pretest ~/Github/sidetree.js/packages/cas-ipfs
> docker-compose up -d ipfs mongo

WARNING: The MNEMONIC variable is not set. Defaulting to a blank string.
sidetreejs_ipfs_1 is up-to-date
sidetreejs_mongo_1 is up-to-date

><EMAIL_ADDRESS>test ~/Github/sidetree.js/packages/cas-ipfs
> tsdx test --passWithNoTests

So we need to track down where that warning is coming from, and if it's needed or not.

This should be defaulted to an example.


Cumulative Distribution Functions

Feature Request
Proposal: Addition of Cumulative Distribution Function as another function exposed to users.
Current behavior: No existing alternative that I am aware of.
Desired behavior: Running data through CDF should result in probability computation as expected from a traditional CDF function, where given data and a value to compare against that data, a probability of that value and anything less than that value being in the distribution is produced.
Use case: This is a useful function for prediction of likely values in the dataset(s), as well as for measuring likelihood of presence of any given value in the data.

Can you give examples of the types of functions you want to see?

+1
PDF/CDF would be quite useful. This issue seems like a good candidate for inclusion in #5930 - similar to the request for Histogram

Moved this to #5930.

Sorry about not replying, I completely missed updates on the thread. Yes, PDF/CDF would be great, right along with histograms.


Fix upstream

[x] Closes #7535
[x] Tests added / passed
[x] Passes black dask / flake8 dask / isort dask

Something might have changed in pandas groupby code, but this seems like a good solution for cases where there might be only one partition in a tree reduction. I didn't check to see if there are more cases where this pattern needs updating.

Good with you @jrbourbeau? I grepped around for groupby(level=0) and reset_index(level=1 and I don't see anywhere else that would have the same issue. So I think this can be merged and doesn't need to be treated as an indicator of a larger issue.


chrome toDataURL() only works on hard reload?

This not sure what the actual cause of this is but made a quick sandbox to demonstrate.
Use Chrome & Open https://yr1r3kkn9.codesandbox.io/
Normal page load toDataURL returns blank after clicking combine

Empty Cache and Hardreload  toDataURL will return properly after clicking combine

Firefox seems to handle toDataURL correctly from the start

I'm assuming the actual problem might be related to konvajs Issue 264
Using: "react-konva": "1.1.6",
Chrome Version

it seems that starting chrome with --disable-web-security flag bypasses the issue

On further inspection I don't think this is related to the canvases somehow being tainted.
In the codepen it looks like the initial load always works, also hard reload + reload cache always works.
subsequent reloads fail

This appears to happen because of cached images in chrome.  Not certain how/why, however appending a timestamp to the img src appears to remove the problem.
From this:
image.src = "http://i.imgur.com/fHyEMsl.jpg";
To This:
image.src = "http://i.imgur.com/fHyEMsl.jpg?" + new Date().getTime();


Lots of Telemetry Warnings

When I compile or run Tableau, I get many Telemetry Warnings

I get the same behavior on Elixir 1.15 and 1.16
I believe the warnings come from a library schematic in the telemetry_wrap function of schematic.ex.
Here's my workaround to remove the telemetry warnings:

Does anyone else see these warnings?
Are the telemetry metrics in Schematic necessary?
Is there a way to suppress the telemetry warnings without altering the Schematic code?

It happens because schematic is being run at compile time, but the telemetry application hasn't started yet.
I haven't looked closely into how to fix it yet.

Gotcha.  If you're up for it, I could submit PR to schematic to disable telemetry at compile time, and/or to allow application config to optionally disable schematic telemetry at run time.  If you'd rather defer, no problem - lmk.

I think it makes more sense to figure out how to start the telemetry application before the modules compile, or (which i think i'll do anyway) rewrite the parts that use schematic at compile time to use it at runtime.

Sounds good.  Just FYI here is the predicate code I used in my fork:
defmodule Schematic.TelemetryCfg do
  @moduledoc """
  Manage telemetry settings for Schematic.

  This module performs two functions:

  1. check to see if the current execution environment is `run-time` or
    `compile-time` (we want to disable telemetry at compile time).

  2. check to see if the application config has disabled telemetry

  ## Example

  In `config/config.exs`:

  ```elixir
    config :schematic, :config, disable_telemetry: true
  \```
  """

  @disable_telemetry_predicate Application.compile_env(:schematic, :config)[:disable_telemetry]

  @doc """
  Predicate used to determine if telemetry should be enabled in `Schematic`.

  To optionally disable schematic telemetry in your application, enter the following into your `config/config.exs` file:

  ```elixir
    config :schematic, :config, disable_telemetry: true
  \```
   """
  def enable_telemetry? do
    running?() && telemetry_allowed?()
  end

  defp telemetry_disabled? do
    @disable_telemetry_predicate
  end

  defp telemetry_allowed? do
    !telemetry_disabled?()
  end

  # from https://elixirforum.com/t/is-it-possible-to-detect-if-code-is-executing-at-compile-time/57728

  def compiling? do
    # TODO: When we depend on Elixir v1.11+ only, remove function_exported and elixir_compiler_pid
    process_alive?(:can_await_module_compilation?) ||
      process_alive?(:elixir_compiler_pid)
  end

  defp process_alive?(:can_await_module_compilation?) do
    Code.ensure_loaded?(Code) &&
      function_exported?(Code, :can_await_module_compilation?, 0) &&
      apply(Code, :can_await_module_compilation?, [])
  end

  defp process_alive?(name) do
    case Process.get(name) do
      nil -> false
      pid when is_pid(pid) -> true
    end
  end

  defp running? do
    ! compiling?()
  end
end

a partial fix i believe is released with 0.14.2

This should be completely fixed with 0.14.3


SSO from drupal + uid caching

Done on the test Drupal server for now.
We need to have an SSO functionality so that we can authenticate users and bring related info.

A view in the Drupal database that aggregates all relevant user info
An endpoint on Spark for authenticating and consequentially retrieving user data

@omerpines / @amotenko - will be best to get access to the drupal code, I've been asking for this for a long time, what's the hold-up?

@OriHoch , we can get access to the test server but not production.
It's a policy we'll have to live with.
Once the solution is ready we will be able to port it to the production server (and maybe then we'll get access :)

I don't want access to the server, I want to see the code and be able to run a local drupal environment locally
this will allow to test the integration and SSO functionality, without this development is much harder and it's possible but before diving to working like that I would like to know why...

if the hold-up is just that there is no one that can do it - I  am happy to do that
give me access to the test server and I will remove any sensitive data and upload to github

Sorry for misreading.
That would be very cool. But let me just ask, when you say drupal code you actually mean the MySQL files themselves with all the table definitions, constraints and stored procedures?

no, I mean the actual drupal modules code (PHP code) - to allow running the actual drupal locally and testing the SSO functionality
just SQL data is not enough

I see. I have no idea where it is but I can try to find that.
Someone also suggested to ignore the Drupal on top, and just create a view on the MySQL instance itself that will bring back the required info (and obviously connect to it from Spark). What do you think about such solution?

I think that to choose the right solution and implement it (whichever one we choose) we will need to look at the drupal php code and be able to run drupal locally

I'll try to find it

@OriHoch, I've opened a request for DevOps to prepare a key for you:
http://jira.midburn.org:8080/browse/DEVOPS-10

thanks @omerpines
@amotenko  - if you think it's worth for me to get the drupal code please let me know, I might do it anyway but it would be better to know if you think it's actually needed or not

@OriHoch To be honest I'm not sure it's super useful for us, we just need to 'hack' the database, in my understanding.

no problem, I'll handle it in this issue - #131

works


Missing documentation regarding git based links and how they are treated when converted to Confluence page

Hi @kovetskiy
I have looked at #25 and in the README, but I could not find clear enough explanation on how the behavior got changed/enhanced after #25 got merged.
Any details regarding what to expect and how to design an MD file to auto generate correct links, especially those links that refer to other MD files in the same git repo, will be most appreciated

@prokod Hi
Here is a thing — this feature is still in a half-working state because it doesn't work perfectly out of the box.
For example, we want to have a link between alice.md and bob.md:
alice.md
<!-- Space: MARKTEST -->
<!-- Parent: mark -->
<!-- Title: Alice -->

Hello.
Here is the link on Bob: [bob](bob.md)

bob.md
<!-- Space: MARKTEST -->
<!-- Parent: mark -->
<!-- Title: Bob -->

Alice: [alice](alice.md)

If you run mark against alice.md and bob.md you will see two different results.
The confluence page called Alice will have a link to https://bob.md because the content of bob.md is not in Confluence at this moment, so Mark was not able to find it there and just put the link "as is".

Running mark against bob.md now will generate a correct link because mark will be able to find the Alice page in Confluence by the space/parent/title.

Therefore, you'll need to run mark against alice.md again to get the expected result.

This feature works without bugs, it's just the way how mark works — it deals with single files instead of dealing with a set of files.
But it still kind of sucks and I don't know how to deal with it yet. Calling mark twice is definitely not a good solution.

@prokod Hi
Here is a thing — this feature is still in a half-working state because it doesn't work perfectly out of the box.
For example, we want to have a link between alice.md and bob.md:
alice.md
<!-- Space: MARKTEST -->
<!-- Parent: mark -->
<!-- Title: Alice -->

Hello.
Here is the link on Bob: [bob](bob.md)

bob.md
<!-- Space: MARKTEST -->
<!-- Parent: mark -->
<!-- Title: Bob -->

Alice: [alice](alice.md)

If you run mark against alice.md and bob.md you will see two different results.
The confluence page called Alice will have a link to https://bob.md because the content of bob.md is not in Confluence at this moment, so Mark was not able to find it there and just put the link "as is".

Running mark against bob.md now will generate a correct link because mark will be able to find the Alice page in Confluence by the space/parent/title.

Therefore, you'll need to run mark against alice.md again to get the expected result.

This feature works without bugs, it's just the way how mark works — it deals with single files instead of dealing with a set of files.
But it still kind of sucks and I don't know how to deal with it yet. Calling mark twice is definitely not a good solution.

@kovetskiy Thanks! this explanation is clear now. Thx again!

@kovetskiy Thanks! this explanation is clear now. Thx again!

@kovetskiy Hi again,
What about the working dir you run the tool from and the links created ?
I have the following tree:
|-roor-dir
  |- README.md
  |- dir-a
    |- dir-b
       |- toc.md
       |- chapter-a.md

README.md has inside it a link to toc.md -> this works fine when syncing to Confluence
toc.md has inside it a link to chapter-a.md -> this breaks already - the link is incorrect

I am running mark from root dir
Any idea ?

@kovetskiy Hi again,
What about the working dir you run the tool from and the links created ?
I have the following tree:
|-roor-dir
  |- README.md
  |- dir-a
    |- dir-b
       |- toc.md
       |- chapter-a.md

README.md has inside it a link to toc.md -> this works fine when syncing to Confluence
toc.md has inside it a link to chapter-a.md -> this breaks already - the link is incorrect

I am running mark from root dir
Any idea ?

Unfortunately markdown links are not are not being replaced with Confluence links for me either. My project has the following structure:
|- apps
   |- README.md
   |- app1
      |- README.md
   |- app2
      |- README.md

When I put links in apps/README.md like this:
[App: app1](app1/README.md)

then run Mark using the file, the link in Confluence just appends app1/README.md to the end of current the URL.

Hi @kovetskiy
couldn't it work like mark would

check for a relative! link (by convention considered to belong to the documentation 'suite' and managed by mark) to another markdown (*.md) file,
get the title of the other file (the metadata headers from within the file or its H1)
compute the link to a valid confluence link, like replace   with + and respecting other linked file's metadata (like space key - defaulting to current space key) and whatever else is needed.

For 3. the 'compute'-logic to create a confluence link would most likely be manually reproduced within mark if the confluence API could not be leveraged for this. In theory, the compute logic could change with newer confluence versions so mark should internally choose the correct algorithm for the recognized version.
Does this make sense? I hope this idea would cover every edge-case.
I think, this would be the key feature to close the gap between engineering people's way-of-working and maintainable and scalable corporate-level documentation requirements


Issue found on page 'PostgreSQL Compatibility'

Page URL: https://duckdb.org/docs/sql/dialect/postgresql_compatibility.html
DuckDB does not support to_date (https://www.postgresql.org/docs/current/functions-formatting.html). The documentation should reflect that and link to https://duckdb.org/docs/sql/functions/dateformat#strptime-examples.

@benjaminbauer thanks for the suggestion, I added this limitation (+workaround) to the Postgres-compatibility page.


Rename ToolsDB data source

The current database name is wrong, the real database name is specified in the connection URL and is "information_schema".
As discussed in the Phab task, it makes sense to call this data source simply "ToolsDB" and let users select their desired database in their SQL queries.
Bug: T367393

@vivian-rook I see that T372395 is now resolved, can I try deploying this PR with deploy.sh?

@vivian-rook I see that T372395 is now resolved, can I try deploying this PR with deploy.sh?

Go for it!

I deployed this PR, I can see the new name, but the old one is also there:

Might be hiding somewhere in the database. We could deploy a fresh system, that would probably remove the old entry.

kubectl get configmap superset-extra-config -o yaml shows that the old name is no longer present in the k8s config.
@vivian-rook do you know why the old name is still showing in the UI, next to the new one?

Sorry, I posted my comment before reading yours :)

What's the procedure to deploy a fresh system? tofu destroy?

For how it would be redeployed, it is poorly described in the README. It follows the same procedure in https://wikitech.wikimedia.org/wiki/PAWS/Admin#Blue_Green_Deployment then the database migrate steps described in the README

Thanks! The UI issue is now fixed (did it require a full redeploy?), so I think this can be merged.

No a redeploy was not required. The database only had one dataset so the UI allowed me to remove it after removing the dataset (For some reason the UI won't allow you to remove objects that are integrated. Can get very problematic sometimes)


refs #714. L.CircleMarker can be edited and drawed for leaflet 1.0.3

As Circle inherits from CircleMarker my first try was to create a Draw.CircleMarker class as parent of Draw.Circle. But as most of the needed functionality was already present in Draw.Marker and commonly CircleMarker as expected to behave like "markers without icons", finally a make Draw.CircleMarker a child of Draw.Marker.
Then I tried to make Edit.CircleMarker a child of Edit.Marker, but CircleMarker is not "draggable", so some errors arise when trying to use an implementation that relies on marker/draggable capabilities like
L.Edit.CircleMarker = L.Edit.Marker.extend({

});

L.CircleMarker.addInitHook(function () {
	if (L.Edit.CircleMarker) {
		this.editing = new L.Edit.CircleMarker(this);

		if (this.options.editable) {
			this.editing.enable();
		}
	}
});

Just to have some working code and open up the discussion I implement Edit.CircleMarker as a parent of Edit.Circle following the hierarchy of Leaflet.
So this is working but there are some caveats:

An icon for the draw toolbar for circlemarker is needed. Now is using the same as marker, but probably it makes sense to create a new one based on circle (for example with the center of the circle empty)

The actual hierarchy does not have so much sense, Edit and Draw should follow the same hierarchy, or just be created as independed. ¿Any idea here?

Thanks for working on this! I built Leaflet.draw myself (as you described in #714). The minified version doesn't work (it only contains undefined), but leaflet.draw-src.js works fine. Thanks!
Your caveats:

I don't know if a separate control icon for circle marker is needed. We only use CircleMarker in the pointToLayer method. Additionally there was no control icon for CircleMarker in the official releases. Thus I think you ignore this.
I'm not really into the Leaflet structure, maybe someone with more knowledge of leaflet can help you.

Thanks for your comments. Glad it works for you.
I haven't work with jake previously so I don't know how the dist is created, I just copy&paste the README instructions :)
I hope that some of the Leaflet.Draw committers have time to review the PR and suggest how to improve it to be merged.

@ddproxy Could you help here or push me or @fpuga in the right direction or what it needs to be merged?
In our project we rely on bower packages, so self-compiling and cherrypicking is not a satisfying option. Since we really need this feature I'm willing to help.
It's sad to see all these great contributions, which seem to be ignored. I completly understand if people don't have much time, but please leave a short message to indicate that working on these PRs is not a complete waste of time.

I think this is fine, I would just like to have a different icon for the circle marker.

@ddproxy @v1r0x Thanks for taking a look at this. I push another commit with an icon.
I have not worked before with sprites, neither have many experience with inkspace. Let me know if it's good for you or some change is needed.

@fpuga thanks for fixing it! Icon itself looks good to me, but I think it is wider than the other icons. The old .svg was 540px and 9 icons (= 60px per icon), the new one added one icon, but 70px instead of 60px. Same for the smaller version (30px vs. 35px). I know this is nitpicking :wink:
Other stuff looks good so far! Happy to see this is going on :)

Thanks for the review. I think is fixed now. Just let me know if more changes are needed.
I can open a new branch/pr if you want to avoid noise in the commits.

Thanks again! One thing I didn't see yesterday, unfortunately. The added icon is a bit darker than the other icons. Would be nice to have them all in the same color.

I can open a new branch/pr if you want to avoid noise in the commits.

You can squash your commits to combine them. Run git rebase -i ba620f69dddb82f303558f76ffc8e73daa482be9. The ID after -i is the commit ID of the commit before the commits you want to squash. Then you can pick the first commit (8f8242f8a6919ded85a242dc594e3ed54a9ade15) and squash the other icon related commits (53aff4fc3cc1067d2e82775fb0c4064c4d6feb8f and maybe others) on it.

I hope that now it's correct. I neither catch that the colour was different. Sorry for the inconvenience. Don't hesitate to ask for more changes if you think that are needed. I rebase the branch against the actual master to make the merge easies.

Looks good! No more change requests from my side 😉

Ok to merge @ddproxy ?

👍 Slick

Thanks to both


Swap Router Fixes & Improvements

Scope

Adds two helper functions to create bi-directional edges for the following cases:
a. Exchanging between two tokens in a pool
b. Moving from an underlying token -> LP token via AddLiquidity, and vice-versa via RemoveLiquidityOneToken
Adds tests for the helpers to ensure both edges (A -> B and B -> A) are accounted for
Introduces tests for some route paths where we want to ensure the router takes the most direct path. Notably:
a. 3CRV -> underlying should always remove liquidity, and vice-versa
b. Stables in 3pool should swap directly on 3pool, no intermediate hops
Add helper expectInstanceOf which is the same as expect.instanceOf` but with a Typescript assertion so any following context is aware of the assertion
Change all action constructor parameters to public readonly

Rationale
Helper functions

The graph.ts file was becoming quite verbose, with lots of repetitive code to define edges
Some swap routes were missing their counterparts, for ex. we had 3CRV -> USDC but not USDC -> 3CRV mapped.

public readonly

Allows for tests to build an action and verify that the values stored in the instance match what you would expect based on router. example:

      const route = sdk.swap.router.getRoute(sdk.tokens.CRV3.symbol, symbol);
      const step = route.getStep(0).build(account, FarmFromMode.EXTERNAL, FarmToMode.INTERNAL);

      expect(route.length).toEqual(1);
      expectInstanceOf(step, RemoveLiquidityOneToken);
      expect(step._pool).toEqual(sdk.contracts.curve.pools.pool3.address); // <-- this

Javascript doesn't actually understand the concept of private, and public readonly will do effectively the same thing with respect to Typescript's warnings

Addition of the 3CRV <> Stable route
Some combinations of this route previously performed inefficient swaps, typically something like 3CRV -> BEAN -> USDT for example.
Testing
Automated: see graph.test.ts
Manual: I rebuilt the SDK locally, set up the following swap paths, and confirmed via console logs that the correct path was selected:

[x] DAI -> USDC
[x] DAI -> USDT
[x] USDC -> USDT
[x] 3CRV -> DAI
[x] 3CRV -> USDC
[x] 3CRV -> USDT
[x] reverse of each of the above

FYI: The above Netlify comment seems to indicate that the UI won't get rebuilt if there's a change in the SDK code., i.e. this won't result in a deploy preview. We might want to change that behavior to also include changes to the SDK itself.

I ran the test suite locally, only seeing this error:
Summary of all failing tests
 FAIL  projects/sdk/src/lib/swap/Swap.estimates.test.ts (5.031 s)
  ● Estimate › Estimate BEAN->x › estimateReverse(ETH, 3CRV)

    call revert exception [ See: https://links.ethers.org/v5-errors-CALL_EXCEPTION ] (method="get_dy(uint256,uint256,uint256)", data="0x", errorArgs=null, errorName=null, errorSignature=null, reason=null, code=CALL_EXCEPTION, version=abi/5.7.0)
    ```

From Discord thread:

TL;DR the changes included in #469 break the reverse estimation of a route involving [DAI/USDC/USDT]->3CRV, because the AddLiquidity and RemoveLiquidityOneToken workflow actions don't account for the workflow "direction".
For context, reverse estimation is used on the Swap page to allow users to type a desired output value into the second input and receive an estimate for how much they'd need to input.
Currently, this "estimation" is performed by assuming that there exists zero hysteresis in swaps. Ex. if I swap 1 USDC -> 0.99 DAI, if I go from DAI back to USDC I should end up with 1 USDC when everything is done. In practice this is not true due to fees and slippage, but for the sake of simplicity and speed we chose this approach for the UI. There is no deterministic way AFAIK to correctly perform the reverse estimation, the best you can get besides this zero-hysteresis assumption is an iterative approach which is highly inefficient.
In other words, when we want to estimate how much input is needed for a particular output, we start by running the entire estimation process backwards to estimate the input amount. When we finalize the transaction we run the transaction forwards to generate the final calldata.
Shipping PR #469 will cause a reversion when going to the Swap page and selecting [DAI/USDC/USDT] as input and 3CRV as output, then typing in the 3CRV box.
Given that this PR fixes a number of other inefficient routes, and the above route is still executable (you just have to set it up by only typing into the first box instead of the second), I would suggest acknowledging this bug will exist in prod and moving fwd with the PR anyway.
If a user tries to perform the route, they'll see an error message toast that reports "call revert exception" when trying to type into that output box.
As followup work, I can extend my comments on this PR to a list of future changes that would fix this issue & better structure those two actions (they need some refactors).

#475 solves the above with an error. Merging!


PolicyUtil Error in retrieving throttle policies

Description
[2022-06-16 13:26:35,690] ERROR - PolicyUtil Error in retrieving throttle policies
org.wso2.carbon.apimgt.throttle.policy.deployer.exception.ThrottlePolicyDeployerException: {"code":901300,"message":"Bad Request","description":"Provided tenant domain '*' is invalid","moreInfo":"","error":[]}Event-Hub status code is : 400
at org.wso2.carbon.apimgt.throttle.policy.deployer.PolicyRetriever.getPolicies(PolicyRetriever.java:203) ~[org.wso2.carbon.apimgt.throttle.policy.deployer_9.22.4.SNAPSHOT.jar:?]
at org.wso2.carbon.apimgt.throttle.policy.deployer.PolicyRetriever.getAllSubscriptionPolicies(PolicyRetriever.java:87) ~[org.wso2.carbon.apimgt.throttle.policy.deployer_9.22.4.SNAPSHOT.jar:?]
at org.wso2.carbon.apimgt.throttle.policy.deployer.utils.PolicyUtil.deployAllPolicies(PolicyUtil.java:159) ~[org.wso2.carbon.apimgt.throttle.policy.deployer_9.22.4.SNAPSHOT.jar:?]
at org.wso2.carbon.apimgt.throttle.policy.deployer.utils.ThrottlePolicyStartupListener.run(ThrottlePolicyStartupListener.java:76) ~[org.wso2.carbon.apimgt.throttle.policy.deployer_9.22.4.SNAPSHOT.jar:?]
at java.lang.Thread.run(Thread.java:829) [?:?]
Steps to Reproduce
N/A
Affected Component
APIM
Version
4.1.0

There are two more error logs as well.

This issue will be fixed as a consolidation of a different fix [1].
[1] https://github.com/wso2/carbon-apimgt/pull/11293

Closing as this as this is fixed via https://github.com/wso2/carbon-apimgt/pull/11651 as per discussion with @tharikaGitHub