Unable to perform subsampling using tf.data.Dataset map() method.

Please make sure that this is a feature request. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:feature_template
System information

TensorFlow version (you are using): 2.3.0
Are you willing to contribute it (Yes/No): No

Describe the feature and the current behavior/state.
The preprocessing of sequences can easily be a bottleneck in the Deep Learning pipeline. And tf.data.Dataset provides us ways to optimize the input pipeline. But there are limitations to using a custom python function as a preprocessing step. For example, I am trying to train a model to predict the next item(an item can be a movie, song, book etc), given a history of the items consumed. And I want to use a subsampling function to be passed in the map method. The main challenge here is that I can not pass the item2prob dictionary to my custom subsampling function through the map method. And I am not sure if there are any methods available to pull this off.
Will this change the current api? How?
The current API will support python dictionaries to be passed into a custom python function through the map method.
Who will benefit with this feature?
This feature will open up possibilities for people trying to optimize their input pipeline using custom preprocessing functions.
Any Other info.
Here is what I have trying to do-
# make a python generator

def get_generator_random():
    '''
    yields 10 sequences of varying lengths (3 to 7) with values ranging from 0, 5
    '''
    num_seq = 10
    seq_count = 0
    while seq_count < num_seq:
        rand_seq_len = np.random.randint(3, 8)
        seq = np.random.randint(0,6,rand_seq_len)
        yield seq
        seq_count += 1

py_gen = get_generator_random()
for i in py_gen:
    print(i)

Output:
[2 2 2 3 5 4]
[3 2 0]
[0 2 1 3 4]
[2 4 4 2 2 4 2]
[5 1 3 0 4 5 4]
[2 5 4 5]
[0 2 2 2 0]
[0 2 3]
[0 4 0]
[2 2 1 4 0 1 0]

# make tf dataset from python generator

def get_tf_gen():
    tf_gen = tf.data.Dataset.from_generator(
        get_generator,
        output_types=tf.int32,
        output_shapes = (None, )
    )

    return tf_gen

tf_gen = get_tf_gen()
for i in tf_gen:
  print(i)

Output:
tf.Tensor([2 2], shape=(2,), dtype=int32)
tf.Tensor([5 1 2 3 4], shape=(5,), dtype=int32)
tf.Tensor([5 1 2 3 4 5], shape=(6,), dtype=int32)
tf.Tensor([5 1 2 2 4 5], shape=(6,), dtype=int32)
tf.Tensor([3 1 2 2 4 2], shape=(6,), dtype=int32)
tf.Tensor([1 1 2 3 3 5], shape=(6,), dtype=int32)
tf.Tensor([2 1 5 3 1 5], shape=(6,), dtype=int32)
tf.Tensor([5 2 2 4 4 2], shape=(6,), dtype=int32)

# define item2prob (prob of removing an item)

item2prob = {}
for i in range(1, 9):
    if i < 5:
        item2prob[i] = 1
    else:
        item2prob[i] = 0.0

print(item2prob)

{1: 1, 2: 1, 3: 1, 4: 1, 5: 0.0, 6: 0.0, 7: 0.0, 8: 0.0}
def subsample(seq):
    seq_new = []
    random_numbers = tf.random.uniform(shape=(len(seq), ), minval=0, maxval=1, dtype=tf.dtypes.float32,).numpy()
    for s_idx, s in enumerate(seq):
        if random_numbers[s_idx] >= item2prob[s]: # this is where the error occurs
            seq_new.append(s)
    return seq_new

def tf_subsample(seq):
    seq_new = tf.py_function(subsample, inp=[seq], Tout=tf.int32)
    return seq_new

tf_gen = get_tf_gen()
tf_gen = tf_gen.map(tf_subsample)
tf_gen = tf_gen.padded_batch(4, padded_shapes=5)

print('\nGenerating data...')
for i in tf_gen:
    print(i)

Following is the error message-
Generating data...
---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/context.py in execution_mode(mode)
   2101       ctx.executor = executor_new
-> 2102       yield
   2103     finally:

10 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/data/ops/iterator_ops.py in _next_internal(self)
    757             output_types=self._flat_output_types,
--> 758             output_shapes=self._flat_output_shapes)
    759

/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/gen_dataset_ops.py in iterator_get_next(iterator, output_types, output_shapes, name)
   2609     except _core._NotOkStatusException as e:
-> 2610       _ops.raise_from_not_ok_status(e, name)
   2611     except _core._FallbackException:

/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py in raise_from_not_ok_status(e, name)
   6842   # pylint: disable=protected-access
-> 6843   six.raise_from(core._status_to_exception(e.code, message), None)
   6844   # pylint: enable=protected-access

/usr/local/lib/python3.6/dist-packages/six.py in raise_from(value, from_value)

InvalidArgumentError: TypeError: Tensor is unhashable. Instead, use tensor.ref() as the key.
Traceback (most recent call last):

  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/script_ops.py", line 242, in __call__
    return func(device, token, args)

  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/script_ops.py", line 131, in __call__
    ret = self._func(*args)

  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/autograph/impl/api.py", line 302, in wrapper
    return func(*args, **kwargs)

  File "<ipython-input-79-e5206ab2321d>", line 5, in subsample
    if random_numbers[s_idx] >= item2prob[s]: # this is where the error occurs

  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py", line 823, in __hash__
    raise TypeError("Tensor is unhashable. "

TypeError: Tensor is unhashable. Instead, use tensor.ref() as the key.

	 [[{{node EagerPyFunc}}]] [Op:IteratorGetNext]

During handling of the above exception, another exception occurred:

InvalidArgumentError                      Traceback (most recent call last)
<ipython-input-79-e5206ab2321d> in <module>()
     16
     17 print('\nGenerating data...')
---> 18 for i in tf_gen:
     19     print(i)

/usr/local/lib/python3.6/dist-packages/tensorflow/python/data/ops/iterator_ops.py in __next__(self)
    734
    735   def __next__(self):  # For Python 3 compatibility
--> 736     return self.next()
    737
    738   def _next_internal(self):

/usr/local/lib/python3.6/dist-packages/tensorflow/python/data/ops/iterator_ops.py in next(self)
    770   def next(self):
    771     try:
--> 772       return self._next_internal()
    773     except errors.OutOfRangeError:
    774       raise StopIteration

/usr/local/lib/python3.6/dist-packages/tensorflow/python/data/ops/iterator_ops.py in _next_internal(self)
    762         return self._element_spec._from_compatible_tensor_list(ret)  # pylint: disable=protected-access
    763       except AttributeError:
--> 764         return structure.from_compatible_tensor_list(self._element_spec, ret)
    765
    766   @property

/usr/lib/python3.6/contextlib.py in __exit__(self, type, value, traceback)
     97                 value = type()
     98             try:
---> 99                 self.gen.throw(type, value, traceback)
    100             except StopIteration as exc:
    101                 # Suppress StopIteration *unless* it's the same exception that

/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/context.py in execution_mode(mode)
   2103     finally:
   2104       ctx.executor = executor_old
-> 2105       executor_new.wait()
   2106
   2107

/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/executor.py in wait(self)
     65   def wait(self):
     66     """Waits for ops dispatched in this executor to finish."""
---> 67     pywrap_tfe.TFE_ExecutorWaitForAllPendingNodes(self._handle)
     68
     69   def clear_error(self):

InvalidArgumentError: TypeError: Tensor is unhashable. Instead, use tensor.ref() as the key.
Traceback (most recent call last):

  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/script_ops.py", line 242, in __call__
    return func(device, token, args)

  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/script_ops.py", line 131, in __call__
    ret = self._func(*args)

  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/autograph/impl/api.py", line 302, in wrapper
    return func(*args, **kwargs)

  File "<ipython-input-79-e5206ab2321d>", line 5, in subsample
    if random_numbers[s_idx] >= item2prob[s]: # this is where the error occurs

  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py", line 823, in __hash__
    raise TypeError("Tensor is unhashable. "

TypeError: Tensor is unhashable. Instead, use tensor.ref() as the key.

	 [[{{node EagerPyFunc}}]]

@animesh0794 Are you working on recommender systems?

@bhack  yes I am.

@bhack  yes I am.

We have a new project in ecosystem:
https://www.tensorflow.org/recommenders
https://blog.tensorflow.org/2020/09/introducing-tensorflow-recommenders.html
https://www.tensorflow.org/recommenders/examples/quickstart

@bhack I am aware of it. But thanks anyway. I just wanted to try my own code first, before trying TFRS : )

Yes but you can still be indpired on how dataset are managed https://github.com/tensorflow/recommenders/blob/main/tensorflow_recommenders/examples/movielens.py

@bhack thanks, i would surely check that out.

Hi,
Thank you for opening this issue. Since this issue has been open for a long time, the code/debug information for this issue may not be relevant with the current state of the code base.
The Tensorflow team is constantly improving the framework by fixing bugs and adding new features. We suggest you try the latest TensorFlow version with the latest compatible hardware configuration which could potentially resolve the issue. If you are still facing the issue, please create a new GitHub issue with your latest findings, with all the debugging information which could help us investigate.
Please follow the release notes to stay up to date with the latest developments which are happening in the Tensorflow space.


Implement schema method required by session interface in client

Schema method was just added in Session interface of datastax php driver.

:+1:

:+1:


Update BF release jobs

This PR propagates recent changes applied to the CI jobs to the contributing page and should turn https://ci.openmicroscopy.org/view/Failing/job/CONTRIBUTING-merge-docs/ green.

following the bio-formats-documentation decoupling, the release jobs are simplified to one job which checks out the tag and the artifacts and builds the component
many of the CI jobs have been migrated towards containerized Jenkins instances and disabled/shelved from https://ci.openmicroscopy.org/ and are removed from the list.

As an addition, we could potentially expose the current https://web-proxy.openmicroscopy.org/west-ci/ and https://web-proxy.openmicroscopy.org/east-ci/ URLs in these pages as most of the Bio-Formats development happens in these devspaces.

Changes look good and updated pages display correctly. PR is good to merge


adding bulk lookup andbulk reverse lookup functions

This PR adds to the lookup interface bulk lookup functions with a basic implementation

@cheddar please checkout this

@cheddar and @fjy thanks, please checkout this

Overall looking pretty straight forward and sensible. Needs unit tests for unapply all.

@drcrallen i have added UTs

@fjy finally the build passed, can we get this in.

:+1:

Missing another +1 so if @cheddar and @drcrallen can do that we can merge this

:+1:  Merging.


Fix Win10SDK detection during Bootstrap

When multiple Win10 SDK's are installed, like Insider previews,
depending on the SDK installation, it may not contain the um headers.
There is already a loop for trying to find the newest Win10 SDK
that has full headers, but it's returning at the first failure,
rather than continuing the loop to the next iteration.
Also, use Join-Path instead of manual path appending, to normalize
the extra \ at the end of the sdk root value.  In some cases paths
with double backslashes can be interpreted as a fallback syntax for
UNC paths.  Join-Path normalizes the path appropriately.

Good catch! Thanks for the PR.


Remove obsolete release for page inc/dec

Patch for core issue #596 (and other related issues) regarding button graphics being incorrect after page changes.
The change in the toggle/latching button action logic for press/release should also fix the stack overflow issue.

@istnv I'm confused by this PR because only code comments are coming through.  There doesn't appear to be an any code changes here?

The comment block (/* */) has my notes and also wraps/disables the two redundant "bank_pressed" emits.

Let’s just blame my earlier comment on my not having finished my morning coffee at the time of reviewing this.

Ok, having read through this properly and your previous commit on device.js, this looks good to me.


SuperDeduper ignoring reads

Testing SuperDeduper on 26bp single reads it ignores all reads, despite high qualities and lack of Ns. Why is this?

Are you running with any parameters?
Maybe try:
-s 5 -l 20
To ask superdeduper to start at base 5 and consider a fragment 20bp long.
If that doesn't give you the behavior you want, could you provide the log file (e.g. -L dedup.log)?

Both with default start/length and 5/20.
Call$ hts_SuperDeduper -U DKR100_S42_L001_R1_001.fastq.gz -L hts_sd_5-20_test.stats.log -s 5 -l 20
DKR100_S42_L001_R1_001.fastq.gz
hts_sd_5-20_test.stats.log
hts_sd_test.stats.log

Hi @channsoden I have been able to replicate the behavior you reported, and I think I found the bug in SuperDeduper. Let me do a little bit of testing and get back to you.

Thanks, @samhunter. Curious what is going on here.

Hi again @channsoden. So it turns out this is actually not a "bug" it's a "feature".
When you work with PE data, we require two pieces, one from each read in order to form a unique key for the fragment: R1[start:length] + R2[start:length]
Our thought was that we should require the same level of evidence for a SE read duplicate, so we use:
R1[start:length*2]
If you use -s 1 -l 12 it should work. You will actually be filtering for duplicates using 24bp of the read as a key.
We will discuss whether this behavior is what we want to keep going forward (with better documentation), or change it somehow.

That makes sense. Thank you very much, @samhunter.
I think documenting the behavior and maybe an exception or warning for length overruns would suffice.


Better messaging when build stuff is missing and we crash on nil

Better messaging for #824

@KrauseFx yeah, I remember that discussion. I think it makes sense in the long run. I was worried some early adopters would get weird nil errors (like I did) and not understand what could be going on, so going right to the root and raising an exception that people can understand might be a good thing. That being said, it might be better to just see what happens. Closing 👍

Cool, thanks 👍


Announce GPS accuracy

@jchudge suggested having the app announce GPS accuracy at startup. This shouldn't require any calculations, just reporting the system's CLLocationAccuracy. But I wanted to open up the implementation details for discussion, specifically that the GPS accuracy can and does change as the user moves around. Should it be announced periodically, or perhaps if it exceeds some threshold?

I would suggest implementing it on opening the app and also when awakening it from 'sleeping'. The intent is to communicate the accuracy at the outset of a person's experience - knowing that they can go into settings to look it up if they need to at other times. During it's earlier incarnation, we implemented something similar that provided updates every time there was a change of about 5 metres or so...this resulted in a fairly constant barrage of notifications which were disruptive and not particularly helpful.

Ah, I hadn't realized this information was already exposed to the user, under the Troubleshooting section. To the implementers, the relevant code is here: https://github.com/soundscape-community/soundscape/blob/main/apps/ios/GuideDogs/Code/Visual UI/View Controllers/Settings/StatusTableViewController.swift#L112

To play a callout at startup and wake, we can locate the places where the appLaunch glyph is played, namely in these two places:
https://github.com/soundscape-community/soundscape/blob/main/apps/ios/GuideDogs/Code/App/AppContext.swift#L245
https://github.com/soundscape-community/soundscape/blob/main/apps/ios/GuideDogs/Code/Behaviors/EventProcessor.swift#L156
However, we can't simply read the accuracy value at that instant, because the GPS location hasn't been determined yet. We probably need to watch for the first LocationUpdatedEvent to trigger after resume.


Real cause of a deletion failed error is masked under a rate limit error

One ServiceBinding resource is in DeleteFailed state.
Its full .status  is the following:
status:
  bindingID: 7709b31e-2c00-4fe7-8b0e-ab32868bfb84
  conditions:
  - lastTransitionTime: "2024-06-29T00:59:25Z"
    message: 'ServiceBinding deletion failed: The allowed request limit of 20 requests
      has been reached please try again later'
    observedGeneration: 2
    reason: DeleteFailed
    status: "False"
    type: Succeeded
  - lastTransitionTime: "2024-08-28T00:00:14Z"
    message: ""
    reason: NotProvisioned
    status: "False"
    type: Ready
  - lastTransitionTime: "2024-10-24T07:51:02Z"
    message: 'ServiceBinding deletion failed: The allowed request limit of 20 requests
      has been reached please try again later'
    observedGeneration: 2
    reason: DeleteFailed
    status: "True"
    type: Failed
  observedGeneration: 2

Notice that from the three messages, is only a single message from today. The two other messages are obsolete.
The real reason behind the rate limit is not displayed at all. The real reason is deeply buried in the sap-btp-service-operator logs:
{"level":"error","ts":"2024-10-24T07:47:23Z","msg":"Reconciler error","controller":"servicebinding","controllerGroup":"services.cloud.sap.com","controllerKind":"ServiceBinding","ServiceBinding":{"name":"subscriber-tenant-2819b4a9-994f-4ad7-83c0-22d1607edd32-binding-64pvqs","namespace":"hana-poc-1"},"namespace":"hana-poc-1","name":"subscriber-tenant-2819b4a9-994f-4ad7-83c0-22d1607edd32-binding-64pvqs","reconcileID":"4cfc3fe5-17e0-4d9d-9976-99199f7aba41","error":"Async unbind operation failed, errors: {\"error\":\"BrokerError\",\"description\":\"failed polling operation for binding with id 7709b31e-2c00-4fe7-8b0e-ab32868bfb84 and name subscriber-tenant-2819b4a9-994f-4ad7-83c0-22d1607edd32-binding-64pvqs due to polling last operation error: Failed to drop service binding '7709b31e-2c00-4fe7-8b0e-ab32868bfb84' for instance '34ef927e-29f5-4ef4-8b85-d4e36285d90e' on database '5e481f2b-22a2-4430-825e-6d60f8b929bf' (5e481f2b-22a2-4430-825e-6d60f8b929bf.hana.prod-eu12.hanacloud.ondemand.com:443), because of: Failed to open connection to the database, because of: Failed to get connection for database '5e481f2b-22a2-4430-825e-6d60f8b929bf' (5e481f2b-22a2-4430-825e-6d60f8b929bf.hana.prod-eu12.hanacloud.ondemand.com:443), because of: SAP DBTech JDBC: [1890]: HANA Database instance is stopped\"}","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:324\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:265\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.15.0/pkg/internal/controller/controller.go:226"}

At the end of this long nested message there's the golden nugget:

 SAP DBTech JDBC: [1890]: HANA Database instance is stopped

The ServiceBinding should not only show the rate limit error. Preferably the two obsolete 2-month-old .status messages could be replaced with the rate limit error AND the actual error.

Hi @pre
We are looking into it.
Best Regards,
Naama

A fix was delivered in the latest release.
Regards,
Naama


[Bug] Icon detection affects other projects (window)

[x] I'm sure this issue is not a duplicate.

How to Reproduce

Create two projects. The first one is an empty project. It only contains ts config and files like src/middlewares/demo.middleware.ts. And the second one is a Nest.js project.
Only open the first project, the icon on .middleware.ts is a TS icon.
Then open the second project, due to the Nest.js detection, the .middleware.ts in first project changed to Nest.js icon.

More info: version 11.4.0

@KingDarBoja maybe related to #1897? The detection should be workspace scope.

Hi @dreamerblue, unfortunately, this is a limitation with vscode. We cannot configure the extension per project. So if you're opening two different projects, the latest will always "win" and impose its characteristics. It's not possible to make the extension work different for different vscode windows so we'll have to live with this for the time being.
You can always disable the auto detection feature and if you're constantly switching between nestjs and normal projects, you can do it manually or don't do it at all.


Place custom button next to pdf button for publications

No idea how to do that. If I use
url_pdf = "..."
url_cite = "..."
url_github = "..."
Only the pdf button appears in the publication page

Duplicate of #765 .
In the meantime, if all buttons are created as links and you avoid any of the automated features then you can choose the order. Obviously, some functionality will be lost using this workaround to meet your requirement.


drivers/net/ksz9477: Add simple port-based static VLAN configuration

Add a static port-based VLAN configuration for KSZ9477 switch. This doesn't use the VLAN tagging, but is a switch's internal mechanism to simply configure if the packet forwarding is allowed from one port to another.

re-based


AadHttpClient - User login is required

Category

[x] Question
[ ] Typo
[x] Bug
[ ] Additional article idea

Expected or Desired Behavior
After reading the articel https://docs.microsoft.com/de-de/sharepoint/dev/spfx/use-aad-tutorial I tried to implement the AadHttpClient to query the Azure AD. Everything works fine while development but now we try to set the solution to productive.
Observed Behavior
We get the following error at the browsers console window:

Error

columnNumber: 33832

fileName: "https://spoprod-a.akamaihd.net/files/sp-client-prod_2018-04-27.025/sp-pages-assembly_de-de_fe39afc9a02d6a7fc3dec634a0dd4931.js"

lineNumber: 592

message: "User login is required"
>
stack: "WEBPACK_AMD_DEFINE_RESULT</</ye</e.prototype.acquireTokenPromise/</<@https://spoprod-a.akamaihd.net/files/sp-client-prod_2018-04-27.025/sp-pages-assembly_de-de_fe39afc9a02d6a7fc3dec634a0dd4931.js:592:33832\n[6]/i</i.prototype.acquireToken@https://spoprod-a.akamaihd.net/files/sp-client-prod_2018-04-27.025/0.sp-http-adal_4a41848c866052caa755.js:3:11368\n__WEBPACK_AMD_DEFINE_RESULT_</</ye</e.prototype.acquireTokenPromise/<@https://spoprod-a.akamaihd.net/files/sp-client-prod_2018-04-27.025/sp-pages-assembly_de-de_fe39afc9a02d6a7fc3dec634a0dd4931.js:592:33400\n__WEBPACK_AMD_DEFINE_RESULT_</</ye</e.prototype.acquireTokenPromise@https://spoprod-a.akamaihd.net/files/sp-client-prod_2018-04-27.025/sp-pages-assembly_de-de_fe39afc9a02d6a7fc3dec634a0dd4931.js:592:33262\n__WEBPACK_AMD_DEFINE_RESULT_</</ye</e.prototype.getToken/<@https://spoprod-a.akamaihd.net/files/sp-client-prod_2018-04-27.025/sp-pages-assembly_de-de_fe39afc9a02d6a7fc3dec634a0dd4931.js:592:32853\n"`

Steps to Reproduce
We extracted the Azure AD related functions into a seperate TypeScript-Class like the following:

export default class AzureAdHelper{
constructor(context: WebPartContext){
this._aadClient = new AadHttpClient(context.serviceScope, "https://graph.microsoft.com");
}
private _aadClient: AadHttpClient;

The error occurs inside the following method while we trying to load users. The URL Parameter is filled with: https://graph.microsoft.com/beta/users?top=100&$filter=department eq 'BU-1' or Department eq 'BU-2'

private  _singleSearchCallWithAad(url:string):Promise{
return new Promise((resolve: (result: any) => void, reject: (error: any) => void) : void => {
this._aadClient
.get(url, AadHttpClient.configurations.v1)
.then(response => {
resolve(response.json());
})
.catch(error => {
console.error(error);
});
});
}`

Why we get the "User Login is required". I understood the AadHttpClient that it will handle the users Login process for me? Where is my mistake?

Have the same exact error message on my tenant. Only happens when the solution runs in production.
If we refresh the page a 2nd time, the login succeeds and the users are retrieved from Graph.
When debugging in workbench, the login succeeds automatically from the first page load. No error message.

Hi,
The following issue has been fixed. It should roll out shortly.
Thank you for you patience

We also get the User Login is required-error.
We use the AadHttpClient for calls to MS Office Graph, to our own REST API and a third party API. It's not clear, when we get this error, as sometimes it works and sometimes it doesn't.
Some additional information:

We are using SPFx 1.4.1.
We are using a classic page that we deploy by ourselves. On that page we place a modern/client web part. The exception is thrown in sp-classic-page-assembly.js.
I added the web part to a modern page and there it happens too, but the exception is thrown in sp-classic-page-assembly.js.
We can sometimes "fix" this error

by navigating to a modern page where our code is executed too (but only one API call is executed) and then navigation back to our classic page.
by clearing the browser cache, cookies, etc. and/or starting a new "private session"

As I said, we are calling different APIs using the AadHttpClient from within one SPFx web part and it happens with all the APIs. The User Login is required-error is only thrown for one call though.

From our understanding it looks like a bug in SP/SPFx as sometimes it works and sometimes it doesn't.
For us this is a very important functionality as we rely strongly on Azure-hosted APIs and cannot accept functionality that does not work in a reliable manor.
Do you have any information on

if and when this will be fixed?
any reliable workarounds?

In case you have further questions or need some more technical details, please contact me. Thanks!

@lahuey - Great. Just for my understanding: Where will this be fixed? In SPFx, i.e. will we need a new SPFx version? Or is this fixed in code that is part of SP itself?

@vocaris out of curiosity: why are you using the AadHttpClient to connect to the Microsoft Graph rather than the MSGraphClient?

Hi
I've found that if you have more than one pending request in the Preview SharePoint Admin centre > API Management - you may have more than one permission request for the same thing, try to approve them, if you get an error, reject them.  I saw this behaviour with the new PnP Starter Kit.
Paul

@waldekmastykarz while our first try's we got some other errors (propably our own mistakes) with the MSGraphClient... today I'm not remembering which errors but we were not able to handle them and so we decided to switch the the AadHttpClient which seems to wook correct...

@lahuey Is this now deployed? We have an issue since a little more than a week https://github.com/SharePoint/sp-dev-docs/issues/1934

It seams to be working but with the issue #1934 Nasicus explained. So we implement the Semaphore and everything is fine...

It should be working without the semaphore. Are you still seeing issues?
On Tue, Jun 12, 2018, 12:47 AM vocaris<EMAIL_ADDRESS>wrote:

It seams to be working but with the issue #1934
https://github.com/SharePoint/sp-dev-docs/issues/1934 Nasicus
explained. So we implement the Semaphore
https://github.com/abrkn/semaphore.js/blob/master/lib/semaphore.js and
everything is fine...
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/SharePoint/sp-dev-docs/issues/1810#issuecomment-396497927,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHBvHL1W9EDyubpnZ3gan1dKGVCAyF3-ks5t73HtgaJpZM4T2Ltu
.

@lahuey just tested it on my demo app & our real app and in both it's working again.
Thank you!


Update and DTOs

This is an experiment.
The lists should contain only identifiers.
Update DTOs of both steps and applied ingredients should be treated separately.

Done. DTOs are updated.


Make dataset + column creation idempotent to avoid punishing "deploy first" workflows

Problem Statement

I typically run my services in an environment, and begin collecting traces, before I have defined any boards or triggers in my Terraform code.
Honeycomb creates new datasets and columns on the fly
If I deploy to some environments but not others, the Terraform code that I write to create boards and triggers may become valid to run in some environments, but not others, because it refers to columns or datasets that do not exist
If, instead,  I try to proactively create columns and datasets using Terraform, then that code be invalid in any environment where I have deployed

I'm forced to literally ship every change to production first, and then create a separate pull request later on that adds observability. This is a waste of my time and creates an observability gap.
Proposed Solution
For boards and columns, the state is trivial and I would argue that this provider should break ranks with Terraform and provide idempotent/upsert semantics. In other words: if the board or column already exists and if it is identical to the resource specification, the provider should simply pretend that create succeeded vs. erroring because the resource already exists.
Workarounds
One can switch to "observe first" i.e. decide on column and dataset names, ship a Terraform change to all environments, and then work on the service in question. That's alienating however.

Thanks for taking the time to open this issue, @xeger.
The idea of doing what are effectively "silent imports" is felt to be surprising behavior: especially when considering that the intention is to manage the lifecycle of a resource.
We have removed the spots where this use to happen in this code base for that reason, and view "error on conflict" as correct behavior in the Terraform ecosystem.
That all said, you are highlighting a rough edge that is top of mind (and felt by us as well) and currently being discussed internally to find a more holistic, product-wide solution.
I'll keep this issue open and report on progress periodically.

terraform can manage anything, but it's not always the best option for anything

Hard agree: I mentioned the other day in a discussion about this that it feels like Terraform has become the hammer (the the famous adage about everything looking like a nail) that many reach for because we don't really offer alternatives currently.
We are making progress with an OpenAPI specification and moving toward the shiny future of autogenerated client SDKs, which I'm hopeful will unlock (read: reduce maintenance burden in addition to delighting folks) new tools and support for additional ecosystems.


Optimise application bundle size

Optimised application bundle size by using nginx image in the dockerfile and by adding nginx configuration.

LGTM.

LGTM. Good job @Msarawan

Thanks!


change entity from Interface to Abstract

Hi
I understand that I need to use abstract to be able to use @Transient.
I changed one of my Entities to Abstract and the rest are still interfaces.
Since them I am not able to compile it anymore becuase the *Entity classes are not generated anymore.
Trying to understand why and I will be really happy to get some help.
Here is my class..
@Entity
public abstract class AbstractNodeKey {

    @Index("indexKey")
    int keyNumber;

    @Index("indexKeyName")
    String Name;

    @ManyToOne
    @Index("indexKey")
    Node Node;

    ENodeKeyType KeyType;

    int Category;

    @ManyToOne
    Room Location;

    @Transient
    KeyStatusData KeyStatusData;
}

Thanks a lot for any help

Hi whenever you make a change that prevents generation from occurring it's highlighted in the gradle message window like so https://github.com/requery/requery/issues/169#issuecomment-223828475. What's the error you are encountering? Posting the gradle build output should also show this.

Since this issue is similar to #169 closing for that. If you have more info please update thanks


Constituting Config.fromenv could also source service_name from env

As a lib User
I want to be able to set the AWS service_name as an envvar
So I don't always have to explicitly set the value

(see here)

this was actually handled in #15.


Bindgen layout tests fail on i686 with leptonica 1.84

Hi. Tests for leptonica-sys fail with leptonica 1.84 on i686 due to a known bug reported to bindgen, see https://github.com/rust-lang/rust-bindgen/issues/1823. Apparently, starting from v1.84, the leptonica headers contain a definition of max_align_t which has incompatible definitions in C and clang on i686, and thus makes the relative test fail with the error
---- bindgen_test_layout_max_align_t stdout ----
thread 'bindgen_test_layout_max_align_t' panicked at 'assertion failed: `(left == right)`
  left: `16`,
 right: `24`: Size of: max_align_t'

For the tests I used libleptonica-dev from Debian testing. The bindgen maintainers suggest either making max_align_t opaque or blacklisting it from the bindings, depending on whether max_align_t is actually used or not in the bindings.
I understand the last version of leptonica supported by this library should be 1.83. However, especially since starting from bindgen v0.70 layout tests became compile-time tests (so this bug would make compiling the library fail), I thought you may want to have a look at this nonetheless.

A possible fix/workaround is to blocklist the type, it doesn't appear to be actually used by leptonica, just picked up from a system header.
1002_blocklist_max_align_t.patch.txt

Thank you both
I've accepted the patch to blocklist the type. It is also what I would have done. I've published as 0.4.9.


docs(toh-5): fix HeroesDetailComponent typo

Looks like this was supposed to refer to the HeroDetailComponent instead (there is no HeroesDetailComponent)

I signed it!

Good catch!


Corrected method names in the getting started notebook

The convenience functions were failing because the method names had been updated in commit 38a8672
which addressed ISSUE-117 (Add list datasets by processing level to podaac_utils) but changed the name of these methods.

Thank you for this @noah-de , very much appreciated.

Coverage remained the same at 96.27% when pulling a922637649901a357b997d303998dfadbc4aae5c on noah-de:master into 318a894a36dff6b697c023540bb119628e0a06fa on nasa:master.


Update dependencies

Update dependencies to date.
Breaking changes of interfaces in TensorRT 8 and FFmpeg 5 were fixed.
Utilized improvements of CMake files of folly and yaml-cpp.

Add a commit to improve image reading speed.


[CT-1628] Update to work with consolidated Parsed and Compiled node classes

In dbt-core/#6384 we are consolidating the Parsed and Compiled node classes as part of the cleanup for creating standard protobuf artifacts for nodes. We need to rename the files and classes for a number of objects.

Resolved by https://github.com/dbt-labs/dbt-rpc/pull/103


record previous create org operation info when return to error page

Fixes: #22833
Put user's previous operation params into context, for the page can get these params when re-rendering

Codecov Report

Merging #22834 (330b1d8) into main (90cf07a) will increase coverage by 47.33%.
The diff coverage is n/a.

:exclamation: Current head 330b1d8 differs from pull request most recent head a20c753. Consider uploading reports for the commit a20c753 to get more accurate results

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@            Coverage Diff            @@
##           main   #22834       +/-   ##
=========================================
+ Coverage      0   47.33%   +47.33%
=========================================
  Files         0     1113     +1113
  Lines         0   150079   +150079
=========================================
+ Hits          0    71034    +71034
- Misses        0    70618    +70618
- Partials      0     8427     +8427

Impacted Files
Coverage Δ

modules/git/commit.go
60.24% <0.00%> (ø)

cmd/keys.go
65.38% <0.00%> (ø)

routers/api/v1/utils/page.go
100.00% <0.00%> (ø)

models/user/setting.go
55.90% <0.00%> (ø)

modules/indexer/stats/db.go
56.25% <0.00%> (ø)

models/repo/language_stats.go
56.17% <0.00%> (ø)

services/webhook/feishu.go
86.95% <0.00%> (ø)

services/cron/setting.go
69.56% <0.00%> (ø)

modules/setting/mime_type_map.go
45.45% <0.00%> (ø)

routers/api/v1/api.go
74.94% <0.00%> (ø)

... and 1103 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

hmm...

is .visibility always set? I have a feeling not and if so this is likely to cause an NPE unless gotemplate have got round to changing the or to short-circuit (which as far as I know was still only in discussion)
If .visibility is not always set we should use its presence or not to determine whether to use the last submitted form value or not.
i.e.

<input class="hidden enable-system-radio" tabindex="0" name="visibility" type="radio" value="0" {{if .visibility}}{{if  .visibility.IsPublic}}checked{{end}}{{else if .DefaultOrgVisibilityMode.IsPublic}}checked{{end}}/>

Or better just change the rendering code to always set ctx.Data["visibility"] with the correct default environment variables and simplify to:

<input class="hidden enable-system-radio" tabindex="0" name="visibility" type="radio" value="0" {{if  .visibility.IsPublic}}checked{{end}}/>


tooltips: Add period (.) to last edited/moved notice.

This PR adds a period .  after the last edited/moved notice which appears in the tooltips.
See https://github.com/zulip/zulip/issues/26581#issuecomment-1742395614

Screenshots and screen captures:

Self-review checklist

[x] Self-reviewed the changes for clarity and maintainability
(variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

[x] Explains differences from previous plans (e.g., issue description).
[x] Highlights technical choices and bugs encountered.
[ ] Calls out remaining decisions and concerns.
[ ] Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

[x] Each commit is a coherent idea.
[x] Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

[x] Visual appearance of the changes.
[ ] Responsiveness and internationalization.
[x] Strings and tooltips.
[ ] End-to-end functionality of buttons, interactions and flows.
[ ] Corner cases, error conditions, and easily imagined bugs.

Merged, thanks @sayamsamal!


AWS missing groupID

I got an issue setting up scrapoxy on my aws/ec2
starting like recommended:
sudo docker run -e COMMANDER_PASSWORD='MY_PASSWORD' \
    -e PROVIDERS_AWSEC2_ACCESSKEYID='MY_KEY' \
     -e PROVIDERS_AWSEC2_SECRETACCESSKEY='MY_SECRET_KEY' \
     -it -p 7777:8888 -p 8889:8889 fabienvauchelles/scrapoxy

running into this issue:
2017-07-09T14:33:59.747Z - info: [Main] The selected provider is awsec2
2017-07-09T14:33:59.764Z - debug: [Main] listen
2017-07-09T14:33:59.770Z - info: [Commander] GUI is available at http://localhost:8889
2017-07-09T14:33:59.772Z - debug: [Manager] start
2017-07-09T14:33:59.773Z - info: Proxy is listening at http://localhost:8888
2017-07-09T14:34:09.774Z - debug: [Manager] checkInstances
2017-07-09T14:34:10.162Z - debug: [Manager] adjustInstances: required:1 / actual:0
2017-07-09T14:34:10.163Z - debug: [Manager] adjustInstances: add 1 instances
2017-07-09T14:34:10.163Z - debug: [ProviderAWSEC2] createInstances: count=1
2017-07-09T14:34:10.446Z - debug: [ProviderAWSEC2] createInstances: actualCount=0
2017-07-09T14:34:11.038Z - error: [Manager] Error: Cannot update or adjust instances: InvalidParameterValue: Value () for parameter groupId is invalid. The value cannot be empty
    at Request.extractError (/usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/services/ec2.js:50:35)
    at Request.callListeners (/usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
    at Request.emit (/usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/request.js:615:14)
    at Request.transition (/usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/request.js:617:12)
    at Request.callListeners (/usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/sequential_executor.js:115:18)
    at Request.emit (/usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/request.js:615:14)
    at Request.transition (/usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/usr/lib/node_modules/scrapoxy/node_modules/aws-sdk/lib/request.js:38:9)

Did I missconfigure the command?
Looks like aws denies request for missing security groupId

+1

+1

Guys,  I set up the project in the N. Virginia region because I could not access the eu west 1 region recommended  in the quickstart. I think I'm having the same problem, I set it up using the docker command, but no instances are starting. Did you set it up in the  eu west 1 region or somewhere else?

Hello kc1,
Did you try to copy the AMI as recommended in the documentation? (http://docs.scrapoxy.io/en/master/standard/providers/awsec2/copy_ami_to_region/index.html)
AWS doesn't share AMI between region. I provide AMI for some regions but if it doesn't exist, you have to copy one yourself.
Fabien

Hello benpetsch,
2 questions :
1/ do you use the eu-west1 region like in the documentation?
2/ do you have created security group and AMI in the same region?
Fabien

Hi Fabien,
First, thank you for making this project available . Even though I don't
have it working yet , but from what I can see it looks fantastic!
In answer to your question, I set the project up in N. Virginia, because I
could not get access to eu-west1region. I have a request into amazon to
allow me to access to eu-west1. Hopefully they will allow it.
If they don't, is there a way to deploy scrapoxy to another ec2 region
directly?
Thanks again,
KC
On Sat, Jul 15, 2017 at 1:39 PM, Fabien Vauchelles<EMAIL_ADDRESS>
wrote:

Hello benpetsch,
2 questions :
1/ do you use the eu-west1 region like in the documentation?
2/ do you have created security group and AMI in the same region?
Fabien
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/fabienvauchelles/scrapoxy/issues/70#issuecomment-315550126,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABqjLn99hJLdCEJ7pr5xcVQzPyyvr68hks5sOPlugaJpZM4OSFKh
.

Hello,
Of course you can do that ;p You have to copy the ami into your region. Check the link below, it will help you to do that.
Fabien

Hi Fabien, I was able to get it set up using docker in the  eu-west1 region. Thank you.

Nice!

Hi @fabienvauchelles. Met the same issue, seems there was a toubles with security groups, we define in config any group name, but aws trying to assign default security group and vpc subnet. So, for me helps next config:
            "instance": {
                "InstanceType": "t2.nano",
                "ImageId": "ami-06220275",
                "SecurityGroupIds": [
                    "sg-xxxxxxxxxxx"
                ],
                "SubnetId": "subnet-xxxxxxxx"
            }

Defined manually security group and vpc subnet ids, ref https://github.com/aws/aws-sdk-js/blob/b8207ee91ae9d833b714db872e2a8d3923f4ab92/apis/ec2-2016-11-15.min.json#L8454
But i met another problem:
2017-08-23T10:38:28.057Z - debug: [Instance/i-04abbf60fbefa3b05] checkAlive: false / -
/usr/local/lib/node_modules/scrapoxy/server/common/pinger/index.js:19
        throw new Error('[ping] should have hostname and port');
        ^

Error: [ping] should have hostname and port
    at Object.ping (/usr/local/lib/node_modules/scrapoxy/server/common/pinger/index.js:19:15)
    at Timeout.self._checkAliveTimeout.setInterval [as _onTimeout] (/usr/local/lib/node_modules/scrapoxy/server/proxies/manager/instance.js:39:28)
    at ontimeout (timers.js:386:14)
    at tryOnTimeout (timers.js:250:5)
    at Timer.listOnTimeout (timers.js:214:5)

So, you are also need to seet this checkbox fot your VPC's subnet


Minor update to Readme

The example output incorrectly displayed the key as difference instead of percentage

Closing, as this pull request is about a year old against a deprecated repository. 😆


Latest CUDA 7.5

It seems like the latest version of CUDA 7.5 (352.63), which fixes a critical bug on ec2, is only available .deb installer but not in .run installers. Hence the dockerfile in this repo will not install the latest version of CUDA 7.5.
I'm not sure if this is an "issue" that should be fixed but I think it's worth pointing out in case someone elses run into the same situation.

Thanks for reporting - I'll leave this issue open for now. I've made the decision to stick with the latest .run installer to at least have a consistent driver to specify, but once https://github.com/NVIDIA/nvidia-docker/issues/7 is finished I'll be deprecating kaixhin/cuda and using different drivers won't be an issue.


allow Override annotation to be a function in http platform

What is the problem this feature would solve?
when I annotate the http platform api with override, that can be then used when rendering the openapi,
i can only add override, which will merge the two objects.
export const api = HttpApi.empty
  .add(restApi)
  .annotate(Override, { components: { schemas: "def" }})

=> all schemas are replaced with "def"
What is the feature you are proposing to solve the problem?
What I'd like to instead have is be able to supply a function that would take the generated specificiation and would return a modificated version of that. In this way, I could for example add an element to the schemas:
const MySchema = ...
export const api = HttpApi.empty
  .add(restApi)
  .annotate(Override, (orig) => ({ ...orig, components: { ...orig.components,
      schemas: { ...orig.components.schemas, MySchema } } } ))

What alternatives have you considered?
alternative would be to copy the HttpApiSwagger.layer function and apply some modification after the OpenApi.fromApi(api) call

looks good, thanks! this also could be used on group and endpoint level the same way Override is used

It can now be used in all places Override can be used :)


Bug fix/recalculate count

Scope & Purpose
Fixed a bug in the internal recalculateCount() method that could lead to count adjustments being applied with the wrong sign.
Added tests.

[x] Bug-Fix for devel-branch (i.e. no need for backports?)
[ ] Bug-Fix for a released version (did you remember to port this to all relevant release branches?)
[x] The behavior in this PR can be (and was) manually tested (support / qa / customers can test it)
[x] The behavior change can be verified via automatic tests

Testing & Verification
This PR adds tests that were used to verify all changes:

[x] Added new integration tests (i.e. in scripts/unittest recovery)

http://<IP_ADDRESS>:8080/view/PR/job/arangodb-matrix-pr/10246/

http://<IP_ADDRESS>:8080/view/PR/job/arangodb-matrix-pr/10249/

Tests blue


Update CI URLs to point to new caching infrastructure

Hello there!
This is an automated pull request submitted by @staticfloat to help package authors transition their Julia installation CI setups to the new binary provider URL.  Please take a look at this PR, and if there is a problem or it doesn't download correctly when CI runs, feel free to ping @staticfloat

Coverage remained the same at 83.378% when pulling b0435420073fd8c8b2fddd7c1fb72df62c6bfc98 on staticfloat:updated_ci_url into 96bc072619d27022b4118e99b891172ccf6274e0 on mpastell:master.


Ali : Asynchronous Programming: 3 weeks

Asynchronous Programming
My forked repo
Stop watch repo
Learning Objectives
Learning objectives

[x] 🥚 You understand the JavaScript Event Loop, and can demonstrate this by using setTimeout and setInterval to schedule simple tasks.
[x] 🥚 You can explain why Asynchronous Programming is important for programs that have blocking and non-blocking tasks.
[ ] 🥚 You can explain the basics of the Client/Server model and HTTP requests and can fetch data from RESTful APIs.
🐣 You can break down an asynchronous problem into smaller tasks and solve it using promises. This includes identify which tasks depend on each other and which are independent:

[x] dependent tasks_: The return value from one task is required to start the next task, these must be completed in a specific order - .then
[ ]  Independent tasks_: These tasks do not use each other's return values, they can be completed at the same time - Promise.allsystem.

[x] 🐣 You can fetch data from an API and render it into the DOM using /api-calls, /handlers and async/await syntax.
[ ] 🐣 You can write unit tests for functions that return promises using async/await syntax.

Week 1
I Need Help With:
nothing
What went well?
1- the core of Asynchronous Programming ⭐  ⭐ ⭐ the event loop ⭐  ⭐  ⭐
2-setTimeout() ... how to us it ...and 1000 millisecond is not always a good choice
What went less well?
setInterval() ..... i need to test it how its working in the project stopwatch
promises ...need to work with it ...
Lessons Learned
setTimeout()
Asynchronous Programming  how its works  in the background

@Ibrahim86Ali You can always still work on the assignment to understand it better.

@Ibrahim86Ali You can always still work on the assignment to understand it better.

@bermarte  hi bermarte i hope you  are doing  good ......thanks for following up  ....


False warning about omitting type

Consider the following:
var rlast io.ReadSeeker = bytes.NewReader([]byte("foo"))

Currently, I get the warning:
main.go:9:12: should omit type io.ReadSeeker from declaration of var rlast; it will be inferred from the right-hand side

I believe this warning should only be shown if the type on the left is exactly the same as the type on the right. In this case, the left is an io.ReadSeeker, while the right side is an bytes.Reader.

Fixed in 48c8655188557fb1ac819edd02d3694237eff1ad


Bug in convertFromHTML (empty lines are missing)

convertFromHTML works wrong. Let's see examples.
https://jsfiddle.net/m6z0xn4r/176/
There we see 3 different blocks instead of 4.  And each block has 1 test row and 1 blank row
https://jsfiddle.net/m6z0xn4r/177/
There we see good pasting text, but third line (empty) is missed.
Actualy, I expect:
The first line                //data-offset-key="b7tn6-0-0"
The second line          //data-offset-key="2oula-0-0"
empty line              //data-offset-key="avbam-0-0"
The fourth line           //data-offset-key="3l8as-0-0"
in v0.9 it worked fine
I use DraftJS 0.10 + Webpack 2.0

I ran into this (or very similar) issue this week also. Though, I could replicate the same problem in v0.9.1 so I figured it was intentional somehow. I've setup a similar test with a range of HTML block elements that I'd expect to get converted to ContentBlocks. Sadly only blocks with text get converted.
Here is my Stack Overflow question on it.
Here is my ver 0.10.0 jsfiddle
Here is my ver 0.9.1 jsfiddle

Same here (missing empty lines) when converting from previously saved as html content to draft-js content. DraftJS 0.10 + Webpack 2.0

I resolved many problems by using https://github.com/sstur/draft-js-import-html instead

It looks like draft-js-import-html might get the job done. I might have to manipulate my HTML a bit but at least it supports empty blocks. Also, I'll have to ignore some console warnings until they implement DraftJs 0.10 support but it could be worse.
Thanks for the tip @NoZiL.

Works here too on first try. Thank you @NoZiL

Update: turns out that draft-js-import-html doesn't import inline styles from html spans if existing. we ended up going with draft-convert which offers both options when using flat:true flag.
const contentState = convertFromHTML({
        // custom plugins for html restoration
        htmlToStyle: (nodeName, node, currentStyle) => {
            if (nodeName === 'span' && node.style.color) {
                // restore custom color values
                // must be present in the editorStyleMap
                const color = replaceAll(node.style.color, " ", "");
                return currentStyle.add(`COL_${color}`);
            } else {
                return currentStyle;
            }
        },
        htmlToEntity: (nodeName, node) => {
            // restore links
            if (nodeName === 'a') {
                return Entity.create(
                    'LINK',
                    'MUTABLE',
                    { url: node.href }
                )
            }
        }
    })(htmlString, { flat: true });

i get this error :   :  'convertFromHTML' is not defined  no-undef
convertFromHTML

This one worked for my issues https://github.com/jpuri/html-to-draftjs
Was able to read multiple new lines and text-align perfectly

If anyone is still looking, I also ended up using draft-convert,  I also had to preserve links as well (can preserve more stuff using the decordator)...and my code was as follows:
const Link = (props) => {
    const {url} = props.contentState.getEntity(props.entityKey).getData();
    return (
        <a href={url} target="_blank">
            {props.children}
        </a>
    );
};

const  findLinkEntities = (contentBlock, callback, contentState) => {
    contentBlock.findEntityRanges(
        (character) => {
            const entityKey = character.getEntity();
            return (
                entityKey != null && contentState.getEntity(entityKey).getType() === 'LINK'
            );
        },
        callback
    );
};

const decorator = new CompositeDecorator([
    {
        strategy: findLinkEntities,
        component: Link,
    }
]);
const html = '...whatever...html';

const blocksFromHTML = convertFromHTML({
            htmlToEntity: (nodeName, node, createEntity) => {
                if (nodeName === 'a') {
                    return createEntity('LINK', 'MUTABLE', { url: node.href });
                }
            }
        })(html);

const contentState = ContentState.createFromBlockArray(
            blocksFromHTML.getBlocksAsArray(),
            blocksFromHTML.getEntityMap
        );

editorState = EditorState.createWithContent(contentState, decorator);
...
...
<Editor
                    editorState={editorState}
                    ...etc
                />


[QUESTION] Troubles fetching tweets with a keyword

Context
I'm trying to fetch a list of tweets with contain a keyword, so I specify the keyword using the -s command. Although I can get a list of tweets, not all the time will each tweet contain the specified keyword. So I'm wondering if I'm incorrectly specifying a keyword to search for.
Command Ran
twint --search "jellybeans" --since 2019-10-22 --until 2019-10-24 --limit 20
Description of Issue
Although tweets are fetched without any issue, the tweets themselves don't contain the keyword.
Environment Details

Using Windows, Linux? What OS version? Running this in Anaconda? Jupyter Notebook? Terminal?

Ubuntu 18.04.3 LTS and MacOS Mojave 10.14.16
Twint 2.1.6
Python 3.7.3
Thank you, please let me know if you would like more information.

https://twitter.com/search?f=tweets&q=jellybeans since%3A2019-10-22 until%3A2019-10-24
scroll down a bit and check for matches in the output from where you see "Oct 23", you'll confirm to me (I guess, I hope) that Twitter matches for that word even in the username and the handle of the user which sent the tweet, plus it will check in the "reply to user" handle

Also if you "force" jellybeans to be interpreted as keyword (-s ""jellybeans"") you'll see the same behaviur

So as of now I'm unable to find a solution to search only for tweets containing a given keyword. I suggest you to do some "post-processing" to filter-out false positives

@pielco11 Ahh, thank you for pointing this out. I wasn't aware how the keyword search were grabbing tweets.


Fixed Scraper Content and Injection

Google had changed the class names for email author and the email body which broke the scraper.
Also changed when the scraper is injected so it would work whenever an email was open, no matter the inbox. Previously, the extension frontent would not load when the page was refreshed on an open email.

Great work. Thank you!

Great work. Thank you!


add KF sample_type values

add new sample_type values to support display of samples in KF portal

Thank you Eric! as discussed we are going to continue to investigate PMBC and Skin Tape to ensure they are not duplicates of Periph. blood mononuclear Cells and Skin Tissue.  These will not cause conflicts in either portal at this time so we are OK to proceed as is but we definitely should confirm and remove one if they are truly duplicates. We will check with @lopierra


Fix font on docs [temporary fix]

For some reason, the redprl account on cs.cmu.edu is blocking CORS requests, even though I have set .htaccess properly; however, it works fine with my personal account. Until we get that figured out, I'm reverting the site to point to jmsterli. Hopefully we can get the CORS issue sorted out after we have a bit more time.

I don't have time now, but someone should dump the HTTP requests and responses to see whether the headers are there.

I got these errors in the console
Loading failed for the <script> with source “https://www.google-analytics.com/analytics.js”.
atomic-judgment.html:1

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://media.readthedocs.org/fonts/fontawesome-webfont.woff2?v=4.6.3. (Reason: CORS header ‘Access-Control-Allow-Origin’ missing).

Use of getPreventDefault() is deprecated.  Use defaultPrevented instead.
jquery-2.0.3.min.js:3:21579

Source map error: request failed with status 404
Resource URL: https://media.readthedocs.org/css/sphinx_rtd_theme.css
Source Map URL: theme.css.map

@favonia That is just a problem with a font that is loaded by the "readthedocs" template, and I don't think it's for us to fix on our end. I think it's just for some wingdings or something...

Well, in any case, the web font does not seem to work for me...?

@favonia Wow, weird... it works for me?? To clarify, you are not seeing the PragmataPro font when you load http://www.redprl.org/en/docs/ (this is the version built from this branch)?
Note that http://www.redprl.org/ should currently be broken, and also if you try to run sphinx locally, the fonts will not work either.

I think the main reason is that https://www.cs.cmu.edu/~jmsterli/fonts/PragmataPro.css gives me 403.

Oh, I see. I spoof all the referrer and potentially tracking headers. Glad that you are the main one who do the demo.

I found the bug---somehow you are using http for ~redprl.

Sorry that's not the reason. Somehow the Origin is missing.

I think maybe the AFS permission is the reason...


Clearer documentation on Streaming.Zip with regards to newlines

When using gunzip on a bytestring from a compressed csv file with 5 columns, I was unable to parse a line in the file as it appeared to break before the newline, e.g. one,two,three,four,five was cut to one,two,three. Is this due to chunks? If so, the documentation should be clearer about how the data is streamed post-decompression and how to handle them (for instance, how to have the stream be by lines not chunks) or provide some utility functions or examples somewhere.

I think this is a bug, as a simple runResourceT $ BS.putStrLn . S.gunzip . BS.readFile $ mf does not return the proper value for some files (comand line gunzip works fine for them).

Seems to be https://github.com/fpco/streaming-commons/issues/37, https://github.com/fpco/streaming-commons/issues/28, and https://github.com/snoyberg/conduit/issues/254, may require some alterations.

Fixed https://github.com/haskell-streaming/streaming-utils/pull/7.


Set up pre-commit.ci

Problem
Closes #1921
Solution

Setup pre-commit.ci

Acceptance Criteria

Pre Commit runs on PR

Checks completed successfully:

coverage: 100.0%. remained the same
when pulling 930b7ad4fa80d1fc253afaab1a78413b70b0ac34 on matthewhegarty:issue-1921-pre-commit-ci
into 1f5a8139ed81f8a734c44f9448cfa5510c5c8970 on django-import-export:main.


File injection test using fakekubeclient

This change is

I'm not fully sure how you would add that. Are you suggesting to add to struct in https://github.com/Mirantis/virtlet/blob/ffad578784807d1773335b7729e23c1b4477fd69/pkg/libvirttools/virtualization_test.go#L343 additional field for parameters needed by fake kube client as in https://github.com/Mirantis/virtlet/pull/861/files#diff-1fd8fde46fcffd8abd9ee1c4e043e6beR296 and basically rest of the lines from this test case before https://github.com/Mirantis/virtlet/blob/ffad578784807d1773335b7729e23c1b4477fd69/pkg/libvirttools/virtualization_test.go#L476 line for testcases with non nil values for fake kube client parameters?
That would do the job, but would imo unnecessary enlarge already long body of https://github.com/Mirantis/virtlet/blob/ffad578784807d1773335b7729e23c1b4477fd69/pkg/libvirttools/virtualization_test.go#L468 loop.
Hmmm... I can refactor that a bit to make it more readable adding also handling this extension to tc struct.

PR updated according to suggestion.

In fact it does. Ok, removed.


newer posts lost after editing older post?

I seem to have lost my newest two posts, when I went back and edited the 3rd-oldest post.
Here's a commit publishing a post a week ago. Then two days ago I edited a post from a month ago and some other posts. When I then updated the config file, the lists of posts it re-published didn't include that post from a week ago (the commits after the update don't show that post).
Since then I've tried republishing all entries by modifying my settings (changing the theme) a couple times, but the newest two posts are still missing.
Any ideas? Thanks.
(Also: are there supposed to be so many identical-looking "Publish page-1 tag" commits? e.g. one, two, three, four...)

Do you still see this behaviour happening now on 0.6.x
Sorry For not triaging his bug earlier.

Closing out due to lack of response from OR.

I haven't noticed this happening lately. Thanks!


Incorporate PDC aliquots, aliquot_run_metadata_ids, study ids, and embargo date

For PDC, we noticed that aliquots aren't currently incorporated. Additionally, we need aliquot_run_metadata_ids in order to create some of our derived tables (e.g. protein abundance log2 ratio). We also need the pdc study ids and embargo date.

It looks like we do not currently extract PDC - aliquots. Must alter pdc_query_lib.py to do so.


Arrow 10623: [CI][R] Version 1.0.1 breaks data.frame attributes when reading file written by 2.0.0

@github-actions crossbow submit test-r-version-compatibility

@github-actions crossbow submit test-r-version-compatibility

@github-actions crossbow submit test-r-version-compatibility

@github-actions crossbow submit test-r-version-compatibility

@github-actions crossbow submit test-r-version-compatibility

@github-actions crossbow submit test-r-version-compatibility

@github-actions crossbow submit test-r-version-compatibility

@github-actions crossbow submit test-r-version-compatibility

@github-actions crossbow submit test-r-version-compatibility

@github-actions crossbow submit test-r-version-compatibility

@github-actions crossbow submit test-r-version-compatibility

@github-actions crossbow submit test-r-version-compatibility

@github-actions crossbow submit test-r-version-compatibility

@github-actions crossbow submit test-r-version-compatibility

@github-actions crossbow submit test-r-version-compatibility

@github-actions crossbow submit test-r-version-compatibility

@github-actions crossbow submit test-r-version-compatibility

@github-actions crossbow submit test-r-version-compatibility

@github-actions crossbow submit test-r-version-compatibility

@github-actions crossbow submit test-r-version-compatibility

And fork CI passing:
https://github.com/jonkeane/arrow/actions/runs/472646827
https://github.com/jonkeane/arrow/actions/runs/472646829

And fork CI passing:
https://github.com/jonkeane/arrow/actions/runs/472646827
https://github.com/jonkeane/arrow/actions/runs/472646829

@github-actions crossbow submit test-r-version-compatibility

@github-actions crossbow submit test-r-version-compatibility

I'll consolidate those tests some more like you mentioned. The only thing left to do (if we want to) would be to add backwards compatibility tests for stream-to-filesystem. I can add them if they add substantially more coverage than we have with feather files themselves.

 The only thing left to do (if we want to) would be to add backwards compatibility tests for stream-to-filesystem. I can add them if they add substantially more coverage than we have with feather files themselves.


Do not use a npm prepare script #48

@ryaa Could you please release a new version containing this change?

@kamilchlebek done. please see the details in CHANGELOG

That was quick! Thank you a lot 🚀


[mgmt][core] upgrade min dependency for azure-core

https://github.com/Azure/azure-sdk-for-python/issues/24221

/check-enforcer override


Feature Request - Add exclusion of specific files within backup source path

Currently the script is only able to exclude paths from the backup.
It would be even more useful if a set of specific files could be excluded from the backup as well.

I am not really maintaining this anymore. Maybe one of the Forks has an updated version


Testing Model Crate Part 2

Improving test coverage of signature, and trade.
Note that this PR also

removed the #[ignore] tag on the order debug test as our report was considering this untested code.
removes #[derive(Hash)] from Trade as it was only every used in a single unit test.
removes #[derive(Deserialize)] from SigningScheme since it wasn't used either.

Test Plan
Only new tests introduced.

Codecov Report

Merging #1110 (ab8356c) into main (3b25d9d) will increase coverage by 0.50%.
The diff coverage is 97.05%.

:exclamation: Current head ab8356c differs from pull request most recent head c75f050. Consider uploading reports for the commit c75f050 to get more accurate results

@@            Coverage Diff             @@
##             main    #1110      +/-   ##
==========================================
+ Coverage   65.57%   66.08%   +0.50%
==========================================
  Files         126      126
  Lines       25296    25394      +98
==========================================
+ Hits        16589    16782     +193
+ Misses       8707     8612      -95


fix uri bug

Test plan

[ ] Open chat, ask a question
[ ] Observe error
[ ] Check out this branch
[ ] Ask a question, observe error fixed

I'll pick this up, @beyang, and get it merged.

this got rolled into https://github.com/sourcegraph/cody/pull/2848

Wait nvm, there's additional changes in this PR

Yeah, the problem is that the fileName is relative not absolute, so vscode.Uri.file would not work. The context would be pulled in, but the links would not be clickable. My additional changes fix that issue by resolving the fileNames to their absolute path (joining them w/ the workspace root).

I'll rebase and test


Make travis run on rc- branches

Codecov Report

Merging #188 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##           master   #188   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files          39     39
  Lines         946    946
  Branches      132    132
=====================================
  Hits          946    946

Flag
Coverage Δ

#builder
100% <ø> (ø)
:arrow_up:

#runtime
100% <ø> (ø)
:arrow_up:

#srcapp
100% <ø> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9f408ad...ed4f7e0. Read the comment docs.


<IP_ADDRESS> compatible

I've updated the core/ files to work with the new instance search mechanism in Agda <IP_ADDRESS>, and begun porting some of the theorems/ folder.
This is a merge request into the agda-2.6.1-compatible branch since PRs can only merge into existing branches.
Would it be worth having a separate branch for <IP_ADDRESS> since the way instances work is breakingly different between the 2.6.* and 2.7.* Agda versions?

Thanks! Merged to new branch latest-agda.


Establish Guidelines for Retaining Artifacts in Blob Storage

Related to https://github.com/dotnet/source-build/issues/4511
We should aim to keep only necessary artifacts in the blob storage. As part of this issue, we should identify which artifacts are necessary and establish guidelines for what should be retained in the blob storage. We should also investigate strategies for cleaning up unnecessary artifacts in the official blob storage.

[Triage] This would need to be automated given the restricted access to blob storage.  There is minimal ROI from this clean given the cost to automate.  Closing as won't fix.


Fuse ESB Role Development, basic installation of FUSE

Basic role setup in line with other middleware roles
has travis ci integration for syntax check alone

LGTM. I do not have access to merge this. @sherl0cks do you?

@sabre1041 let me know if you all need access, I have the power.

@itewk i do. can i get some good karma my way?

@itewk, @sherl0cks gave me the power. thanks for the offer!


Add option domain-search

Add support for very useful option  domain-search

Thank you so much for this contribution! Very useful indeed :)


Start a container on my rktnetes cluster with an alternate KVM stage1, using the supported annotations.
What did you expect to see?
The pod starting properly
What did you see instead?
The pod error-cycled forever, with a logging unsupported error.
rpc error: code = Unknown desc = logging unsupported for pod "afdf8445-d0c6-4f60-9225-c80ea11ba4e1"

Trying to find some information on it, it seemed to have been solved a long time ago but I guess not?

While I don't exactly know the details behind this, it seems to be an issue with rktlet usage thus you'd better report it at https://github.com/kubernetes-incubator/rktlet.
/cc @iaguis @alban who may have better knowledge on this

@lucab My apologies I didn't realize! I actually wasn't even using rktlet but was instead using the old rktnetes guide


duplicate of index column

Does we need every time when we download table add new index column?
see http://localhost:8888/notebooks/doc/python/TableAPI.ipynb
Let say you have table and you format and save it.

Then you display that new table.
The new table will have duplicate for index column

for groovy kernel, the index column is added to 5 position

Our decision for this issue:

don't adding new column when save table to csv file.
don't use own label for index column (i.e. the same as pandas).
if there is index in csv its normal column

I think this issue is only about 1st point, it fixes problem with duplicate index.
I suggest to create new issue for 2nd and 3rd point, because the issue won't fix a bug but provide similar behaviour like in pandas view

issue #7994 for 2nd and 3rd  points


Using package github.com/Shopify/sarama

When using using the command
go test 123_test.go
I get
command-line-arguments
package command-line-arguments (test)
imports github.com/Shopify/sarama: found packages main (123.go) and test (123_test.go) in C:\Gocode\src\123
FAIL    command-line-arguments [setup failed]
FAIL
I have used go mod init github.com/Shopify/sarama
and
go get -u -d github.com/Shopify/sarama
and
go get github.com/Shopify/sarama
and it is returning the same error. Please help me fix this error

@AwesomeAlpha this looks like a mistake in your local files rather than a problem with Sarama

found packages main (123.go) and test (123_test.go)

You’ve put your test file in a package called “test” rather than “main”. Make both files the same package.

if I use same package names, it throws cyclic dependency issue.

@AwesomeAlpha upload all your files to a git repo and share the link, but it doesn’t sound like Sarama issue

Thank you but even if i try
go list or go fix
can't load package: import cycle not allowed
package github.com/Shopify/sarama
imports github.com/Shopify/sarama
same kind of error popping up


Add features in convert_script

Description of the feature

[ ] Add the possibility to begin the code with mapdl.clear().
[ ] If the command is mapdl.eplot(), add a parameter to convert it to mapdl.eplot(vtk=False). If this is not added, an error is raised when running the code from Python

Steps for implementing the feature

Useful links and references
No response

I like the "clear" option. Although if we are launching MAPDL with the default arguments, it should create a new session (hence cleared).
But I'm not sure why mapdl.eplot(vtk=False)... Shouldn't we take advantage of Pyvista plotting?

Makes sense @mikerife. Please @clatapie add the optional /clear argument.

Sure, I can implement it that way. Thank you for this precision @mikerife!

Rethinking this issue:

Add the possibility to begin the code with mapdl.clear(). I dont believe it is needed because of the previous comments. Normally, the converter will generate code to launch a new instance.
If the command is mapdl.eplot(), add a parameter to convert it to mapdl.eplot(vtk=False). If this is not added, an error is raised when running the code from Python I rather use vtk=True by default.
Check the mapdl.prep7() command. Not sure what you mean here.

Closing issue until new information arises.


CentOS support for masakari-deploy

Modify cookbook for CentOS support
In centos env,
(1) Install the dependencies from yum repos (epel enable)
(2) Build rpms from source
(3) Install the rpms
(4) Deploy conf and other required files
Add support for build environment with masakri source from custom branch
In chef.json, use "masakari_ci_conf" section and set "masakari_git" and "masakari_branch".
Example is in Vagrantfile (Comment section).
Deploy masakari behind proxy
Add instructions to comment section of Vagrantfile.

LGTM, waiting others comments.  I'll merge this PR once [1] has been merged.

https://github.com/ntt-sic/masakari/pull/21

Added some minor comments above. Other than these, looks good to me.

Thanks for your fix. Now, it's ready to merge.


Have a default URL at Collections level

I would like to propose the addition of a feature that allows users to define a default URL, either a variable or a typed URL, at the collection level and or folder level in order to use it on all requests within that collection.

Definitely a needed feature. Appears to be on the roadmap for v1 already.

Totally agree!
This might also fix response previews with relative urls, for assets or whatever non-absolute url.
Also, this would allow to be less verbose in the path definition:
/api/foo
instead of (something like)
{{baseUrl}}/api/foo
PS: @helloanoop awesome job and philosphy on Bruno :raised_hands: , sorry not be confident enough in JS to submit a PR

It looks like there is a Base URL preset we can set per collection now:

But it doesn't seem to be picked up in the requests within the collection.
I tried adding a {{baseUrl}} variable in the request path, but that also doesn't work.
Am I missing something?

For now I just added a variable to the environment. Useful anyway since, in my case, it actually differers for test and production envs.

This issue seems quite related with #292

Maybe a better approach to this request would be to implement variables at the collection level which would align with the issue @David-Lannion mentioned in the previous comment.

Ran into this bug today. The collection's preset Base URL is not being picked up.
I'm a dummy...
Now I understand what @peteruithoven meant when he said "Update: Ah wait, it's just pre-filled when you create a new request..."
It just prefills the input textbox with the baseURL when you create the new request. I was wondering why the baseURL was there. Maybe it copied it from my clipboard? Nah, it is just copying it from the preset baseURL for the collection.
Definitely needs an UX improvement.

I added a base url, but all the imported requests still only have the request path. Would like to not have to manually update every request.

The feature to add a default base URL at the collection level has already been implemented in Bruno. Thank you, @garposmack, for submitting this valuable enhancement request.

It definitely needs a UI/UX improvement.

I agree, @Ativerc. Could you please create a new issue for this and include any additional suggestions or ideas you have?

Can someone point to documentation on how this is supposed to work or explain it. I have imported my OpenAPI spec and set Base URL to my local test instance: "http://localhost:8000", but clearly I'm misunderstanding something because when I select an API endpoint to execute, it does not insert the provided Base URL at the front of that request.

Okay, for anyone new to Bruno and not understanding what those above were saying about "Base URL". It is pre-filled when creating a new "request", that is, you're creating a brand new verb and path option on the left-hand side. So, it's not useful for a pre-existing API as far as I can tell. See here, my "Base URL" was set as above and it is filled into this dialog box...

It would also be useful if the base URL for every request changed when the base URL setting is updated. Right now, it looks like the base URL is just a "prefill" for new requests and not actually a variable that can be changed and update all existing requests.


Code of Conduct presentation

Jessica Hilt contacted 8/23

8/27 Jessica Hilt confirmed

Jessica registered 9/9


refactor: Avoid needing to directly inspect shiny's bootstrap_deps()

This PR is pre-work for posit-dev/py-shiny#1282 to avoid errors in Shiny's test suite caused by shinyswatch's assumptions about the now changing structure of bootstrap_deps().
The primary goal is to avoid having to inspect the Bootstrap dependencies provided by Shiny. Instead, we suppress the bootstrap, bootstrap-js and bootstrap-css dependencies in shinyswatch's html dependency.
In posit-dev/py-shiny#1282, Shiny's Bootstrap dependency will switch to bootstrap-js, bootstrap-css and bootstrap-meta, which can be suppressed with a new helper function shiny.ui._html_deps_external.bootstrap_deps_suppress().
The goal of posit-dev/py-shiny#1282 is to introduce shiny.ui.include_bootstrap_css() that could be used by shinyswatch in the future, although it's not strictly required. The approach used by include_bootstrap_css() is quite similar to the one taken here, except that its stated goal is to replace only the CSS and not the entire Bootstrap bundle. That function will also provide additional runtime checks to make sure that Shiny's Bootstrap is compatible.
In a future PR, I'll come back to shinyswatch and explore using the new function. In the mean time, the approach in this PR should work for both current and the expected future implementation in Shiny. As long as this PR doesn't break shinyswatch, I'd like to merge this so that work in Shiny can be tested.

Moving back to draft temporarily... I'll have a better approach shortly and have a plan to improve the theme picker while I'm here.


Fix #314: Delete old album art during re-import

Part of #1679 by @reiv.

With a few changes to importer.py, the re-import workflow deletes orphaned album art by keeping track of which files aren't reused by fetchart during the import process.

Here, we should probably write a test for two different situations:

No metadata changes, so the album stays in the same location.
The metadata changes so that the album needs to be moved to a new directory, and the art moved along with it.

In both cases, we should verify that we end up with exactly one image file (not two).

I'm fairly certain that #1683 takes care of the first case, but it can't hurt to add more tests. As for the second case, a move could also be triggered by modifying the path templates, though I think we can treat that the same way as a metadata change.
I've taken a gander at the tests for re-imports located in test_importer.py and it looks as though it's meant for situations where files are being re-imported from outside the library, rather than from inside (correct me if I'm wrong). Any pointers on setting up a test case for the latter?

I've taken a gander at the tests for re-imports located in test_importer.py and it looks as though it's meant for situations where files are being re-imported from outside the library, rather than from inside (correct me if I'm wrong). Any pointers on setting up a test case for the latter?

Hmm; the way I read it, the setUp method adds the music to the library, and then the _setup_import_session call instructs the tests to import the same music again. So think this is indeed importing already-in-the-library files, right?

Regarding your second point: I've been thinking about this as well. A better approach might indeed be to let the importer handle it. Specifically, we would need to grab the artpath from a replaced album and assign it to the artpath of the corresponding imported album. Then calling item.move() in manipulate_files() should automatically move the art along with it.

Yes, exactly! That would be ideal. The remaining decision is whether the importer should be responsible or fetchart should. I lean toward the former: a re-import can just preserve the album art as it was before. Then we would need to ensure that the fetchart plugin doesn't interfere with that logic.

Going with this approach simplified things a lot, actually; the fix was a one-liner.
Now we'd have to make fetchart a bit smarter, too; is there currently a way for plugins to distinguish between re-imports and regular imports?

Awesome! I'm so excited the fix could be that simple. Woohoo!
It depends on when the plugins run, but ideally, a plugin should be able to inspect the task's replaced_albums field. If that's non-empty, then it's a re-import. Would that work?

Awesome! I'm so excited the fix could be that simple. Woohoo!

Me too! Turns out the mechanism was already there, it was just missing a single cog for it to work.
Anyway, if the fetchart plugin has access to the album associated with the task, then instead of ignoring re-imports altogether, we could check if an album already has its artpath set (again making sure that the image file actually exists), and if so, skip it.

Can we merge this PR?

Yes, looks 100% ready. Thanks again! :sparkles:


Feat GitIngest Browser Extension

https://github.com/user-attachments/assets/815978b7-9e2b-4e97-99a9-5a7ae511f474

https://github.com/user-attachments/assets/ce42411d-f3da-4415-a9ec-4e5bde7912a8
for initial version, it works well.

Screen.Recording.2024-12-30.at.17.25.00.mov
for initial version, it works well.

@cevatkerim Interesting that your browser didn't  load the icon at the left of the button
What browser are you using on which platform?


MacOS support

Can I build this on MacOS Catalina?
Currently when I'm trying to launch stack build I have an error
Installed GHC.

Error: While constructing the build plan, the following exceptions were encountered:

In the dependencies for Cabal-<IP_ADDRESS>:
    unix must match >=<IP_ADDRESS> && <2.8, but the stack configuration has no specified version  (latest matching version is <IP_ADDRESS>)
needed due to idris-cam-<IP_ADDRESS> -> Cabal-<IP_ADDRESS>

In the dependencies for base-compat-0.10.5:
    unix needed, but the stack configuration has no specified version  (latest matching version is <IP_ADDRESS>)
needed due to idris-cam-<IP_ADDRESS> -> base-compat-0.10.5

In the dependencies for directory-<IP_ADDRESS>:
    unix must match >=2.5.1 && <2.9, but the stack configuration has no specified version  (latest matching version is <IP_ADDRESS>)
needed due to idris-cam-<IP_ADDRESS> -> directory-<IP_ADDRESS>

In the dependencies for haskeline-<IP_ADDRESS>:
    unix must match >=2.0 && <2.9, but the stack configuration has no specified version  (latest matching version is <IP_ADDRESS>)
needed due to idris-cam-<IP_ADDRESS> -> haskeline-<IP_ADDRESS>

In the dependencies for hfsevents-0.1.6:
    unix needed, but the stack configuration has no specified version  (latest matching version is <IP_ADDRESS>)
needed due to idris-cam-<IP_ADDRESS> -> hfsevents-0.1.6

In the dependencies for idris-1.3.2:
    unix must match <2.8, but the stack configuration has no specified version  (latest matching version is <IP_ADDRESS>)
needed due to idris-cam-<IP_ADDRESS> -> idris-1.3.2

In the dependencies for idris-cam-<IP_ADDRESS>:
    unix must match <2.8, but the stack configuration has no specified version  (latest matching version is <IP_ADDRESS>)
needed since idris-cam is a build target.

In the dependencies for network-<IP_ADDRESS>:
    unix must match >=2, but the stack configuration has no specified version  (latest matching version is <IP_ADDRESS>)
needed due to idris-cam-<IP_ADDRESS> -> network-<IP_ADDRESS>

In the dependencies for process-<IP_ADDRESS>:
    unix must match >=2.5 && <2.8, but the stack configuration has no specified version  (latest matching version is <IP_ADDRESS>)
needed due to idris-cam-<IP_ADDRESS> -> process-<IP_ADDRESS>

In the dependencies for unix-compat-0.5.1:
    unix must match >=2.4 && <2.9, but the stack configuration has no specified version  (latest matching version is <IP_ADDRESS>)
needed due to idris-cam-<IP_ADDRESS> -> unix-compat-0.5.1

In the dependencies for zip-archive-0.4:
    unix needed, but the stack configuration has no specified version  (latest matching version is <IP_ADDRESS>)
needed due to idris-cam-<IP_ADDRESS> -> zip-archive-0.4

Some different approaches to resolving this:

  * Recommended action: try adding the following to your extra-deps in /Users/bmwant/projects/idris-cam/stack.yaml:

- unix-<IP_ADDRESS>@sha256:e69269a17b9fa26cb77f3f55e86c39d0a4940ccfa0c4bc20826919d2572076ad,3496

Plan construction failed.

Is it related to the OS used or some specific versions of toolset is required? Any guidance of how I should debug/resolve this error would be helpful.

you have to change stack.yml, comment and uncomment platform specific settings:
https://github.com/thautwarm/idris-cam/blob/d25827fa8cf2203fdef65d685a04c1b305fcf430/stack.yaml#L6
personally I'd suggest you use
https://github.com/thautwarm/Quick-Backend
I will keep maintaining it as a future research, the technique stuffs of this project are out of date.

So I was able to install on my Ubuntu 18.04 machine with a following configuration stack.yml
resolver: lts-13.17

packages:
- .

# for unix:
extra-deps:
- idris-1.3.1@sha256:8586191c8722199c608b86a44cbf6571875fd425a0041e798ecc100d2e9537fc
- Cabal-<IP_ADDRESS>@sha256:2a80d8fb655474f0eaeb20434c47f64f84e6302e55973056f00df8ca050b9683
- aeson-<IP_ADDRESS>@sha256:1ab29743b4c4c9be7df57200ed2c8779048adda4cead075944b7b471e93ed4c3
- code-page-0.1.3@sha256:9e13dee5c6d23e570f72cb376e7996ffcd9728c4a6daa60f05f2f2c204296eec
- containers-<IP_ADDRESS>@sha256:28ad7337057442f75bc689315ab4ec7bdf5e6b2c39668f306672cecd82c02798
- megaparsec-6.5.0@sha256:a1dfed92863607ea9f7609a0578d486761a052a4d01aaff95b90f25da2564346
- network-<IP_ADDRESS>@sha256:7cb8d7bc2080be946cf87890c807b12dae9c59314fa030b2687999100ce60512
- zip-archive-0.3.3@sha256:53f871653059f87285f434319598f380d986cd7fd9219de555dcafc496121a7c
- binary-<IP_ADDRESS>@sha256:e9387a7ef2b34c6a23b09664c306e37cc01ae2cb4e4511a1c96ffb14008c24b0,6262

and then also
$ pip install toolz
$ pip install numpy

before doing pip install idris-python


PT#140565959-Requestes sorted by most recent

https://www.pivotaltracker.com/story/show/140565959
Refactored requests explorer/service to align with codebase
Updated sort order so default is by created_on descending
No visual changes, just a bunch of cleanup for an explorer that was super old (in code years 😆 )

Fine backport details:
$ git log -1
commit bef5df0ce26ba17ca2bff4e2ad1c61495da29e9c
Author: Chris Kacerguis<EMAIL_ADDRESS>Date:   Fri Mar 31 13:00:20 2017 -0500

    Merge pull request #621 from AllenBW/enhancement/140565959-update-requests-explorer

    PT#140565959-Requestes sorted by most recent
    (cherry picked from commit 3a97d9d01c9d87bb53740a1530ffed07e42a4437)


Added types definition

How do I use this revision (typescript):
import * as express from 'express';
import * as swagger from 'swagger-spec-express';
import * as swaggerUi from 'swagger-ui-express';

const app = express() as swagger.ISwaggerizedExpress;
swagger.swaggerize(app);

swagger.initialize(app, {
  title: 'API Title',
  version: '1.0.0'
});

swagger.common.parameters.addQuery({
  name: 'query',
  description: '',
  required: true,
  type: 'string'
});

app.get('/search', (req, res) => {
  const { query } = req.query;

  res.status(200).json(query);
}).describe({
  responses: {
    200: {
      description: 'Okay'
    }
  },
  tags: ['Search'],
  common: {
    parameters: {
      query: ['query']
    }
  }
});

swagger.compile();

app.use('/', swaggerUi.serve);
app.get('/', swaggerUi.setup(swagger.json()));

app.listen(8000, () => {
  console.log('Server is running on http://localhost:8000');
});

Thank you so much for your contribution, PR merged and a new release should go out from CI shortly!


Align incentives for IICO participants by offering affiliate referral commission baked into the smart contract

"old style" ICO
The more people participate, the more hype, the better for me.
IICO
I want my 1 ether to give me the most amount of tokens.
I do not have incentives for promoting the sale, otherwise 1 ether gives me less tokens.

Allow users to register their nickame.
In sale front-end allow field for referral.
To incentivise use of front-end over sending ETH via fallback payable:

put a referral, get 2% extra each
leave referral empty, get 1% extra

There could be some multi-level referral bonuses too.
They would be calculated during finalize / redeem phase.
Totally not use about gas usage (amount of lookups and efficient data structures), more concerned about the right incentives - so that poeple investing ETH into sale had incentives to promote the project.
After all the goal of the core team is to raise enough fund to allow for continous development.

They could just set up a fake account to get all the bonus themselves and we already stated there were no referral in the T&C of the sale.
You are right participants are not incentivized to promote the project during a round of IICO.
But as soon as the round is over, they'll have incentive to do so and the project can benefit from that due to tokens having a higher price in the subsequent rounds.

We could have a special default value or EFF referral field - so even if you don't know anyone who has recommended you, there is 1% or 2% bonus.
The bonus anyway is to incentivize the referral programme - so that those who haven't been referred are not in a worse position.
(I'm core supporter, biggest fan - I'm the first one to know and me getting penalized for not being referred is not fair)
About the incentives after the IICO - yeah, that is true... Me trying to find balance, so that incentives are aligned - otherwise cannot act with full commitment and integrity.


croutoncycle - broken symlink

I don't see in code where croutoncycle is installed, so perhaps this is my own issue, but I wanted to be sure. The file in /usr/local/bin does not exist. Is it possible there is a missed install step?

How do I switch to Fedora?
Type croutoncycle :0 to another terminal or press CTRL+ALT+SHIFT+F2.

Broken link (red text)
ls -la /bin/croutoncycle
lrwxrwxrwx 1 root root 27 sept 27 13:42 /bin/croutoncycle -> /usr/local/bin/croutoncycle

Copying /usr/local/crouton-fedora/chroots/fedora/usr/local/bin/croutoncycle to /usr/local/bin/ seemed to work ok.

Maybe it is unclear (again), you need to execute croutoncycle :0 in a crosh shell, not Fedora shell. The binaries are installed with crouton, but I never experienced this issue. Does CTRL+ALT+SHIFT+F2 key combo work?
Also if you used a different path with -p parameter the binaries would be installed in that path rather than the default one. Maybe this is the issue?
Cheers.

I did use a custom path, that's likely why I had to copy the binary to /usr/local/bin
On September 28, 2016 6:22:07 AM EDT, Nemanja Milosevic<EMAIL_ADDRESS>wrote:

Maybe it is unclear (again), you need to execute croutoncycle :0
in a crosh shell, not Fedora shell. The binaries are installed with
crouton, but I never experienced this issue. Does CTRL+ALT+SHIFT+F2 key
combo work?
Also if you used a different path with -p parameter the binaries
would be installed in that path rather than the default one. Maybe this
is the issue?
Cheers.
--
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub:
https://github.com/nmilosev/crouton-fedora/issues/9#issuecomment-250128967

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Closing this, since I think this is how Crouton is intended to work. If you use a custom prefix you have to make a symlink manually.


Changing the status of HORM

Changes made to documentation. HORM has not been deprecated in windows 10, and it was brought back in 1511.

@sameersaiya,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

Per the email from Oleg, I changed "version 1607." to "version 1607 and later." The change should be live tomorrow. Thanks!


AccessViolationException when calling the callback function

Instead of:
public static extern void WorldRayCast(WorldId worldId, Vec2 origin, Vec2 translation, QueryFilter filter, System.IntPtr* fcn, void* context);
You need to use:
public static extern void WorldRayCast(WorldId worldId, Vec2 origin, Vec2 translation, QueryFilter filter, System.IntPtr fcn, void* context);
The difference in fcn.
Otherwise, an error occurs:
System.AccessViolationException: 'Attempted to read or write protected memory. This is often an indication that other memory is corrupt.'

Thanks! I fixed the issue and function pointers should be generated correctly now. I updated the bindings to the latest box2c commit and pushed a new nuget package as well.
https://www.nuget.org/packages/Box2D.NET.Debug/0.0.8


Using multiprocess pool to address the OOM issue

Previously, when using multiprocess mode with 20 generatorWorkers, the process will be killed as Out of memory error.
After adopting pool with maxtasksperchild, the processes are now down for OOM after more than 24 hours.
Data ingestion with perdayvolume=600:

Process status and memory usage:

I will check why the modular input test case failed.

Running more than three days with stable output and without process down.

I will check why the modular input test case failed.

All tests are passed now.