OtelAxumLayer and OtelInresponseLayer don't work with opentelemetry_otlp exporter

I switched from opentelemetry-jaeger/opentelemetry-zipkin to opentelemetry-otlp, and these two middlewares are no longer work for me. Previously, I can get trace id in response header for user debugging, but now it's missing.
let tracer = opentelemetry_otlp::new_pipeline()
        .tracing()
        .with_exporter(
            opentelemetry_otlp::new_exporter()
                .tonic()
                .with_endpoint("http://localhost:4317"),
        )
        .with_trace_config(sdktrace::config().with_resource(Resource::new(vec![
            opentelemetry::KeyValue::new("service.name", "kakashi"),
        ])))
        .install_batch(runtime::Tokio).unwrap();
let tracer_layer = tracing_opentelemetry::layer().with_tracer(tracer);

https://github.com/open-telemetry/opentelemetry-rust/issues/1338

Sorry, I missed to set propagator. After replacing global::set_text_map_propagator(opentelemetry_jaeger::Propagator::new()); with global::set_text_map_propagator(TraceContextPropagator::new());.
The response trace id became something like traceparent: 00-75950901f5529540ee4c93f55c5b273a-ea2176fd6996905a-01, how can I customize the different format? I remember Jaeger uses uber-id.

Close since I missed to pass proper propagater.

@npuichigo how did your fix look like?


Add essential metrics for CockroachDB

Feature Description
Collect essential CockroachDB metrics to enable customers to completely rely on Datadog for their mission critical workloads.
Essential Monitoring Metrics

CockrochDB 23.1 Metric Name
Metric Name in Datadog (add cockroachdb. leading prefix)
Category
Description
Narrative

sql.mem.root.current
[SUBMITTED TO DATADOG]
Platform
Current sql statement memory usage for root
Shows how memory set aside for temporary materializations (like hash tables, intermediary result sets, etc.) is utilized. Enables an operator to optimize memory allocations based on long term observations. The max amount is set with --max_sql_memory. If the utilization of sql memory is persistently low, perhaps some portion of this memory allocation can be shifted to --cache.

sys.host.disk.write.count
[SUBMITTED TO DATADOG]
Platform
Disk write operations across all disks since this process started
Reports the effective storage device write IOPS rate. To confirm the storage is sufficiently provisioned, assess the I/O performance rate (IOPS and MBPS) rate in the context of sys_host_disk_iopsinprogress metric.

sys.host.disk.read.count
[SUBMITTED TO DATADOG]
Platform
Disk read operations across all disks since this process started
Reports the effective storage device read IOPS rate. To confirm the storage is sufficiently provisioned, assess the I/O performance rate (IOPS and MBPS) rate in the context of sys_host_disk_iopsinprogress metric.

sys.host.disk.iopsinprogress
[SUBMITTED TO DATADOG]
Platform
IO operations currently in progress on this host
This is the storage device average queue length. It characterizes the storage device's performance capability. All I/O performance metrics are Linux counters and will match what would be reported by iostat. This metric corresponds to the avgqu-sz in the iostat command output. Operators need to view the device queue graph in the context of the actual read/write IOPS and MBPS metrics that show the actual device utilization. If the device is not keeping up, the queue will grow. Over 10 is bad. Around 5 means the device is working hard, trying to keep up. For internal (on chassis) nvme devices the queue is typically 0. For network connected devices, like AWS EBS volumes the normal operating range is 1-2. Spikes are OK. Means there was an IO spike, the device fell behind, but caught up. End-users may experience inconsistent response times but there should be no cluster stability issues. If the queue is >5 for extended period of time and IOPS or MBPS are low - the storage is most likely not provisioned per Cockroach Labs guidance. In AWS EBS it's commonly an EBS type not suitable as database primary storage, like gp2. If I/O is low and the queue is low, the most likely scenario the CPU is lacking and not driving I/O. With 2 vcpus (not supported for production deployments), it's a very likely scenario. There are quite a few background processes in the database that take CPU away from the workload, so the workload is just not getting the CPU. Review this runbook template article.

storage.write-stalls
[SUBMITTED TO DATADOG]
Storage
Number of instances of intentional write stalls to backpressure incoming writes
This metric records actual disk stall events. Arguably, all reports of disk stalls should be investigated be an operator. As a rule of thumb, 1 stall per minute is not likely to have a material impact on workload (beyond an occasional increase in response time). However 1/second should be viewed as problematic and investigated vigorously. It's particularly problematic if the rate persists over an extended period of time, and worse - is increasing.

admission.io.overload
[SUBMITTED TO DATADOG]
Health
1-normalized float indicating whether IO admission control considers the store as overloaded with respect to compaction out of L0 (considers sub-level and file counts).

sys.runnable.goroutines.per.cpu
[SUBMITTED TO DATADOG]
Health
Average number of goroutines that are waiting to run, normalized by number of cores
A value over 30 indicates a CPU overload. If the condition lasts a short period of time (a few seconds), the database users are likely to experience inconsistent response times. If the condition persists for an extended period of time (tens of seconds, or minutes) the cluster may start developing stability issues. Review this runbook template article.

seconds.until.enterprise.license.expiry
[SUBMITTED TO DATADOG]
License
Seconds until enterprise license expiry (0 if no license present or running without enterprise features)

rebalancing.queriespersecond
[SUBMITTED TO DATADOG]
KV - replication
Number of kv-level requests received per second by the store, considering the last 30 minutes, as used in rebalancing decisions.
This metric shows hotspots along the QPS dimension, providing insights into the ongoing rebalancing activities

queue.replicate.replacedecommissioningreplica.error
[SUBMITTED TO DATADOG]
KV - replication
Number of failed decommissioning replica replacements processed by the replicate queue

range.merges
[SUBMITTED TO DATADOG]
KV - replication
Number of range merges
DELETE metric? This is CRDB optimization for performance. Indicates that there has been deletes in the workload but how is this actionable?

sql.new_conns
[SUBMITTED TO DATADOG]
SQL
Number of new connection attempts.
Looking at the rate of it will show how frequently new connections are being established. This can be useful in determining if a high rate of incoming new connections is causing additional load on the server due to a misconfigured application.

txnwaitqueue.deadlocks_total
[SUBMITTED TO DATADOG]
SQL
Number of deadlocks detected by the transaction wait queue

txn.restarts.writetoooldmulti
[SUBMITTED TO DATADOG]
SQL
Number of restarts due to multiple concurrent writers committing first
See above

txn.restarts.unknown
[SUBMITTED TO DATADOG]
SQL
Number of restarts due to a unknown reasons
See above

txn.restarts.txnpush
[SUBMITTED TO DATADOG]
SQL
Number of restarts due to a transaction push failure
See above

txn.restarts.txnaborted
[SUBMITTED TO DATADOG]
SQL
Number of restarts due to an abort by a concurrent transaction
These errors are generally due to deadlocks. Deadlocks can often be prevented with a considerate transaction design. Identify the conflicting/deadlocking transactions and consider redesigning deadlock prone business logic implementation.

jobs.auto_create_stats.resume_failed
[SUBMITTED TO DATADOG]
Jobs - Table Statistics
Number of auto_create_stats jobs which failed with a non-retriable error
High-level indicator that auto-create statistics is failing which can lead to the query optimizer running with stale statistics. This can cause suboptimal query plans to be selected leading to poor query performance.

jobs.auto_create_stats.currently_running
[SUBMITTED TO DATADOG]
Jobs - Table Statistics
Number of auto_create_stats jobs currently running
Number of active auto create statistics job that could also be consuming resources. Ensure that foreground SQL traffic is not impacted by correlating with SQL latency and query volume metrics.

jobs.auto_create_stats.currently_paused
[SUBMITTED TO DATADOG]
Jobs - Table Statistics
Number of auto_create_stats jobs currently considered Paused
High-level indicator that auto-create statistics is paused which can lead to the query optimizer running with stale statistics. This can cause suboptimal query plans to be selected leading to poor query performance.

jobs.create_stats.currently_running
[SUBMITTED TO DATADOG]
Jobs - Table Statistics
Number of create_stats jobs currently running
Number of active create statistics job that could also be consuming resources. Ensure that foreground SQL traffic is not impacted by correlating with SQL latency and query volume metrics.

jobs.backup.currently_paused
[SUBMITTED TO DATADOG]
Jobs - Backup/Restore
Number of backup jobs currently considered Paused
This is a hedge against an inadvertent operational error leaving a backup job in a paused state for an extended period of time. In functional areas where a paused job can tie up resources or have concurrency impact or some other negative consequence. Paused backup may break the RPO.

jobs.changefeed.currently_paused
[SUBMITTED TO DATADOG]
Jobs - Changefeeds
Number of changefeed jobs currently considered Paused
Changefeed jobs should not be paused for long time b/c the protected timestamp prevent garbage collections. This is a safety catch to guard against an inadvertent pause. Consider setting an alert as a hedge for an oversight/operational error.

changefeed.emitted_bytes
[SUBMITTED TO DATADOG]
Jobs - Changefeeds
Bytes emitted by all feeds
Provides an operator with a useful context when assessing the state of changefeeds. This characterizes the throughput bytes being streamed from the CRDB cluster.

jobs.changefeed.protected_age_sec
[SUBMITTED TO DATADOG]
Jobs - Changefeeds
Alert on too much data being protected
Changefeed protects the data from being garbage collected. The operator should ensure the protected timestamp should not significantly exceed the GC TTL zone configuration. It's recommended to alert if the protected timestamp is greater than 3 times the GC TTL.

jobs.row_level_ttl.resume_completed
[SUBMITTED TO DATADOG]
Jobs - Row TTL
Number of row_level_ttl jobs which successfully resumed to completion
If TTL is enabled, this should be nonzero and correspond to the ttl_cron setting that was chosen. If it's zero, it means the job isn't running

jobs.row_level_ttl.resume_failed
[SUBMITTED TO DATADOG]
Jobs - Row TTL
Number of row_level_ttl jobs which failed with a non-retriable error
This should remain at zero. Repeated errors means the TTL job is not deleting data.

jobs.row_level_ttl.rows_selected
[SUBMITTED TO DATADOG]
Jobs - Row TTL
Number of rows selected for deletion by the row level TTL job.
Correlate with the metric below to ensure all the rows that should be deleted are getting deleted.

jobs.row_level_ttl.rows_deleted
[SUBMITTED TO DATADOG]
Jobs - Row TTL
Number of rows deleted by the row level TTL job.
Correlate with the metric above to ensure all the rows that should be deleted are getting deleted.

jobs.row_level_ttl.currently_paused
[SUBMITTED TO DATADOG]
Jobs - Row TTL
Number of row_level_ttl jobs currently considered Paused
Check this to ensure the TTL job isn't paused unless you intended it to be.

jobs.row_level_ttl.currently_running
[SUBMITTED TO DATADOG]
Jobs - Row TTL
Number of row_level_ttl jobs currently running
Monitor this to ensure there aren't too many TTL jobs running at the same time. Generally, this should be in the low single digits.

schedules.scheduled-row-level-ttl-executor.failed
[SUBMITTED TO DATADOG]
Jobs - Row TTL
Number of scheduled-row-level-ttl-executor jobs failed
Check this to ensure the TTL job is running. If it's non-zero, it means the job could not be created.

Motivation
Enable customers to completely rely on Datadog for their mission critical workloads running on CockroachDB
Related CockroachDB issue: https://github.com/cockroachdb/cockroach/issues/101362

Hallo, @kevin-v-ngo the following metrics from this request are not listed on cockroach documentation. We can only collect metrics that are exposed by the underlying system:

sql.mem.root.current, but sql.mem.current exists
sys.host.disk.write.count, sounds like this should be a metric on the disk integration.
sys.host.disk.read.count, sounds like this should be a metric on the disk integration.
sys.host.disk.iopsinprogress, sounds like this should be a metric on the disk integration.
storage.write-stalls, sounds like this should be a metric on the disk integration.
admission.io.overload
sys.runnable.goroutines.per.cpu, sounds like it should be on the go integration
seconds.until.enterprise.license.expiry
rebalancing.queriespersecond
range.merges
txnwaitqueue.deadlocks_total
txn.restarts.writetoooldmulti, but txn.restarts.writetooold is present and already collected
txn.restarts.unknown
txn.restarts.txnpush
txn.restarts.txnaborted
jobs.auto_create_stats.resume_failed
jobs.auto_create_stats.currently_running
jobs.auto_create_stats.currently_paused
jobs.create_stats.currently_running
jobs.backup.currently_paused
jobs.changefeed.currently_paused
changefeed.emitted_bytes
jobs.changefeed.protected_age_sec
jobs.row_level_ttl.resume_completed
jobs.row_level_ttl.resume_failed
jobs.row_level_ttl.rows_selected
jobs.row_level_ttl.rows_deleted
jobs.row_level_ttl.currently_paused
jobs.row_level_ttl.currently_running
schedules.scheduled-row-level-ttl-executor.failed

This one is present so they could be added as extra_metrics in the config, we can add it to the default collection as well:

queue.replicate.replacedecommissioningreplica.error

Hallo, @kevin-v-ngo the following metrics from this request are not listed on cockroach documentation. We can only collect metrics that are exposed by the underlying system:

Hi @hithwen, thanks for calling this out Julia! We will get our documentation updated. This is a docs issue where the metrics are already exposed and available through '/_status/vars' today.
FYI @thtruo

@kevin-v-ngo another thing to look at on the documentation is that the metrics do not really correspondo with how they're coming in from the endpoint. On the prometheus payload the metrics come with underscore separators instead of '.'. That was a bit confusing to me and I guess it can be to other users?

Hi @hithwen - I'll also chime in too alongside @kevin-v-ngo

On the prometheus payload the metrics come with underscore separators instead of '.'. That was a bit confusing to me and I guess it can be to other users?

Yes, we acknowledge that we use different delimiters (underscores for our _status/vars Prometheus compatible endpoint; periods for our docs and the metrics that get exposed in Datadog) and that it could be confusing (it did come up in an internal exercise we did) but for now since we've basically used the same conventions in the integration, we're requesting to keep the period '.' delimiters in the metrics that get exposed to Datadog.
Let me know if there's a concern on your end @hithwen.

Also I see you mention on your original post that these are CockrochDB 23.1 Metrics but that version is still not available on dockerhub. Are these metrics new on that version?

You might want to try this which contains a beta version of CRDB 23.1 https://hub.docker.com/r/cockroachdb/cockroach-unstable/tags
So our list contains a mix of metrics - some are new in 23.1, others already exist in 22.2 but were never added to the integration. Let us know if you need to know what corresponds to which version, or if you are sufficiently unblocked to proceed - thanks!

Hi @hithwen quick follow up from the last message: we'd like to also include 6 additional metrics related to our various certificate expiration metrics, which are also noted in our public runbook template.
Can you let us know if we can include these as part of this iteration of changes? Thanks!

CockroachDB 23.1 Metric Name
Metric Name in Datadog(add cockroachdb. leading prefix)
Category
Description
Narrative

security.certificate.expiration.ca
[SUBMITTED TO DATADOG]
Expiration

security.certificate.expiration.client-ca
[SUBMITTED TO DATADOG]
Expiration

security.certificate.expiration.ui-ca
[SUBMITTED TO DATADOG]
Expiration

security.certificate.expiration.node
[SUBMITTED TO DATADOG]
Expiration

security.certificate.expiration.node-client
[SUBMITTED TO DATADOG]
Expiration

security.certificate.expiration.ui
[SUBMITTED TO DATADOG]
Expiration

Hey @thtruo,
I'm working on getting these metrics added to our CockroachDB integration but am having some difficulty exposing the security.certificate metrics. Is it possible for you provide steps on how to expose these metrics or an example payload of them? It would a great help in getting all this added to the same pull request: https://github.com/DataDog/integrations-core/pull/14484
I've also confirmed that these 7 metrics aren't included in the 22.01 image but are available on the unstable image provided by CockroachDB:
admission.io.overload
queue.replicate.replacedecommissioningreplica.error
jobs.auto_create_stats.currently_paused
jobs.backup.currently_paused
jobs.changefeed.currently_paused
jobs.changefeed.protected_age_sec
jobs.row_level_ttl.currently_paused

Thanks @Kyle-Neale ! Regarding exposing security.certificate.* metrics, those only get emitted from CockroachDB (and appear in _status/vars) when you run the CockroachDB cluster in secure mode. https://www.cockroachlabs.com/docs/v23.1/secure-a-cluster#step-2-start-the-cluster
Check to make sure that you're not passing in the --insecure flag in your cockroach start command. If so, that --insecure needs to be removed and you will need to create certificates and keys to run in secure mode (see steps here)

The addition of both sets of essential metrics have been merged and should be available in an up coming agent release. @thtruo if you have no other questions, are we good to close out this issue?

It doesn't seem like it will be packaged in v7.45 so I would expect to see it in the following release, especially since the PR's have been merged 👍

👋 I confirm this change has been release with Agent 7.46: https://github.com/DataDog/integrations-core/blob/master/cockroachdb/CHANGELOG.md
Closing this issue now, feel free to reopen if needed 🙂


Angular Material

Have you tried use this with angular material 5 ?

I have not, I don't foresee any problems that would prevent it from working though.


Add a declaration attribute for implicitly unwrapped optional.

Attach this attribute to VarDecls declared as IUO, and to function decls
that have a result type that is an IUO.
NFC at the moment. Eventually we'll use these to determine where to
implicitly unwrap optional values.

@swift-ci Please smoke test

Well, technically this is not NFC since it does change some test output in terms of what we're printing for attributes.

@swift-ci Please smoke test


[4.0] support for custom events

Fixes #38
Added custom event API:
    class FooEvent(EventBase):
        pass

    class MyObjEvents(CharmEvents):
        foo = EventSource(FooEvent)

    class MyObject(Object):
        my_on = MyObjEvents()

    class MyCharm(CharmBase):
        def __init__(self, *args, **kwargs):
            super().__init__(*args, **kwargs)
            self.obj = MyObject(self, "child")
            self.framework.observe(self.obj.my_on.foo, self._on_foo)

    trigger(
        State(),
        "foo",
        MyCharm,
        owner_path=('obj', 'my_on')   # <<<--- this will tell Scenario where to find the event we're trying to emit
    )

From a user / tester point of view it would be great not to have to tell scenario about those paths. We are already telling scenario about the charm that we want to use and this charm observes some custom events. It feels like scenario already has all the info it needs.

That however does not generalize: it might be the charm lib the one that observes the custom event, not the charm. So the charm might be totally unaware of the custom event. What do we do then? Recursively search all child Objects for an event with that name? What if there are conflicts?

From a user / tester point of view it would be great not to have to tell scenario about those paths. We are already telling scenario about the charm that we want to use and this charm observes some custom events. It feels like scenario already has all the info it needs.

That however does not generalize: it might be the charm lib the one that observes the custom event, not the charm. So the charm might be totally unaware of the custom event. What do we do then? Recursively search all child Objects for an event with that name? What if there are conflicts?
We could make some cautious assumptions, of course, so that this situation is still nicely surfaced but we handle the common case automagically.
Another, completely different approach to implementing this could be to skip altogether the Framework.emit() machinery by pointing out manually the listener you want to exercise. This is somewhat complementary perhaps, because it could be that you're interested in testing all listeners (however many or dynamically defined), instead of just a predetermined one.
ctx = Context(MyCharm)
ctx.call_method(MyCharm._on_my_custom_event, args=(MyCustomEventType(MyMockedArg1, MyMockedArg2)))

WDYT?

I agree that it's possible and often the case that the listening of the event is done in the charm library file but my point is that in the end, it's still the charm that observes it regardless in which "file" / "part of the code" it's done.
Listening to event is always done in the charm's init, either directly or by instantiating other "Object" objects right?  I wouldn't necessarily search all file recursively from the get go, since there can be a bunch of events that the charm is not listening to in the libs if those classes aren't instantiated. I think I'd start by whatever the charm is instantiating (?).
And yes calling the charm's _on_my_custom_event can be an option, it's what we do with harness. My only concern with this approach is that it bypasses the event trigger and it's possible for the handler to work perfectly well but never be called (if it's not instantiated in the charm init) and have the illusion that all works well.

I agree that it's possible and often the case that the listening of the event is done in the charm library file but my point is that in the end, it's still the charm that observes it regardless in which "file" / "part of the code" it's done.

No my point is that we can't assume that this is the case.
Suppose you have this setup:

MyCharm instantiates MyLib and listens to MyLib.on.ready
MyLib instantiates MyOtherObj and listens to MyOtherObj.on.something, using it to trigger MyLib.on.ready

How do we test MyOtherObj.on.something? It's not in MyCharm._observers, but in MyLib._observers: we have no way to know who owns that event.
Somewhat simpler scenario:
Suppose some evil character defines a custom event called start: MyLib.on.start.
ctx.run(MyCharm, 'start') will match the 'wrong' event now. How do we tell scenario that no, we mean that 'other' start event?

And yes calling the charm's _on_my_custom_event can be an option, it's what we do with harness. My only concern with this approach is that it bypasses the event trigger and it's possible for the handler to work perfectly well but never be called (if it's not instantiated in the charm init) and have the illusion that all works well

The same point could be made about exercising custom events in isolation in the first place: maybe the code leading up to that event being emitted is buggy, and the event is not emitted when we think it should be.
That's why usually I recommend to construct the state in such a way that  is emitted 'the normal way'

I agree with @PietroPasotti that we should be explicit here. However, I don't love the proposed API -- having a separate event_owner_path arg (with a long name) that's a tuple of strings seems awkward.
I wonder if we can do something more like unittest.mock does when you patch something. The patch target can be a dotted name that it looks up using pkgutil.resolve_name, for example @patch('time.sleep'). Example of resolve_name:
>>> pkgutil.resolve_name('ops.CharmBase.on.config_changed')
<BoundEvent ConfigChangedEvent bound to CharmEvents.config_changed at 0x7fa7e183fa60>

(The target-getting code in unittest.mock looks up the first part using pkgutil.resolve_name and the last element is the attribute.)
My thinking is we'd just use the existing event: str parameter, and if it has a . in it, use the dotted-lookup approach; if not, search the regular places (I think just charm.on right now). I'm not sure we can directly use pkgutil.resolve_name, but worth considering (for consistency with unittest.mock). If not, we should be able to fairly easily use a getattr loop like you're doing already. The example would become something like this:
trigger(State(), 'MyCharm.obj.my_on.foo', meta=...)

# or maybe
trigger(State(), 'obj.my_on.foo', meta=...)

Also in my mind is the new collect_status event that I'm working on, which will likely be an event not on CharmBase.on but on (a newly-added) Unit.on and App.on event source. So this would have to be something like:
trigger(State(), 'ops.Unit.on.collect_status')
  # or
trigger(State(), 'unit.on.collect_status')

I realize having event_owner_path as a separate parameter that's a tuple of strs is more "conceptually pure", but just an awkward API. Hence the reason they did it using dotted names in unittest.mock, I suppose.

Nit: I think owner_path should be event_owner_path in the PR description. (Though this parameter might go away if we do something like the above.)

trigger(State(), 'MyCharm.obj.my_on.foo', meta=...)

# or maybe
trigger(State(), 'obj.my_on.foo', meta=...)

Also in my mind is the new collect_status event that I'm working on, which will likely be an event not on CharmBase.on but on (a newly-added) Unit.on and App.on event source. So this would have to be something like:
trigger(State(), 'ops.Unit.on.collect_status')
  # or
trigger(State(), 'unit.on.collect_status')

I really like the idea, I'd thought about it as well but my main concern was (as with many 'string-based' approaches) the lack of type hinting and the burden of having to keep that in sync as your code changes. I don't like the idea of baking the charm name in the event emitter path for example.
I thought of a workaround, reminiscent of a piece of code I wrote long ago:
def get_event(charm_class: Type[T]) -> T:   # fake return type to leverage type hints and autocompletion
    """Helper class to construct an event path."""
    return _PathBuilder([])

class _PathBuilder:
    """Proxy class to construct a path of strings based on getattr."""
    path: List[str]
    def __getattr__(self, attr):
        return _PathBuilder(self.path + [attr])

This way, you'd be able to write
trigger(State(), get_event(MyCharm).foo.bar.on.baz, meta=...)

and enjoy autocompletion and type checking on the path to on.baz.
Pros:

less validating and parsing to be done to help guard against user mistakes in building the path

Cons:

magical, possibly counterintuitive API

trigger(State(), 'MyCharm.obj.my_on.foo', meta=...)

decided to go with this suggestion of Ben.


"Protocol status"   stuck at    lastVerifiedBlockId=103,633

Describe the bug
I've been waiting for my bonds to be back to my wallet, it's been 28 hours now and I checked the logs and noticed that last verified block is locked at block     lastVerifiedBlockId=103,633.  gap between latest block head and last proved block reached to around 6000 blocks now

Steps to reproduce
Steps to reproduce here.
Spam policy

[X] I verify that this issue is NOT SPAM and understand SPAM issues will be closed and reported to GitHub, resulting in ACCOUNT TERMINATION.

Some changes are happening, it might take a while for verified blocks to catch back up; thanks for the report @StoneMac65


Clarification on handling UnprocessedKeys in executeBatchGet

To handle DDB limits effectively, I am splitting the items into chunks of 100 and attempting to retry any items returned in the UnprocessedKeys property until all items are successfully retrieved. However, I couldn't find clear examples in the documentation or elsewhere on how to properly use this property.
My current data is simple, and I haven’t been able to encounter a case where items are left unprocessed. This is method from a general repository meant to handle future use cases where more complex data or throttling might result in unprocessed keys.
The response structure I’m seeing is:
UnprocessedKeys?: Record<string, Omit<KeysAndAttributes, "Keys"> & {
  Keys: Record<string, NativeAttributeValue>[] | undefined;
}> | undefined;

Here is the approach I’m taking:

batchGet(items: KeyInputItem<this>[], options?: BatchGetCommandOptions<[this]>) {
  const command = options ? this.table.build(BatchGetCommand).options(options) : this.table.build(BatchGetCommand);

  const batch = items.map((record) => this.build(BatchGetRequest).key(record));

  return executeBatchGet(command.requests(...batch));
}

async batchGetInChunks(
  keys: KeyInputItem<this>[],
  options?: BatchGetCommandOptions<[this]>,
): Promise<ReturnType<typeof executeBatchGet>> {
  const chunkSize = 100;
  const allItems: any[] = [];
  const chunks: KeyInputItem<this>[][] = [];

  let lastResponse = {};

  // Split items into chunks
  for (let i = 0; i < keys.length; i += chunkSize) {
    chunks.push(keys.slice(i, i + chunkSize));
  }

  for (const chunk of chunks) {
    let unprocessedKeys = chunk;

    do {
      lastResponse = await this.batchGet(chunk, options);

      // Collect processed items
      if (lastResponse.Responses) {
        allItems.push(...Object.values(lastResponse.Responses).flat(1));
      }

      // Handle unprocessed keys
      if (lastResponse.UnprocessedKeys?.length) {
        unprocessedKeys = Object.values(lastResponse.UnprocessedKeys).flatMap(
          (unprocessed) => unprocessed.Keys || [],
        );
      } else {
        unprocessedKeys = [];
      }
    } while (unprocessedKeys.length > 0);
  }

  return { ...lastResponse, Responses: [allItems] };
}

Is this approach correct? Thanks in advance!

Hi @nevolgograd !
BatchGetCommands don't handle splitting requests in chunks of 100. However, there is a maxAttempts option that serves this specific purpose. You can set it to Infinity to make sure all your keys are processed (although it make take some time).
Does that answer your question?

@ThomasAribart wow, so you telling toolbox can handle both chunking and UnprocessedItems for me? I was adoring this package before but now I think I'm love.
Just to clarify, DDB docs states:

For example, if you ask to retrieve 100 items, but each individual item is 300 KB in size, the system returns 52 items (so as not to exceed the 16 MB limit). It also returns an appropriate UnprocessedKeys value so you can get the next page of results. If desired, your application can include its own logic to assemble the pages of results into one dataset.

Does maxAttempts: Infinity will continuously be getting hundreds and hundreds of records until the end of passed list of keys? And even handle the case when some records didn't fit due to their excessive size?

If none of the items can be processed due to insufficient provisioned throughput on all of the tables in the request, then BatchGetItem returns a ProvisionedThroughputExceededException. If at least one of the items is successfully processed, then BatchGetItem completes successfully, while returning the keys of the unread items in UnprocessedKeys.

I'm using PAY_PER_REQUEST billing mode, but still wondering how batch operation will behave when mode is provisioned?

@nevolgograd No DynamoDB-Toolbox does not handle chunking, although that could be a nice feature. But it does handle UnprocessedItems :)
From what I understand:

maxAttempts: Infinity will handle the case when some records didn't fit due to their excessive size. It will not be getting hundreds of record until the end as I think sending >100 requests will return an exception from AWS.
In the rare case where a ProvisionedThroughputExceededException is returned, DynamoDB-Toolbox will not catch it at the moment. The whole BatchGetItem will fail. However, it will continue as long as a single item is successfully processed at each try.

@ThomasAribart thanks a lot!
I've refactored batchGet method to support chunking and callers will decide whether to pass maxAttempts option:
async batchGet(items: KeyInputItem<this>[], options?: BatchGetCommandOptions<[this]>) {
  const command = options ? this.table.build(BatchGetCommand).options(options) : this.table.build(BatchGetCommand);

  const allResponses: FormattedItem<this>[][] = [];

  // Split items into chunks
  const chunks: KeyInputItem<this>[][] = [];
  const chunkSize = 100; // Maximum number of items per batch (DynamoDB limit is 100)
  for (let i = 0; i < items.length; i += chunkSize) {
    chunks.push(items.slice(i, i + chunkSize));
  }

  let lastResponse: { Responses: typeof allResponses } = { Responses: [] };

  for (const chunk of chunks) {
    // Execute the batch request
    const batch = chunk.map((record) => this.build(BatchGetRequest).key(record));
    lastResponse = await executeBatchGet(command.requests(...batch));

    // Collect processed items
    if (lastResponse.Responses) {
      allResponses.push(...lastResponse.Responses);
    }
  }

  return { ...lastResponse, Responses: allResponses };
}

Only problem I see now is lack of TS type for lastResponse variable. I see definition for ExecuteBatchGetResponse, but it is not exported and doing something like Awaited<ReturnType<typeof executeBatchGet>> unfortunately does not help (it's any).
Is there a conventional way of doing it?


Release/3.4

#18 Max delay, waitFor
#19 Event emitter / promise mode

Coverage decreased (-0.1%) to 96.941% when pulling 5246f8b703541a3240fda7e891f551978debf6ec on release/3.4 into bff07eae59c1ce0ed264b2f0cd0ee33cbb14b25e on master.

🉑  👍  💯


history-substring-search defines function that collides with beep alias

I would love to activate ZLE and get rid of this:
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active

... every time I hit enter. I changed systems from Ubuntu to openSUSE, and now zsh complains about my custom theme. How can I activate ZLE? There doesn't seem to be any documentation on google or any of the zsh, oh-my-zsh projects. I'm not even sure if this is a zsh problem or an .oh-my-zsh problem.
I'd also like to know if there is a way to check to see if the option is set; the documentation for zshzle indicates the option is set by default in interactive shells, so I don't know why it wouldn't be set in mine.
I have no idea where to start troubleshooting this. Help!
 Please post your full custom theme to see what we're dealing with.

Thanks for helping with this! Here you go:
# Name:   trapd00r zsh theme
# Author: Magnus Woldrich<EMAIL_ADDRESS>#
# This theme needs a terminal supporting 256 colors as well as unicode. It also
# needs the script that splits up the current path and makes it fancy as located
# here: https://github.com/trapd00r/utils/blob/master/zsh_path
#
# By default it spans over two lines like so:
#
# scp1@shiva:pts/9-> /home » scp1 (0)
# >
#
# that's  user@host:pts/-> splitted path (return status)
#
# If the current directory is a git repository, we span 3 lines;
#
# git❨ master ❩ DIRTY
# scp1@shiva:pts/4-> /home » scp1 » dev » utils (0)
# >

autoload -Uz vcs_info
autoload -U add-zsh-hook

[ -n "$TMUX" ] && export TERM=screen-256color

#prompt_lvl="𝄞%"
prompt_lvl="$"
if $PUTTY ; then prompt_lvl="$"; fi

# We dont want to use the extended colorset in the TTY / VC.
if [ "$TERM" = "linux" ]; then
  source $ZSH/themes/afowler.zsh-theme
  return
fi

local c0='%{\e[m%}'

# colors
local c1="%{$(printf "\e[38;5;231m")%}" # white
local c3="%{$(printf "\e[38;5;250m")%}" # gray-
local c5="%{$(printf "\e[38;5;245m")%}" # gray
local c7="%{$(printf "\e[38;5;240m")%}" # gray+
local c9="%{$(printf "\e[38;5;322m")%}" # black

local c11="%{$(printf "\e[38;5;208m")%}" # orange
local c12="%{$(printf "\e[38;5;142m")%}" # yellow-green
local c13="%{$(printf "\e[38;5;160m")%}" # red
local c14="%{$(printf "\e[38;5;112m")%}" # green
local c15="%{$(printf "\e[38;5;230m")%}" # light yellow

# background colors
# local bc1="%{$(printf "\e[48;5;321m")%}" # white
# local bc3="%{$(printf "\e[48;5;250m")%}" # gray-
# local bc5="%{$(printf "\e[48;5;245m")%}" # gray
# local bc7="%{$(printf "\e[48;5;240m")%}" # gray+
# local bc9="%{$(printf "\e[48;5;322m")%}" # black

local bc11="%{$(printf "\e[48;5;208m")%}" # orange
# local bc12="%{$(printf "\e[48;5;142m")%}" # yellow-green
# local bc13="%{$(printf "\e[48;5;160m")%}" # red
# local bc14="%{$(printf "\e[48;5;112m")%}" # green
local bc15="%{$(printf "\e[48;5;230m")%}" # light yellow

zstyle ':vcs_info:*' actionformats \
    '%{$c8%}(%f%s)%{$c7%}-%F{5}[%F{2}%b%F{3}|%F{1}%a%F{5}]%f '

zstyle ':vcs_info:*' formats \
    "%{$c8%}%s%%{$c7%}❨ %{$c9%}%{$c11%}%b%{$c7%} ❩%{$reset_color%}%f "

zstyle ':vcs_info:(sv[nk]|bzr):*' branchformat '%b%F{1}:%F{3}%r'
zstyle ':vcs_info:*' enable git

add-zsh-hook precmd prompt_jnrowe_precmd

# Get the status of the working tree
custom_git_prompt_status() {
  STATUS=$(git_prompt_status)
  if [[ -n "$STATUS" ]]; then
    STATUS="$ZSH_THEME_GIT_PROMPT_STATUS_SEPARATOR$STATUS"
  fi
  echo $STATUS
}

prompt_jnrowe_precmd () {
  vcs_info
  local prompt_git_dir
  dir_status="$(pwd | sed "s@${HOME}@~@g")"
  dir_status="$(echo ${dir_status} | sed "s@/@%{$c11%}/%{$c14%}@g")"
  dir_status="%{$c14%}${dir_status}%{$reset_color%}"

  prompt_git_dir="$(git rev-parse --show-prefix 2> /dev/null)"
  prompt_git_dir=${prompt_git_dir%?}

  prompt_git_home=$(git rev-parse --show-toplevel 2> /dev/null)
  if [ "" = "${prompt_git_home}" ]; then
    prompt_git_home="$(pwd | sed "s@${HOME}@~@g")"
  else
    if [ "" != "$prompt_git_dir" ]; then
      prompt_git_dir="/${prompt_git_dir}"
    fi
    prompt_git_home=${prompt_git_home/${~HOME}/"~"}
    local top="%B${prompt_git_home:t}%b"
    prompt_git_home="${prompt_git_home:h}/${top}${prompt_git_dir}"
    prompt_git_dir="${top}${prompt_git_dir}"
  fi

  # color the directory dividers
  prompt_git_home="$(echo ${prompt_git_home} | sed "s@/@$c11/$c14@g")"
  prompt_git_home="%{$c14%}${prompt_git_home}%{$reset_color%}"
  prompt_git_dir="$(echo ${prompt_git_dir} | sed "s@/@$c11/$c14@g")"
  prompt_git_dir="%{$c14%}${prompt_git_dir}%{$reset_color%}"

  user_and_pc="%{$c5%}%n%B%{$c11%}@%b%{$c3%}%m%{$reset_color%}%B%{$c11%}:%b"

  if [[ $EUID -ne 0 ]]; then
    my_prompt="%B%{$c11%}${prompt_lvl}%b%{$reset_color%} "
  else
    my_prompt="%B%{$c11%}#%b%{$reset_color%} "
  fi

  ZSH_THEME_GIT_PROMPT_SUFFIX="%B%b$(custom_git_prompt_status) %{$c0%}%{$c15%}%B%b${prompt_git_dir}%{$c14%}%B%{$c11%}± %b%{$reset_color%}"
}
ZSH_THEME_GIT_PROMPT_PREFIX="
%{$fg[black]%}%{${bc15}%}%B "

ZSH_THEME_GIT_PROMPT_DIRTY="%B%{$c13%}✱ %b" # ❗❕☔☂Ⓓ
ZSH_THEME_GIT_PROMPT_CLEAN="%B%{$c14%}✔ %b" # ✔✅☀Ⓞ

ZSH_THEME_GIT_PROMPT_UNTRACKED="%{$fg[cyan]%} ?" # ✭ⓣ⬒⬓
ZSH_THEME_GIT_PROMPT_ADDED="%{$fg[cyan]%} ✚" # ⓐ ⑃
ZSH_THEME_GIT_PROMPT_MODIFIED="%{$c12%} ⚡"  # ⓜ ⑁
ZSH_THEME_GIT_PROMPT_DELETED="%{$c13%} ✖" # ⓧ ⑂
ZSH_THEME_GIT_PROMPT_RENAMED="%{$fg[blue]%} ➜" # ⓡ ⑄
ZSH_THEME_GIT_PROMPT_UNMERGED="%{$fg[magenta]%} ♒" # ⓤ ⑊

ZSH_THEME_GIT_PROMPT_STATUS_SEPARATOR=""

# More symbols to choose from:
# ☀ ✹ ☄ ♆ ♀ ♁ ♐ ♇ ♈ ♉ ♚ ♛ ♜ ♝ ♞ ♟ ♠ ♣ ⚢ ⚲ ⚳ ⚴ ⚥ ⚤ ⚦ ⚒ ⚑ ⚐ ♺ ♻ ♼ ☰ ☱ ☲ ☳ ☴ ☵ ☶ ☷
# ✡ ✔ ✖ ✚ ✱ ✤ ✦ ❤ ➜ ➟ ➼ ✂ ✎ ✐ ⨀ ⨁ ⨂ ⨍ ⨎ ⨏ ⨷ ⩚ ⩛ ⩡ ⩱ ⩲ ⩵  ⩶ ⨠
# ⬅ ⬆ ⬇ ⬈ ⬉ ⬊ ⬋ ⬒ ⬓ ⬔ ⬕ ⬖ ⬗ ⬘ ⬙ ⬟  ⬤ 〒 ǀ ǁ ǂ ĭ Ť Ŧ
# ‼ • ‣

# ${user_and_pc}${dir_status}%{$reset_color%}${my_prompt}$(git_prompt_info)%{$reset_color%}%{$reset_color%}'
PROMPT='
${user_and_pc}${prompt_git_home}%{$reset_color%}${my_prompt}$(git_prompt_info)%{$reset_color%}'

RPROMPT='%{$c1%}-%?- %*%{$reset_color%}'

#  vim: set ft=zsh sw=2 et tw=0:

Ok, I can't see anything obvious. Please post the output of the following commands:
zle -l

and
which -a echo

$ which -a echo                                                                                -0- 13:17:14
echo () {
        builtin zle .beep && _zsh_highlight
}
echo: shell built-in command
/usr/bin/echo
/bin/echo
/usr/bin/X11/echo
/usr/bin/echo
/bin/echo
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active

$ zle -l
_bash_complete-word -C .complete-word _bash_completions
_bash_list-choices -C .list-choices _bash_completions
_complete_debug -C .complete-word _complete_debug
_complete_help -C .complete-word _complete_help
_complete_tag -C .complete-word _complete_tag
_correct_filename -C .complete-word _correct_filename
_correct_word -C .complete-word _correct_word
_expand_alias -C .complete-word _expand_alias
_expand_word -C .complete-word _expand_word
_history-complete-newer -C .complete-word _history_complete_word
_history-complete-older -C .complete-word _history_complete_word
_list_expansions -C .list-choices _expand_word
_most_recent_file -C .complete-word _most_recent_file
_next_tags -C .list-choices _next_tags
_read_comp -C .complete-word _read_comp
accept-and-hold
accept-and-infer-next-history
accept-and-menu-complete
accept-line
accept-line-and-down-history
accept-search
argument-base
auto-suffix-remove
auto-suffix-retain
backward-delete-char
backward-delete-word
backward-kill-line
backward-kill-word
backward-word
beep
beginning-of-buffer-or-history
beginning-of-history
beginning-of-line
beginning-of-line-hist
capitalize-word
clear-screen
complete-word
copy-prev-shell-word
copy-prev-word
copy-region-as-kill
delete-char
delete-char-or-list
delete-word
describe-key-briefly
digit-argument
down-case-word
down-history
down-line-or-search
edit-command-line
emacs-backward-word
emacs-forward-word
end-of-buffer-or-history
end-of-history
end-of-line
end-of-line-hist
end-of-list
exchange-point-and-mark
execute-last-named-cmd
execute-named-cmd
expand-cmd-path
expand-history
expand-or-complete
expand-or-complete-prefix
expand-or-complete-with-dots
expand-word
forward-word
get-line
gosmacs-transpose-chars
history-beginning-search-backward
history-beginning-search-backward-end (history-search-end)
history-beginning-search-forward
history-beginning-search-forward-end (history-search-end)
history-incremental-pattern-search-backward
history-incremental-pattern-search-forward
history-incremental-search-backward
history-incremental-search-forward
history-search-backward
history-search-forward
history-substring-search-down
history-substring-search-up
infer-next-history
insert-last-word
kill-buffer
kill-line
kill-region
kill-whole-line
kill-word
list-choices
list-expand
magic-space
menu-complete
menu-expand-or-complete
menu-select
neg-argument
orig-complete-word -C .complete-word _main_complete
orig-delete-char-or-list -C .delete-char-or-list _main_complete
orig-expand-or-complete -C .expand-or-complete _main_complete
orig-expand-or-complete-prefix -C .expand-or-complete-prefix _main_complete
orig-list-choices -C .list-choices _main_complete
orig-menu-complete -C .menu-complete _main_complete
orig-menu-expand-or-complete -C .menu-expand-or-complete _main_complete
orig-menu-select -C .menu-select _main_complete
orig-reverse-menu-complete -C .reverse-menu-complete _main_complete
overwrite-mode
pound-insert
prepend-sudo
push-input
push-line
push-line-or-edit
quote-line
quote-region
quoted-insert
read-command
recursive-edit
redisplay
redo
reset-prompt
reverse-menu-complete
self-insert (ordinary-key-press)
self-insert-unmeta
send-break
set-local-history
set-mark-command
spell-word
split-undo
transpose-chars
transpose-words
undefined-key
undo
universal-argument
up-case-word
up-history
up-line-or-search
vi-add-eol
vi-add-next
vi-backward-blank-word
vi-backward-char
vi-backward-delete-char
vi-backward-kill-word
vi-backward-word
vi-beginning-of-line
vi-caps-lock-panic
vi-change
vi-change-eol
vi-change-whole-line
vi-cmd-mode
vi-delete
vi-delete-char
vi-digit-or-beginning-of-line
vi-down-line-or-history
vi-end-of-line
vi-fetch-history
vi-find-next-char
vi-find-next-char-skip
vi-find-prev-char
vi-find-prev-char-skip
vi-first-non-blank
vi-forward-blank-word
vi-forward-blank-word-end
vi-forward-char
vi-forward-word
vi-forward-word-end
vi-goto-column
vi-goto-mark
vi-goto-mark-line
vi-history-search-backward
vi-history-search-forward
vi-indent
vi-insert
vi-insert-bol
vi-join
vi-kill-eol
vi-kill-line
vi-match-bracket
vi-open-line-above
vi-open-line-below
vi-oper-swap-case
vi-pound-insert
vi-put-after
vi-put-before
vi-quoted-insert
vi-repeat-change
vi-repeat-find
vi-repeat-search
vi-replace
vi-replace-chars
vi-rev-repeat-find
vi-rev-repeat-search
vi-set-buffer
vi-set-mark
vi-substitute
vi-swap-case
vi-undo-change
vi-unindent
vi-up-line-or-history
vi-yank
vi-yank-eol
vi-yank-whole-line
what-cursor-position
where-is
yank
yank-pop
zle-line-finish
zle-line-init
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active
echo:zle: widgets can only be called when ZLE is active

echo () {
        builtin zle .beep && _zsh_highlight
}

There you go, echo is a function which uses zle, and that is not allowed unless the function is a zle widget. Where is this defined? You can quickly test that this is the culprit by deleting the function with unfunction echo

The theme works after I unfunction echo. I have no idea where echo is defined. I certainly didn't do it, and it isn't defined in .oh-my-zsh. I wonder if its an openSUSE thing (I'm still learning the system)? Any idea on how to fix it, besides running unfunction echo in .zshrc?

Let's look at your rc files and figure out where the definition is coming from. Grab this diagnostic script, source it, run omz_diagnostic_dump, and post the output as a gist. That'll grab copies of all your relevant zsh config files. We may be able to help you track it down from there.

Here you go: https://gist.github.com/jbulchergist/c3592693d18b5b6dac63
Note: when I ran the diagnostic dump, the command complained, _omz_diagnostic_dump_one_big_text:93: command not found: infocmp
Also, I discovered that I needed to run unfunction echo after source $ZSH/oh-my-zsh.sh (line 663 in the gist). If I ran it before, it would complain unfunction: no such hash table element: echo.

Oh yeah... if basic things like echo are redefined, that's going to affect the debugging script. I'll have to do something about that.
Thanks! I'll have a look through it when I have some time.

Okay, could you also post these files too, since they're sourced from the /etc/z* or ~/.zshrc rc files?

/etc/profile
/etc/bash.bashrc
~/.zsh_aliases
~/.zsh_personal
I think you can just add them as additional files to your existing gist.

Okay, could you also post these files too, since they're sourced from the /etc/z* or ~/.zshrc rc files?

/etc/profile
/etc/bash.bashrc
~/.zsh_aliases
~/.zsh_personal

@jbulchergist:

Also, I discovered that I needed to run unfunction echo after source $ZSH/oh-my-zsh.sh (line 663 in the gist). If I ran it before, it would complain unfunction: no such hash table element: echo.

This is guarantee that /etc/* files are not to blame. But me too I'm suspectful of those ~/.zsh_* files. The diagnostic dump you posted also confirms that there are no custom files defining this. So, to reiterate, please post these files:

~/.zsh_aliases
~/.zsh_personal

@apjanke: you can use builtin echo to force using the "good" echo, and so on for other builtins. For commands, you can use command.

I'd still like to see all the files, just to get a complete picture of his configuration. (And for my benefit too, to help decide whether to add them or some source-chasing logic to the diagnostic dump function.)

Good point, can't argue with that :smile:

Okay, I've updated that omz_diagnostic_dump function with some defensive programming to handle redefinition of echo and other core commands. It should now run correctly even if they're defined weirdly, and also detect their redefinition and include that in the diagnostic dump file.
@jbulchergist, would you mind running the new version without doing unfunction echo first and see if it a) works for you and b) catches the fact that echo was redefined? (Worked for me in my test but I'd like to see confirmation in the wild.)

Sorry for the blackout. I don't have access to the .zsh_* files, but I wrote them myself and they don't define an echo function. I'll try to post them when I get a chance, but the unfunction trick works on the line after sourcing oh-my-zsh.sh. The only reason I placed it after the other two files are sources was so I could keep the file organized better. In other words, sourcing the oh-my-zsh.sh script is creating the echo function.
source $ZSH/oh-my-zsh.sh
source ~/.zsh_aliases
source ~/.zsh_personal

# The echo functio was abducted from the shell, causing issues with my theme.
# Return control back to the shell built-in.
unfunction echo

I'll also try running the new version as soon as I get a chance.

Okay, well. Now we've got a mystery I'm excited about. This is very curious.
There's nothing in OMZ itself that redefines echo, at least as far as I can see from grepping the whole source tree for "echo". Unless maybe something is doing a dynamic eval or something.
I think the next place to look is in the plugins you're loading, to see if they're pulling in anything external that redefines echo. I'll have a look when I'm back at a computer.
As a test of this hypothesis, could you turn off all your plugins and see if "echo" still gets overridden? (Just put a "which echo" after the "source oh-my-zsh.sh" in your ~/.zshrc.) And if that does it, then turn them back on one by one, to pinpoint the culprit.
Andrew

On May 22, 2015, at 8:03 PM, John Bulcher<EMAIL_ADDRESS>wrote:
Sorry for the blackout. I don't have access to the .zsh_* files, but I wrote them myself and they don't define an echo function. I'll try to post them when I get a chance, but the unfunction trick works on the line after sourcing oh-my-zsh.sh. The only reason I placed it after the other two files are sources was so I could keep the file organized better. In other words, sourcing the oh-my-zsh.sh script is creating the echo function.
source $ZSH/oh-my-zsh.sh
source ~/.zsh_aliases
source ~/.zsh_personal
The echo functio was abducted from the shell, causing issues with my theme.
Return control back to the shell built-in.
unfunction echo
I'll also try running the new version as soon as I get a chance.
—
Reply to this email directly or view it on GitHub.

Ok, at this point I'd use the big guns. Please run the command in this Troubleshooting page and post the file in a gist.

I found the culprit - it was the plugin history-substring-search. When I take it out of the plugins array, I find that I no longer needed unfuction echo.
I've gotten accustomed to ctrl+r for history searches since I bounce back and forth between bash, so loosing this plugin isn't a big deal for me personally. Still, if you want me to help fix this issue, I can run code for you and tell you the results.

I would still be curious to see that debugging output, if you have time. I'm not able to reproduce the issue (on OS X or Debian), even running the exact same set of plugins as you are. This is a really interesting problem, and figuring out the "why" could help with other stuff. (Or at least my own understanding.)
Based on what your echo got redefined as, I'll bet it's line 255 in history-substring-search.zsh that's doing the redefinition. But I don't know what it would be interacting with such that it thought echo was one of the widgets it should redefine.

OK, here's the omz-debug.log from zsh -xv &> >(tee ~/omz-debug.log 2>/dev/null)
https://gist.github.com/jbulchergist/a7d64657bdadfdd6c6ae

Thank you, John! I'm stumped, but this is going to be interesting to pore over for a while.

Ok, I have a hunch but I still don't know the full details. Basically, this bug is a combination of both the history-substring-search plugin, which implements a very outdated version of zsh-syntax-highlighting (culprit being line 273 of the plugin); and a file in /etc/profile.d directory which defines a whole bunch of aliases, one of them being alias 'beep=echo -en "\007"' that I think makes the bug appear. Look at the file /etc/profile.d/alias.bash on lines 49 to 53.
In a gist, the zsh-syntax-highlighting implementation in history-substring-search plugin defines, on line 273, a beep function:
eval 'beep() { builtin zle .beep && _zsh_highlight } ; zle -N beep'

You can see this in your gist as well.
Then, since we had a beep alias defined on that alias.bash file – and this is the part where I don't really know how it happens (maybe some obscure zsh setting or bug?) – a collision happens in which the beep function is turned into an echo function.
You can test my hypothesis in the following ways (either one of them, not both at once):

Go to lines 49 to 53 of /etc/profile.d/alias.bash and comment them (I presume you'll need to be root). If the bug is not triggered with the history-substring-search plugin enabled, I'm on the right track.

Add a canary; put in place a command to nail down the offending lines in the history-substring-search plugin. Specifically, write zsh which -a echo  on plugins/history-substring-search/history-substring-search.zsh before line 253 and after line 281 (before and after that big for).

You go try that and let me know if my hypothesis is correct.
ProTip: you can quickly reload the terminal by running exec zsh.
Cheers

Wow. Woww. Nice sleuthing!
That looks like it. I've got it reproducing on OS X 10.9 and Debian 7 using your diagnosis now. Simply adding alias 'beep=echo -en "\007"' to ~/.zshrc before sourcing OMZ with just the history-substring-search plugin is sufficient to get the breakage.
Last login: Thu May 28 19:00:48 on ttys001
echo:zle: widgets can only be called when ZLE is active
[in ~]
$ which echo
echo () {
	builtin zle .beep && _zsh_highlight
}
echo:zle: widgets can only be called when ZLE is active
[in ~]
$

I think the mechanism is pretty straightfoward, if unintuitive, if you make an assumption about zsh parsing. Consider:
eval "$clean_event() { builtin zle $event && _zsh_highlight } ; zle -N $clean_event"

Inside that string that's being eval'ed, $clean_event gets substituted with beep. Then it's evaled as its own statement. If ( is a token delimiter, then beep is in "command position" and is eligible for alias expansion, which is a simple string substitution. That would give you this.
echo -en "\007"() { builtin zle $event && _zsh_highlight } ; zle -N $clean_event }

Which looks nonsensical, but whaddya know, it's a valid syntax for defining a function, or at least zsh thinks it is. (!!!) Example with bare zsh (no ~/.zshrc):
Last login: Thu May 28 19:13:17 on ttys003
eilonwy% which echo
echo: shell built-in command
eilonwy% echo -en "\007"() { printf foobar }
eilonwy% which echo
echo () {
	printf foobar
}
eilonwy% echo
foobar%                                                                         eilonwy%
```

I... did not see that coming.

This may be part of why the core `zsh` documentation recommends using functions instead of aliases in all cases.

I think maybe it can be fixed by just sticking an explicit `function` in the function definition, so `$clean_event` is no longer in command position and is not eligible for alias expansion.

eval "function $clean_event () { builtin zle $event && _zsh_highlight } ; zle -N $clean_event"

And if that's the case, maybe we should be using explicit `function` everywhere in OMZ as part of our coding style, as defense against this alias-collision bug.

Okay, check out Complex commands in the ZSH ref manual and scroll down to "function". It lets you specify any number of words to be alternate names to a function. So the presence of () in a line basically takes everything leading up to it and redefines it as a function. (??)
Last login: Thu May 28 19:28:11 on ttys002
eilonwy% echo Here -is -a 'bunch of junk'() { printf DARNIT }
eilonwy% echo Hello
DARNIT%                                                                         eilonwy% which echo
echo () {
	printf DARNIT
}
eilonwy% which -- -is
-is () {
	printf DARNIT
}
eilonwy% which -- 'bunch of junk'
'bunch of junk' () {
	printf DARNIT
}
eilonwy%

I guess you have to be really careful with lines that have () and {...} in them. And aliasing opens up a whole can of worms. I think this means it would have broken if echo or something else important had been any of the words in the beep alias, not just the first.

Fun ZSH tricks: this one-liner does not actually print out some math tidbits.
echo for k [ 1 .. 5 ] map builtin . fact() printf "$k"

Ok, so that's what's happening... To sum up, here's the relevant section from zsh documentation:

Note also the unhelpful interaction of aliases and function definitions:
alias func='noglob func'
func() {
    echo Do something with $*
}

Because aliases are expanded in function definitions, this causes the following command to be executed:
noglob func() {
    echo Do something with $*
}

which defines noglob as well as func as functions with the body given. To avoid this, either quote the name func or use the alternative function definition form ‘function func’.

So we'll have to use the function func form:
function beep { ... }

or quote the function name:
'beep' () { ... }

Fun fact: if the alias is global (alias -g beep=echo), the function func form will still be wrong.
Another option we have, but with potential side effects, is disabling alias expansion and enabling it again after loading omz, with setopt no_aliases.

@jbulchergist we found the culprit, please update the title to history-substring-search defines function that collides with beep alias (or something similar), and try the following fix:
diff --git a/plugins/history-substring-search/history-substring-search.zsh b/plugins/history-substring-search/history-substring-search.zsh
index 22f03dd..45c0280 100644
--- a/plugins/history-substring-search/history-substring-search.zsh
+++ b/plugins/history-substring-search/history-substring-search.zsh
@@ -252,11 +252,11 @@ if [[ $+functions[_zsh_highlight] -eq 0 ]]; then
   # Override ZLE widgets to make them invoke _zsh_highlight.
   for event in ${${(f)"$(zle -la)"}:#(_*|orig-*|.run-help|.which-command)}; do
     if [[ "$widgets[$event]" == completion:* ]]; then
-      eval "zle -C orig-$event ${${${widgets[$event]}#*:}/:/ } ; $event() { builtin zle orig-$event && _zsh_highlight } ; zle -N $event"
+      eval "zle -C orig-$event ${${${widgets[$event]}#*:}/:/ } ; function $event { builtin zle orig-$event && _zsh_highlight } ; zle -N $event"
     else
       case $event in
         accept-and-menu-complete)
-          eval "$event() { builtin zle .$event && _zsh_highlight } ; zle -N $event"
+          eval "function $event { builtin zle .$event && _zsh_highlight } ; zle -N $event"
           ;;

         # The following widgets should NOT remove any previously
@@ -270,7 +270,7 @@ if [[ $+functions[_zsh_highlight] -eq 0 ]]; then
             (completion|user):*)
               ;;
             *)
-              eval "$clean_event() { builtin zle $event && _zsh_highlight } ; zle -N $clean_event"
+              eval "function $clean_event { builtin zle $event && _zsh_highlight } ; zle -N $clean_event"
               ;;
           esac
           ;;

To apply the patch, navigate to the oh-my-zsh folder, run git apply and paste the patch above, then press CTRL+D to apply the patch.

I tested your patch on OS X 10.9 and it looks good.
I also updated the Coding style guide wiki page to call for explicit use of function and link to that Aliases doco to explain why.

The patch works for me as well. Should I leave the issue open till the patch is applied to the repo?

Yes, leave it open. Let me work on a PR and I'll auto-close this one when it is merged.
Thanks!

It looks like upstream for this plugin is actually https://github.com/zsh-users/zsh-history-substring-search, and they've pulled in a more recent version of the zsh-syntax-highlighting code which fixes this. Maybe we should just pull the current version of zsh-history-substring-search in to the OMZ plugin.
Thoughts? @sunaku, @sorin-ionescu – is the HEAD version of zsh-history-substring-search good to use?

HEAD is fine.

:+1: Agreed.  The master branch of zsh-history-substring-search has been stable for a long time. :older_man:

Great.
Hey, your new work on zsh-history-substring-search is good stuff, especially the documentation on key bindings.
I wrote a little script to automatically do the update from upstream and add OMZ-specific stuff to it, and used it to update OMZ to the current HEAD. It's on my branch https://github.com/apjanke/oh-my-zsh/tree/histsubstr-update-2015_04_26; no PR for it yet.
The OMZ-specific stuff is:

An additional section to the README that explains the relationship between the OMZ-distributed plugin and your upstream repo, and includes a direct link to your repo.
We bind the up and down arrow keys in the plugin by default, instead of requiring the user to do it in ~/.zshrc. This is a matter of convenience, and matches the existing behavior for the OMZ plugin.

If history-substring-search users are okay with it, I think we should do the keymap-specific bindings (as documented in the upstream README) by default as well in the OMZ plugin layer.

I put in a PR to do the updating from upstream: #4190.

Fixed by #4190


Replace cats.implicits._ with cats.syntax.all._

Reminded by https://twitter.com/mpilquist/status/1331810540595916800, this is the new recommendation.
This should mostly be a search-and-replace.  A few will be tricky: either leave those the way they are, or we can puzzle them out.
Please base the work on the series/0.21 branch.

I would like to work on this one.

@rossabaker Thought I linked this Issue with the PR by writing in the description:
Fix: #3944
I do not know why this was not closed automatically :/

I usually say "fixes #XXXX" or "closes #xxxx".  From the docs, I'm not sure why yours didn't work.
Thanks for the PR, and thanks for calling attention to the issue.  I'm bad about coming back and closing these when the automation doesn't work.


Add context menu for each document type

Right now, our main activities (notebooks, text editors, etc.) don't have a document/activity wide context menu. We have seen in user tests that users do go looking for these things and we have context menus elsewhere (file browser). We should make sure that each doc panel activity has a context menu with common per activity actions.

Steve did implement a context menu for all documents, but at least one or two people complained that the normal browser menu didn't show up then...

See #577.

Thanks for the background. I do see the issue with the browser menus not showing up.
I am not (yet) willing to give up on the idea of context menus entirely. The reason is that during user tests, when users couldn't find what they were looking for, many of them tried to access context menus. This included (to my surprise) my 10 year old son. Before these tests I would have said that context menus were weak in their discoverability and should be used sparingly. However, based on this, I think that the users of today and become used to attempting to accomplish actions using them. Thus, while I don't think they should be the only way of accompilshing things, I do think we should offer them.

What about offering them on more specific elements rather than the entire document? The types of things we should avoid are images and links. So maybe:
Globally:

The tab for each document

Notebook

The prompt areas

Editor

Should be OK to pin to the entire document as there can't be images or links...

Terminal

Should be OK on everything...

Console:

Just the prompt areas

Another option is for us to figure out

Was this closed on accident?


High cpu when launching as a process, and parent app quits

I have an app and I added a feature to launch this 'chat' tool as a separate process and I send it stdin and parse stdout.
It works great, but if I quit my app, the chat process keeps running and the cpu goes to 80%.
It seems to get stuck in an endless loop when stdin goes down, but I haven't looked at it that carefully.

I think its cool that this project is getting used in apps! :)
The chat has by default:
Runs default in interactive and continuous mode.
  --run-once            disable continuous mode
  --no-interactive      disable interactive mode altogether (uses given prompt only)

which means that it's running an infinite whileloop waiting for stdin input.
When you close your app, your app needs to send the chat a quit message too:
Type 'quit', '/quit', 'exit', '/exit' to quit
This makes sure that the chat closes all open threads and frees the model and all the pointers pointing to the model in memory. It should send a normal c++ closing code 0 when it closes properly. Something like this if all goes well exit:entry chat (pid: 8620) exited with return value 0
For testing, you could try the run-once and no-interactive flags to make sure that the chat runs just once and closes inside your app properly.

Oh.
I just learned that in linux and mac there's a SIGHUP signal when the parent app closes but in windows its something else. I don't currently handle any of those things.
If its not too complicated, I'll try to add elementary signal handling in the future versions. I think that could be useful for people.

Thanks.  I'm now trying to see if I can link the code with my app rather than using it as a process, but I can't figure it out yet.

This hopefully works now with v0.3.0. I did add the signal handling but I haven't made a parent app so I haven't been able to test it in practice.

I just tested this on a mac using a simple python script as the parent program and looks like it works with the newest version. If I close the script, it will also close "./chat" program.


Migrate Identity ConfigureServices to 2.1: add before/after code, example with services.AddIdentity<TUser, TRole> migration

It would be helpful to have before and after code for the ConfigureServices Identity changes.
I didn't realize I should change services.AddIdentity to services.AddDefaultIdentity. That change is very important because it eventually calls AddDefaultUI which does some important setting up of the Identity Area, like adding authorization to /Identity/Account/Manage pages.
It would also be helpful to show how to migrate services.AddIdentity overloads. Specifically, I ran into a problem where I had:
services.AddIdentity<ApplicationUser, IdentityRole>(options => ...)
.AddEntityFrameworkStores<ApplicationDbContext>()
...

There's no services.AddDefaultIdentity overload for TRole. When I used the regular services.AddDefaultIdentity<ApplicationUser> I got dependency injection exceptions because RoleManager<IdentityRole> was not registered.
Looking at the default IU sample (https://github.com/aspnet/Identity/blob/master/samples/IdentitySample.DefaultUI/Startup.cs) led me to change my code to
services.AddDefaultIdentity<ApplicationUser>(options => ...)
.AddRoles<IdentityRole>() //MUST come before AddEntityFrameworkStores
.AddEntityFrameworkStores<ApplicationDbContext>()...

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 4fa0f562-bdf0-eeea-f076-0c3e05222f75
Version Independent ID: 8269d917-3b32-1607-71da-c756d4d7ad32
Content: Migrate from ASP.NET Core 2.0 to 2.1
Content Source: aspnetcore/migration/20_21.md
Product: aspnet-core
GitHub Login: @Rick-Anderson
Microsoft Alias: riande

@HaoK  Should I add a section:
Migrating AddIdentity overloads
@pholly  can you provide the before/after code?

@Rick-Anderson AddIdentity overloads may not need its own section - I updated your suggested changes in your pull request https://github.com/aspnet/Docs/pull/7562 .

So apps shouldn't need to change their identity code when they migrate to 2.1, switching to DefaultUI is not required, its probably better to just recommend adding a call to AddDefaultUI if they want to start using that as opposed to switching to the AddDefaultIdentity which isn't meant to be the replacement for AddIdentity or AddIdentityCore

AddDefaultIdentity is targeted at making the intial template code simplier, its not meant to be the replacement for AddIdentity/Core

@HaoK  Currently I have
The default 2.1 Identity UI doesn't currently provide significant new features over the 2.0 version. Replacing Identity with the RCL package is optional.
...
If you've made non-trivial changes to the template generated Identity code:
The preceding advantages may not justify converting to the the RCL version.
You can keep your ASP.NET Core 2.0 Identity code, it's fully supported.
I'll find a way to repeat or point to that section.

Yeah we just need to make it very clear that they shouldn't replace existing identity configuration with AddDefaultIdentity, it most likely will break their app.

I know this was just closed, but somewhere it should be stated that AddDefaultUI provisions the Identity Area, most notably adding authorization to /Identity/Account/Manage and /Identity/Account/LogOff. Developers migrating from Identity 2.0 MVC, where the whole ManageController had authorization, after scaffolding Identity or using the built-in compiled views will not have the new Manage pages authorized unless they manually add the auth code, as in https://github.com/aspnet/Identity/blob/c7276ce2f76312ddd7fccad6e399da96b9f6fae1/test/WebSites/Identity.DefaultUI.WebSite/NoIdentityStartup.cs#L33 .


v74.1-1.0.0 [Synapse break. Vanishment, this world!]

Changes
4 New Dark Themes!

Decimate errors in the code alongside the Wicked Lord Shingan. Let your inner fantasies go rampant with Rikka Takanashi from: "Love, Chuunibyou, and Other Delusions".
It is comfy time! Don't let feature requests stress you out, because you can now code with Nadeshiko from Yuru Camp.
A Certain Scientific RailGun go: bzzzzzzt. Zap bugs out of existence with the electromaster Mikoto Misaka.
Raccoon + Tanuki = one really cute cinnamon bun. Enjoy your time coding with Raphtalia from: "Rising of the Shield Hero."

Screens
Rikka

Nadeshiko

Mikoto

Raphtalia

Leaving this in draft for the time being, without a version bump, so others can study the changes.
I can get this ready and merged if anybody wants any of the v74 themes ASAP.
Thanks!

I'm probably just going to merge this for the time being, because I am going to completely redo how the chrome theme works & removing the firefox extension from this repository.


LESYMAP Errors - Mask in different Space & antsAverageImages

Hello Dorian,
I have been trying to use lesymap for some analysis but have run into some errors that a colleague and I have not been able to figure out. I am hoping that you can help!
The first error I received (attached) stated that the lesion files and mask are in different spaces. However, as far as I can tell, they are in the same space. Even after using 3dresample to be sure that the lesion files and mask are in the same space, I continue to receive this error. Have you encountered this problem before?
Additionally, I tried to run the analysis without my mask file and simply inputed a number into minSubjectPerVoxel and received the attached error. A colleague and I tried to troubleshoot and cannot seem to understand what it may be referring to. Have you encountered this error before?
Thank you!
Best,
Holly

Hi Holly,
First problem:
Someone reported an issue where very tiny differences in image orientation caused the issue. There is a simple fix I need to make in lesymap to circumvent the issue, but I thought this wasn't a common problem. The workaround until I implement a fix is to copy the headers from an image to your mask, as shown in this thread: https://github.com/ANTsX/ANTsR/issues/301
However, the workaround only makes sense if you are absolutely certain the two images are in the same space by checking something like antsGetOrientations() in ANTsR or PrintHeader in ANTs. You can also load the images in ITKsnap and check the header/orientations. In a normal workflow, this does not happen because the mask is derived from the images themselves. You seem to have computed the mask on your own, maybe with other software, which is a bit concerning that headers truly change (different software treat nifti headers differently).
Second problem:
I need to troubleshoot, looks like a trivial bug caused by some ANTsR change of a function. But to do that properly, I need to know which package versions are you using. Please paste the outputs of library(LESYMAP) ; sessionInfo().

Second problem (update):
The suspect ANTsR function has not changed, you seem to have installed a new LESYMAP version alongside an old ANTsR (10+ months). If that is the case, please update ANTsR to the current version.

Hi Dorian,
Thank you for your response. We have updated the ANTsR to the current version and are no longer receiving the error. Thank you!
Holly A. Aleksonis
Doctoral Student
Clinical Neuropsychology & Cognitive Neuroscience
Brains & Behavior Fellow
Georgia State University
Lab Website: http://sites.gsu.edu/kinglab/
On May 21, 2020, at 1:45 PM, dorianps <notifications<EMAIL_ADDRESS>wrote:
Second problem (update):
The suspect ANTsR function has not changed, you seem to have installed a new LESYMAP version alongside an old ANTsR (10+ months). If that is the case, please update ANTsR to the current version.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdorianps%2FLESYMAP%2Fissues%2F24%23issuecomment-632248415&data=02|01|haleksonis1%40student.gsu.edu|d3accb76aafc43870c4008d7fdaec0f5|704d822c358a47849a1649e20b75f941|0|0|637256799318159727&sdata=KE58bBTx%2BZlgSlu1aVoJ%2BpP5sOr8ArmeWfAZNt99Pug%3D&reserved=0, or unsubscribehttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAPVOE2D7GD35FYFNDXUGXD3RSVSDTANCNFSM4NG6ADFA&data=02|01|haleksonis1%40student.gsu.edu|d3accb76aafc43870c4008d7fdaec0f5|704d822c358a47849a1649e20b75f941|0|0|637256799318159727&sdata=OJpJXhWIrbrA6z1nUBZ02d2udT91LcdLg2lF%2FKp%2BK%2FQ%3D&reserved=0.

Holly, please note that you need to specify minSubjectPerVoxel=2 and not minSubjectPerVoxel="2" if you want to threshold by 2 subjects. From what I see in the code, lesymap will misinterpret a character input as a percentage, it needs to be numeric when defining exact number of subjects.


Unable to run specs after fork

Hey there.
When forking the project I'm unable to run the specs because:
LoadError:
  cannot load such file -- [...]/rspec-rails-swagger/spec/testapp/config/environment
# ./spec/rails_helper.rb:3:in `require'
# ./spec/rails_helper.rb:3:in `<top (required)>'
# ./spec/swagger_helper.rb:2:in `require'
# ./spec/swagger_helper.rb:2:in `<top (required)>'
# ./spec/requests/request_spec.rb:1:in `require'
# ./spec/requests/request_spec.rb:1:in `<top (required)>'

Is there some step that I'm missing. I have looked in the README, and there doesn't seem to be any mention about any special steps to run the specs for the project it self.
Thanks,
Emil

Hmm.. it was in the contribution doc. Never mind.


Swin-B vs CycleMLP-B on image classification

For classificaion on ImageNet-1k, the acuracy of Swin-B is 83.5, which is 0.1 higher than the proposed CycleMLP-B. But, in this paper, the authors reprot that the accuracy of Swin-B is 83.3, which is 0.1 lower than the proposed CycleMLP-B. Why are these accuracies different?

Hi,
Thanks for your interest.
We reported Swin-B (83.3) following the original Swin arxiv paper (Table 1 a): https://arxiv.org/pdf/2103.14030v1.pdf
I noticed that they updated this value in the latest version.


[ChatGPT]: fix some bug

Description

Fix: Conversation data not right when the Ask component unmounts
Fix: Socket proxy not working
Fix: fix fetch model data lag caused incorrect option https://github.com/raycast/extensions/issues/9146

Screencast

enter conversation and ask

go back and re-enter

Checklist

[x] I read the extension guidelines
[x] I read the documentation about publishing
[x] I ran npm run build and tested this distribution build in Raycast
[x] I checked that files in the assets folder are used by the extension itself
[x] I checked that assets used by the README are placed outside of the metadata folder

currently focusing on the bug after I update the dependencies


Weird rounding behaviour when scroll down playlists

I noticed when you scroll down your playlists, it looks funny since the rounded corners from the top section stay and are put against the non-rounded corners of the bottom section.
Potential fix:

See if it's possible to keep a thin bar in between the two sections when you scroll (maybe using position: sticky)
This would look best if the rounded corners were kept. You could maybe add a border-radius on the playlist parent container.

When scrolled at the top:

When scrolled down:

This is not an issue with our theme. It is an issue with the cover art extension. Feel free to make a PR.

Nevermind i see what you mean, i dont really dev this anymore but @kyrie25 might be able to get around to it, seems very easy.

Changing lines 20 in _navbar.scss
margin: 24px 12px 0 12px;
>
margin: 24px 12px 12px 12px;

Is a quick and dirty fix, though it moves everything down.

solved via e920f635c96572413583b8df078d944ae7f23002

Haha yeah I guess that works, thanks


[ 2.5pt ] - Login, Signup Views  - Frontend

Add styles for User login page and User Signup pages.

#https://github.com/tan12082001/Vehicle-Booking-App-Frontend/pull/3


How to Upload Dashboard

🐛 Summary
What's wrong? Please be specific.
I am having a hard time understanding how to create/download the dashboard. I ran the sparrow script and have the csv logs but now I do not know what to do with the aviary xml script.
Also, once I ran the sparrow script and when it was retrieving the data, I received this error message.
"WARNING: There were errors during the search process which might affect the validity of the data."
Do I need a splunk account to create the dashboard? Please assist. Thanks!
To reproduce
Steps to reproduce the behavior:

Do this
Then this

Expected behavior
What did you expect to happen that didn't?
Any helpful log output or screenshots
Paste the results here:

Add any screenshots of the problem here.

Hi,
The aviary xml script is for the Splunk dashboard and you will need Splunk in order to use it.
Thank you.

Where can I download splunk?

https://www.splunk.com/en_en/goto/Download_4_2


Fix Error in the Universities Section.

Describe the task

In the Universities listing page inside the HR module, there is a bug when we click on the Add university button,.

It is an undefined variable has been passed, is needed to be checked and fixed.

Also remove the Add university button from each university edit page and just show it on the main listing page of the university.

Thing to do

[ ] Fix the Add University button.
[ ] Also remove the Add University button from edit page.

closing as issue is fixed


Throw error when push is missing configuration

Currently, when there are no custom push adapter, default push adapter will be use even without configuration.
var server = new ParseServer({
  appId: '...',
  masterKey: '...',
  serverURL: 'http://localhost:1337/parse',
  ...moreOptions
  // Missing `push` here
});

This would allow sending push with {result: true} but in return is fail status if we look at _PushStatus. Therefore, it is better to provide error message (Missing push configuration) before trying to send the push, developer will not feel confuse and can fix the problem easier.

[Current coverage][cc-pull] is 92.11%

Merging [#2035][cc-pull] into [master][cc-base-branch] will increase coverage by 0.13%

2 files (not in diff) in src were modified. more

Misses -9
Hits +1

@@             master      #2035   diff @@
==========================================
  Files            91         91
  Lines          6449       6443     -6
  Methods        1108       1108
  Messages          0          0
  Branches       1358       1359     +1
==========================================
+ Hits           5932       5935     +3
+ Misses          517        508     -9
  Partials          0          0

Powered by Codecov. Last updated by [7e868b2...257ca88][cc-compare]
[cc-base-branch]: https://codecov.io/gh/ParsePlatform/parse-server/branch/master?src=pr
[cc-compare]: https://codecov.io/gh/ParsePlatform/parse-server/compare/7e868b2dcc98a9010ee582d0dfc1ccbd85a450f8...257ca883a53c4e88893be654989bdd80b7fd549f
[cc-pull]: https://codecov.io/gh/ParsePlatform/parse-server/pull/2035?src=pr

@Marco129 updated the pull request.


expose unsafeInterpret

There are uses of hint where giving a type hint is the only way it is possible for code to
typecheck. As an example, hakaru uses un-exposed functionality to parse Haskell code
which makes use of finally tagless interpreters, and needs the type annotations to run
https://github.com/hakaru-dev/hakaru/blob/v0.2.0/Language/Hakaru/Simplify.hs#L98

Ah! This reminds me of another hint bug I had forgotten about: we're allowed to import modules qualified, but the type annotation added by hint doesn't use qualified names, so if the type of the expression comes from a qualified module, evaluation fails with an "unknown type" error. If this pull request is merged, I'll be able to specify the qualified type myself, which is better than my current workaround.

This is already named unsafe, so I agree that it is a good idea exporting it as long as it can serve a purpose. Hiding it won't help anyone.

@zaxtax add yourself to AUTHORS too, please

I moved the function Interpreter/Unsafe, and added myself to AUTHORS.

Squash the to unsafeInterpret commits and I'll merge.

Squashed.


storing status info as return from bot framework and giving it back t…

…o the bot framework with next request

I'm sorry we cannot accept your changes. CurrentState isn't a reserved entity type in the protocol, and therefore not something we'd make generally available to all users.
Can you describe what you're trying to accomplish? Maybe there's another way to accommodate it.


Added the Sign-up page.

Short Description

Added the sign-up page for the website

Fixes Issue
Closes #14
Screenshots

Note to reviewers
this PR is not yet done i have to link the sign-up button to Dashboard page.

@AliReza1083 created a draft PR for now i still have to add the link For buttons to the dashboard page .

It is great.
The things if we do, will be awesome: 👇

Can we reduce the gap between the buttons?
Can we increase the padding-x for each buttons?

@AliReza1083  thanks for the review i'll work on it

@AliReza1083  is this okay??

LGTM, Thanks for your Contribution 🎉


Any chance this will work one day with Jellyseerr?

Do you see any chance this will work with Jellyseerr one day? Thanks for all your work. Hoping you bring it to Jellyseerr in the future.
https://github.com/Fallenbagel/jellyseerr

Pages on IMDB show "Please update to Overseerr 1.29.0+"

Thanks for looking into this!

100% fine! This is great! I was hoping it would be a super simple change. Thank you so much for your work!


python311Packages.urwid: 2.4.3 -> 2.5.3

Changelog: https://github.com/urwid/urwid/releases/tag/2.5.3
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 288996 run on x86_64-linux 1

34 packages marked as broken and skipped:

python312Packages.bpython
python312Packages.bpython.dist
python312Packages.configshell
python312Packages.configshell.dist
python312Packages.hachoir
python312Packages.hachoir.dist
python312Packages.netapp-ontap
python312Packages.netapp-ontap.dist
python312Packages.nomadnet
python312Packages.nomadnet.dist
python312Packages.pijuice
python312Packages.pijuice.dist
python312Packages.pudb
python312Packages.pudb.dist
python312Packages.pytest-pudb
python312Packages.pytest-pudb.dist
python312Packages.python-fx
python312Packages.python-fx.dist
python312Packages.python-openzwave
python312Packages.python-openzwave.dist
python312Packages.qiling
python312Packages.qiling.dist
python312Packages.recline
python312Packages.recline.dist
python312Packages.upass
python312Packages.upass.dist
python312Packages.urwid
python312Packages.urwid-readline
python312Packages.urwid-readline.dist
python312Packages.urwid.dist
python312Packages.urwidgets
python312Packages.urwidgets.dist
python312Packages.urwidtrees
python312Packages.urwidtrees.dist

4 packages failed to build:

alot
alot.dist
stig
stig.dist

70 packages built:

deltachat-cursed
deltachat-cursed.dist
hachoir (python311Packages.hachoir)
hachoir.dist (python311Packages.hachoir.dist)
khal
khal.dist
mongoaudit
mongoaudit.dist
nvmet-cli
nvmet-cli.dist
pijuice (python311Packages.pijuice)
pijuice.dist (python311Packages.pijuice.dist)
python311Packages.bpython
python311Packages.bpython.dist
python311Packages.configshell
python311Packages.configshell.dist
python311Packages.netapp-ontap
python311Packages.netapp-ontap.dist
python311Packages.nomadnet
python311Packages.nomadnet.dist
python311Packages.pudb
python311Packages.pudb.dist
python311Packages.pytest-pudb
python311Packages.pytest-pudb.dist
python311Packages.python-fx
python311Packages.python-fx.dist
python311Packages.python-openzwave
python311Packages.python-openzwave.dist
python311Packages.qiling
python311Packages.qiling.dist
python311Packages.recline
python311Packages.recline.dist
python311Packages.upass
python311Packages.upass.dist
python311Packages.urwid
python311Packages.urwid-readline
python311Packages.urwid-readline.dist
python311Packages.urwid.dist
python311Packages.urwidgets
python311Packages.urwidgets.dist
python311Packages.urwidtrees
python311Packages.urwidtrees.dist
qemu-python-utils
qemu-python-utils.dist
s-tui
s-tui.dist
scli
scriv
scriv.dist
targetcli
targetcli.dist
todoman
todoman.dist
toot
toot.dist
topydo
topydo.dist
trackma-curses
trackma-curses.dist
turses
turses.dist
urlscan
urlscan.dist
vit
vit.dist
wikicurses
wikicurses.dist
wikicurses.man
zulip-term
zulip-term.dist

I support this update, but there is a even newer version available, v2.6.0:
https://github.com/urwid/urwid/releases/tag/2.6.0

I support this update, but there is a even newer version available, v2.6.0:
https://github.com/urwid/urwid/releases/tag/2.6.0

And an even newer one now https://github.com/urwid/urwid/releases/tag/2.6.1

Result of nixpkgs-review pr 288996 run on x86_64-linux 1

4 packages failed to build:

python312Packages.qiling
python312Packages.qiling.dist
stig
stig.dist

104 packages built:

alot
alot.dist
deltachat-cursed
deltachat-cursed.dist
hachoir (python311Packages.hachoir)
hachoir.dist (python311Packages.hachoir.dist)
khal
khal.dist
mongoaudit
mongoaudit.dist
nvmet-cli
nvmet-cli.dist
pijuice (python311Packages.pijuice)
pijuice.dist (python311Packages.pijuice.dist)
python311Packages.bpython
python311Packages.bpython.dist
python311Packages.configshell
python311Packages.configshell.dist
python311Packages.netapp-ontap
python311Packages.netapp-ontap.dist
python311Packages.nomadnet
python311Packages.nomadnet.dist
python311Packages.pudb
python311Packages.pudb.dist
python311Packages.pytest-pudb
python311Packages.pytest-pudb.dist
python311Packages.python-fx
python311Packages.python-fx.dist
python311Packages.python-openzwave
python311Packages.python-openzwave.dist
python311Packages.qiling
python311Packages.qiling.dist
python311Packages.recline
python311Packages.recline.dist
python311Packages.upass
python311Packages.upass.dist
python311Packages.urwid
python311Packages.urwid-readline
python311Packages.urwid-readline.dist
python311Packages.urwid.dist
python311Packages.urwidgets
python311Packages.urwidgets.dist
python311Packages.urwidtrees
python311Packages.urwidtrees.dist
python312Packages.bpython
python312Packages.bpython.dist
python312Packages.configshell
python312Packages.configshell.dist
python312Packages.hachoir
python312Packages.hachoir.dist
python312Packages.netapp-ontap
python312Packages.netapp-ontap.dist
python312Packages.nomadnet
python312Packages.nomadnet.dist
python312Packages.pijuice
python312Packages.pijuice.dist
python312Packages.pudb
python312Packages.pudb.dist
python312Packages.pytest-pudb
python312Packages.pytest-pudb.dist
python312Packages.python-fx
python312Packages.python-fx.dist
python312Packages.python-openzwave
python312Packages.python-openzwave.dist
python312Packages.recline
python312Packages.recline.dist
python312Packages.upass
python312Packages.upass.dist
python312Packages.urwid
python312Packages.urwid-readline
python312Packages.urwid-readline.dist
python312Packages.urwid.dist
python312Packages.urwidgets
python312Packages.urwidgets.dist
python312Packages.urwidtrees
python312Packages.urwidtrees.dist
qemu-python-utils
qemu-python-utils.dist
s-tui
s-tui.dist
scli
scriv
scriv.dist
targetcli
targetcli.dist
todoman
todoman.dist
toot
toot.dist
topydo
topydo.dist
trackma-curses
trackma-curses.dist
turses
turses.dist
urlscan
urlscan.dist
vit
vit.dist
wikicurses
wikicurses.dist
wikicurses.man
zulip-term
zulip-term.dist

qiling fails as capstone and unicorn are still using distutils. stig is broken for a while already.


feat: using datepicker for expiry

Feature Request
Describe the Feature Request
We are currently allowing the user to select between simple date times and they are also hardcoded values [ 1 day, 1 week, 1 month.. ] This feature aims to add a Datepicker menu that will allow users to openly choose their expire periods.
Describe Preferred Solution

[x] Learn about Datepickers at Tailwind
[x] Implement the Tailwind Datepicker into the OfferSummary component
[x] Make sure the time is being correctly addressed for the swaps created and also after creating, when they are displayed at the swap offers
[x] Make sure the design follows the same style of Swaplace dApp

BONUS

[ ] Add the time selector as well. Or this will be addressed to a new issue later.

Additional Context

Closed by issue #335


[Feature request] Diff tool for dependency graphs (plus lockfile support)

'Just ran into a situation at work where we had to blow away our lockfile and rebuild it as part of resolving a bug in one of the dependencies.   Because we caret-prefix our dependencies in package.json, and haven't been diligent about updating, this led to ~400 modules receiving minor updates, which led to a variety of breaking changes. (Despite none of the updates being major-version updates!)
For example, some jest tests started failing due to circular import dependencies, even though none of the tests changed, nor did our version of jest change.
It's been a bit of a headache and, unfortunately, there isn't really a simple way to identify which of the module changes might have been the cause.  jest has 278 dependencies.  It could've been any one of them.
One of the things that would have helped would be a tool for visualizing the changes to our lockfile.  Basically a visual lockfile diff-tool, but for dependency graphs, with the additional ability to display subgraphs (e.g. for jest) so we could see exactly what the impact of all this on jest was.

Closing this out for the time being.  With the recent changes that put the contents of uploaded files in the URL-hash, managing lockfile contents (often 100KB+) isn't feasible at this time.
While this would be nice to have, I don't think there's enough demand for it to justify the work needed.


Fix mkdocs instant loading problem, close #2

Add support for instant loading - MkDocs Material.
Thanks to mkdocs_charts_plugin plugin that supports that feature.
@timvink
Closes issue:
#2 - Does not work with material navigation.instant

Happy to read you could copy a solution! Cool plugin also.


Decide on a name and subdomain

We're currently calling this project "busICT" because it's easy and plays with the MakeICT and devICT names. I'd like to investigate better name options.
We should come up with a short list of possible names then poll the Open Wichita group. Names should communicate the vision/goal of the project which is to *"make it easier for Wichitans to use the bus system"

Also we need to decide on the *.openwichita.com domain to use.

BrightBus, BasicBus. are my personal favorites.

Considering this isn't a business and that, in my opinion, the strongest candidate for brand recognition here is "Open Wichita" my vote is to keep project names very simple and directly tied to the Open Wichita "brand."
So instead of coming up with a clever name, maybe just transit.openwichita.com - Open Wichita Transit. This can also illustrate how we are piggybacking and (hopefully) enhancing the already existing city services. It's currently called "Wichita Transit" so "Open Wichita Transit" could be a nice way to tie to the existing service. On the other hand, it could be confusing. Thoughts?


Add p2p support

Since dadi-p2p has been open-sourced several  months. It's time to let overlaybd support lazy loading in dadi-p2p.

i have added this part of configuration #108