Add Support for Peewee 3.6.4

What current issue(s) does this address, or what feature is it adding?
Attempting to add support for Peewee 3.6.4 (latest)
See issue #262
How did you solve this problem?
Trial and error. The issue is not 100% solved.
How did you make sure your solution works?
I ran several tests including:
test_wallet.py
test_KeyPair.py
test_user_wallet.py
test_create_user_wallet.py (fails 1 test)
^Please note this last test still fails and I cannot figure out why at the moment. See below for traceback.
Are there any special changes in the code that we should be aware of?
Adds implementation of BlobField, which is new in newer versions of peewee
Please check the following, if applicable:

[x] Did you add any tests?
[ ] Did you run make lint?
[ ] Did you run make test?
[x] Are you making a PR to a feature branch or development rather than master?
[x] Did you add an entry to CHANGELOG.rst? (if not, please do)

Here is the traceback for the 1 failed test.
======================================================================
ERROR: test_7_change_password (neo.Implementations.Wallets.peewee.test_create_user_wallet.UserWalletTestCase)
Traceback (most recent call last):
File "/mnt/c/users/seagrave/neo-python-peewee37/venv/lib/python3.7/site-packages/peewee.py", line 2640, in execute_sql
cursor.execute(sql, params or ())
sqlite3.InterfaceError: Error binding parameter 1 - probably unsupported type.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/mnt/c/users/seagrave/neo-python-peewee37/neo/Implementations/Wallets/peewee/test_create_user_wallet.py", line 98, in test_7_change_password
wallet.ChangePassword('awesomepassword', 'awesomepassword2')
File "/mnt/c/users/seagrave/neo-python-peewee37/neo/Wallets/Wallet.py", line 224, in ChangePassword
self.SaveStoredData("PasswordHash", password_key)
File "/mnt/c/users/seagrave/neo-python-peewee37/neo/Implementations/Wallets/peewee/UserWallet.py", line 365, in SaveStoredData
k.save()
File "/mnt/c/users/seagrave/neo-python-peewee37/venv/lib/python3.7/site-packages/peewee.py", line 5572, in save
rows = self.update(**field_dict).where(self._pk_expr()).execute()
File "/mnt/c/users/seagrave/neo-python-peewee37/venv/lib/python3.7/site-packages/peewee.py", line 1574, in inner
return method(self, database, *args, **kwargs)
File "/mnt/c/users/seagrave/neo-python-peewee37/venv/lib/python3.7/site-packages/peewee.py", line 1645, in execute
return self._execute(database)
File "/mnt/c/users/seagrave/neo-python-peewee37/venv/lib/python3.7/site-packages/peewee.py", line 2063, in _execute
cursor = database.execute(self)
File "/mnt/c/users/seagrave/neo-python-peewee37/venv/lib/python3.7/site-packages/peewee.py", line 2653, in execute
return self.execute_sql(sql, params, commit=commit)
File "/mnt/c/users/seagrave/neo-python-peewee37/venv/lib/python3.7/site-packages/peewee.py", line 2647, in execute_sql
self.commit()
File "/mnt/c/users/seagrave/neo-python-peewee37/venv/lib/python3.7/site-packages/peewee.py", line 2438, in exit
reraise(new_type, new_type(*exc_args), traceback)
File "/mnt/c/users/seagrave/neo-python-peewee37/venv/lib/python3.7/site-packages/peewee.py", line 177, in reraise
raise value.with_traceback(tb)
File "/mnt/c/users/seagrave/neo-python-peewee37/venv/lib/python3.7/site-packages/peewee.py", line 2640, in execute_sql
cursor.execute(sql, params or ())
peewee.InterfaceError: Error binding parameter 1 - probably unsupported type.

Ran 7 tests in 24.027s
FAILED (errors=1)

With this latest version, I still have a failure for the test 7 (change password) in test_create_user_wallet. It seems like UserWallet.py is unable to complete SaveStoredData. I think this might be due to the type of database created by using Blobfield and how SaveStoredData interacts with it. This reasoning comes from testing with my privatenet. When I run my privatenet, I am able to create a new wallet. However, if I close and then try to reopen that wallet, I get this message "Could not open wallet: This wallet is currently vulnerable. Please execute the "reencrypt_wallet.py" script on this wallet before continuing". Again, I think it is a problem with SaveStoredData.

At this point, I only receive one error during the travis run:
======================================================================
ERROR: test_7_delete_holds (contrib.nex.test_withdraw.WithdrawWalletTestCase)
Traceback (most recent call last):
File "/home/travis/build/CityOfZion/neo-python/neo/contrib/nex/test_withdraw.py", line 176, in test_7_delete_holds
self.assertEqual(wallet._holds, [])
File "/opt/python/3.6.3/lib/python3.6/unittest/case.py", line 829, in assertEqual
assertion_func(first, second, msg=msg)
File "/opt/python/3.6.3/lib/python3.6/unittest/case.py", line 819, in _baseAssertEqual
if not first == second:
File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/peewee.py", line 714, in eq
return self._hash == other._hash
AttributeError: 'list' object has no attribute '_hash'
Not sure, how this is related to peewee yet, or if this test is required to pass.
One thing to note: when I try to create a new wallet, I get an error:
[E 180727 02:39:33 UserWallet:93] Could not build database unable to open database file /Wallet2.wallet
Exception creating wallet: unable to open database file

Update to the last comment: I couldn't build the database because I was giving an incomplete path. Now, I am able to create a new wallet but I get this warning:
[W 180727 03:34:08 Wallet:675] Could not process ::: UNIQUE constraint failed: key.Name
And, when I close and try to reopen the wallet, I get this error:
"Could not open wallet: This wallet is currently vulnerable. Please execute the "reencrypt_wallet.py" script on this wallet before continuing"

When I run using Python 3.6.5, I did not received the previously mentioned warning. However, I still get the same error:
"Could not open wallet: This wallet is currently vulnerable. Please execute the "reencrypt_wallet.py" script on this wallet before continuing"

I figured out why the Migration issue was happening. When the MigrationState was saved as '1', and subsequently recalled using LoadStoredData, it returned b'1'. So, the string was converted into a byte. Converting it back to a string as a part of LoadStoredData solves this problem but then I am unable to match the inputted password hash to the saved password key, so I am still unable to open the new wallet after creating it.
@metachris @localhuman
BLUF: Do you know if there is a way to force peewee to return a stored value from the UserWallet.py code to give a similar result to the legacy version of CharField?
Background:
Since, the original problem with peewee 3 vs 2 stems from this difference:
_CharField calling _StringField in peewee 3.6.4
class _StringField(Field):
    def adapt(self, value):
        if isinstance(value, text_type):
            return value
        elif isinstance(value, bytes_type):
            return value.decode('utf-8')
        return text_type(value)

    def __add__(self, other): return self.concat(other)
    def __radd__(self, other): return other.concat(self)

class CharField(_StringField):

vs _CharField calling _StringField  calling coerce_to_unicode in peewee 2.6.2
def coerce_to_unicode(s, encoding='utf-8'):
    if isinstance(s, unicode_type):
        return s
    elif isinstance(s, string_type):
        try:
            return s.decode(encoding)
        except UnicodeDecodeError:
            return s
    return unicode_type(s)

class _StringField(Field):
    def coerce(self, value):
        return coerce_to_unicode(value or '')

    def __add__(self, other):
        return self.concat(other)
    def __radd__(self, other):
        return other.concat(self)

class CharField(_StringField):

You can see that in peewee3.6.4, there is no option to force the return of value (or 's') in case of a UnicodeDecodeError.
The peewee maintainer said that I should just use a BlobField (new in peewee 3), but I am running into more/different problems in this case. I feel like BlobField is converting the saved data into unusable formats.

@metachris I understand that travis is still failing, but I can't figure out if it is the test itself or if there is something actually wrong. That being said, please take a look at what I have. I tested it on my privatenet and it seems to be working (finally!).

Coverage decreased (-0.06%) to 80.282% when pulling fe74aa9021f110045f6946faa8ea68554ab68d17 on jseagrave21:patch-4 into 06d32201e0f3f9cd9ca6cf137f45907fc45ba3a3 on CityOfZion:development.

@metachris  I updated NEX's test_wallet.py without disturbing the test itself. I believe this version is better and all tests pass. Let me know what you think.

Great job!


captured span.sync attribute for ES restclient spans plugins

partially covers #3168
What does this PR do?
Set the the span.sync = false for asynchronous spans of ES restclient plugins
Checklist

[x] This is an enhancement of existing features, or a new feature in existing plugins

[x] I have updated CHANGELOG.asciidoc
[x] I have added tests that prove my fix is effective or that my feature works
[x] Added an API method or config option? Document in which version this will be introduced
[x] I have made corresponding changes to the documentation

@Mergifyio update

@Mergifyio queue default

@elasticmachine run elasticsearch-ci/docs

https://github.com/Mergifyio queue default

@elasticmachine run elasticsearch-ci/docs


Fix(apps)/not update app config

Description
I encountered the same issue detailed in #183. While executing zcli apps:update within a GitHub action, the CLI overwrote the app configurations set by my client. I believe a straightforward flag to prevent the execution of the API endpoint /api/${product}/apps/installations/${installation_id}.json would suffice in scenarios like mine. However, I'm open to further discussion.

Do NOT write here! This section will be filled in by GitHub Action
automatically. If you don't want this, either remove the markers or write
outside the fences.

Resolves #183
Checklist

[ ] :guardsman: includes new unit and functional tests

Any progress on this?


added my name to guestbook.md

Thanks for the course. Well done!

Thank you! I am happy you enjoyed it :)


moving link to SPServer TOC

https://github.com/MicrosoftDocs/office-docs-powershell/issues/5288

Docs Build status updates of commit 6222e9a:
:clock10: Full build: the average full build time is 12 min(s) 54 sec(s), this is based on the last 1000 build(s) for this repository.

Docs Build status updates of commit 6222e9a:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

sharepoint/sharepoint-ps/sharepoint-online/sharepoint-online.md
:white_check_mark:Succeeded
View (sharepoint-ps)

sharepoint/sharepoint-ps/sharepoint-server/sharepoint-server.md
:white_check_mark:Succeeded
View (sharepoint-ps)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


gym.error.UnregisteredEnv

Hello,
I have been running your code. I got an error that the environment is not registered properly within the openai Gym:
/home/yantian/Research/venv/bin/python /home/yantian/Research/hindsight-experience-replay-with-demo/train.py seed=1 task_name=NeedlePick-v0
workspace: /home/yantian/Research/hindsight-experience-replay-with-demo/exp_local/NeedlePick-v0/HerDemo/HerDemo_seed=1,task_name=NeedlePick-v0
Error executing job with overrides: ['seed=1', 'task_name=NeedlePick-v0']
Traceback (most recent call last):
File "/home/yantian/Research/venv/lib/python3.8/site-packages/gym/envs/registration.py", line 158, in spec
return self.env_specs[id]
KeyError: 'NeedlePick-v0'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/yantian/Research/hindsight-experience-replay-with-demo/train.py", line 174, in main
exp = Experiment(cfg)
File "/home/yantian/Research/hindsight-experience-replay-with-demo/train.py", line 22, in init
self.setup(cfg)
File "/home/yantian/Research/hindsight-experience-replay-with-demo/train.py", line 41, in setup
self.train_env = gym.make(self.cfg.task_name)
File "/home/yantian/Research/venv/lib/python3.8/site-packages/gym/envs/registration.py", line 235, in make
return registry.make(id, **kwargs)
File "/home/yantian/Research/venv/lib/python3.8/site-packages/gym/envs/registration.py", line 128, in make
spec = self.spec(path)
File "/home/yantian/Research/venv/lib/python3.8/site-packages/gym/envs/registration.py", line 203, in spec
raise error.UnregisteredEnv("No registered env with id: {}".format(id))
gym.error.UnregisteredEnv: No registered env with id: NeedlePick-v0
Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
Thanks a lot!
Yantian

Hi Yantian,
Thanks for your information. Have you tried to register the SurRoL environment by following this step:
# directory: anaconda3/envs/dex/lib/python3.8/site-packages/
# Then add one line of code in gym/gym/envs/__init__.py to register SurRoL tasks:

import surrol.gym

Meanwhile, I'd like to suggest you look at the instructions in the DEX repo where more detailed descriptions will be given.
Please let me know if any further questions.

Thank you! I will take a look and get back to you soon.

Your solution works. Thank you!


Print observations as they come.

With this change text reporter will report observations as they come, so it will greatly reduce memory consumption.

LGTM, see one minor comment, you'll have to rebase and deal with merge conflicts though.


Simplify usage of make

All variants of Chrome binaries are already defined in makefile, no need to change it.
Developers should be concerned only of two make goals, make js and make browserpass

/cc @jayme-github - this is what mostly needed from Docker to support
make release doesn't make sense to be used for regular people, as without my *.pem file extension will get a different ID and host app will not work with it.

Dependencies will always be fetched automatically.
Package maintainers should not be using make anymore as browserpass-src.tar.gz (made with make tarball) contains everything.

@qbit, @chrboe - let me know if this change makes sense to you, as I'm still not very comfortable with makefiles.

Looks good to me. I'd like to see all get defined so we could just run make (gmake in my case) and have it do TheRightThing™. It would also be super cool if we could only run dep or yarn if the node_modules/vendor directories were missing.. but I am not sure how that could be done.

I agree with @qbit, an all target would be nice.
Regarding

It would also be super cool if we could only run dep or yarn if the node_modules/vendor directories were missing

yarn actually checks this by itself, so it effectively no-ops when there's nothing to do. A short test confirms this behaviour:
$ yarn
yarn install v1.0.2
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
Done in 4.85s.

$ yarn
yarn install v1.0.2
[1/4] Resolving packages...
success Already up-to-date.
Done in 0.42s.

I'm not sure about dep, but I'd expect the same behaviour there.
Of course, if we absolutely wanted to do the check (saving 0.42s), we could do [[ ! -d "node_modules" ]] && yarn, but that wouldn't actually update the dependencies after a git pull, for instance.

dep ensure is quite fast when dependencies have already been pulled. Checking for existence of node_modules and vendor directories is indeed a bad idea.

Regarding the all target: it might actually be as simple as doing something like this
diff --git a/makefile b/makefile
index ee347a0..53e74b4 100644
--- a/makefile
+++ b/makefile
@@ -3,8 +3,7 @@ CHROME := $(shell which google-chrome 2>/dev/null || which google-chrome-stable
 PEM := $(shell find . -name "chrome-browserpass.pem")
 JS_OUTPUT := chrome/script.js chrome/inject.js

-.PHONY: empty
-empty:
+all: browserpass js

 .PHONY: chrome
 chrome:

Thanks to all of you!
The new commit adds the following:

Added all goal so that running make just works
Merged the firefox goal into js, so that js prepares both Chrome and Firefox unpacked extensions at once (also extended documentation in CONTRIBUTING.md about this)
Renamed chrome to crx make goal

Running dep / yarn only if there is no vendor / node_modules is a bad idea because from time to time I will be updating dependencies, and when I do so and commit Gopkg.lock / yarn.lock, you want to also update the dependencies on your side. Running dep / yarn does exactly that, when there are no changes to the lock files, they do nothing, when there are changes to the lock files, they pull new dependencies.
Let me know if you think this change is ready to be merged.


fix: Update accountCreateEmailSession to accountCreateEmailPasswordSession

What does this PR do?
Fixes the error Error accountCreateEmailSession is not a function when running appwrite login
Test Plan
Tested logging into self-hosted appwrite instance successfully.
Related PRs and Issues
(If this PR is related to any other PR or resolves any issue or related to any issue link all related PR and issues here.)
Issue #116
Have you read the Contributing Guidelines on issues?
Yes

Closing in favor of PR in sdk-generator repo


Fixed methods for getting todays results

Per a spatie issue https://github.com/spatie/laravel-analytics/issues/361#issuecomment-630431898, you can see that when retrieving todays results, the period set should be 0, not 1.

As discussed on discord not an issue.


Fix #9008 - Add Airflow callback, introduce Workflow Status Mixin and remove retries

Describe your changes :

Fix #9008

Pipelines won't retry anymore.
Simplify status logic by introducing a mixin
Reorganize managed APIs modules
Moved success computation on the ingestion workflow out of the stop call, otherwise it was being marked as a success even if the pipeline failed.
Handle failure updates in the Workflow class directly, not on the Airflow side. We want the workflows to be self-managed as much as possible (although we added the callback to be prepared for any big fire on Airflow's side that we cannot handle)

Type of change :

[x] Bug fix
[x] Improvement
[x] New feature
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation

Frontend Preview (Screenshots) :
For frontend related change, please link screenshots of your changes preview! Optional for backend related changes.

Checklist:

[x] I have read the CONTRIBUTING document.
[ ] I have commented on my code, particularly in hard-to-understand areas.
[ ] I have added tests that prove my fix is effective or that my feature works.
[x] All new and existing tests passed.

Reviewers

can we make sure retries are also removed for the profiler, where it can get expensive? I've seen few times one or other runtime exception triggers the whole workflow retry

can we make sure retries are also removed for the profiler, where it can get expensive? I've seen few times one or other runtime exception triggers the whole workflow retry

This removes the retries from all workflows. It's the only place where we create the Task


Propagate actual error message upon staging failure

Give the user some more context upon staging failure

@georgethebeatle Thanks for your contribution.  Could you make your membership of cloudfoundry-incubator public? Thanks.

Should be public now

Could you please address the comment above? Once that is addressed, we believe we can merge this.

It turned out that this proposal is merged by the Diego team with another commit. Closing.


Improve TableGroup Rendering for Special Characters in Names

The current implementation of TableGroup rendering in pydbml does not wrap the TableGroup name in double quotes. This can lead to issues when using special characters like dashes (-) in TableGroup names.
For example:
TableGroup Unanimate {
    "books"
    "countries"
}

TableGroup mygroup-1 { // dbdocs validate error!
    "books"
    "countries"
}

If the TableGroup name contains a dash, it may result in syntax errors or unexpected behavior.
Proposed Improvement:
Update the TableGroup rendering logic to wrap TableGroup names in double quotes. This ensures compatibility with special characters and aligns with DBML’s syntax flexibility.
Proposed change in table_group.py:
result += f'TableGroup "{model.name}" {{\n'

With this change, a TableGroup with a name like mygroup-1 would be rendered as:
TableGroup "mygroup-1" {
    "products"
    "orders"
}

Benefits:

Enables the use of special characters like -, etc., in TableGroup names.
Improves compatibility and usability when naming TableGroups.
Avoids potential DBML syntax issues caused by unquoted names.

Additional Note:
If the maintainers agree with this improvement, I am happy to contribute a pull request with the necessary code changes and test updates.
Let me know your thoughts!

Oh, and and if you wish, feel free to create a pull requests with this change, as you suggested. I still have to update the syntax compliance to the latest version anyway : }


Feature Request: Support for using emoji or custom icons

Hi there, it would be great if we had the possibility to change the default icon for the Wordpress BOT.
Apart from the default integrations w/ custom post types and so on, we've also made use of it for notifying us in case of errors w/ API calls and caching issues etc. We'd love to be able to access the icon-url or icon-emoji params on sending messages using $slack_plugin method.
Thanks a bunch!

:+1:


Fix PHP8.2 str_split function returns empty arrays for empty strings

Changes description
In PHP 8.2, the str_split function will returns empty arrays for empty strings.
We can use mb_str_split() instead
Checklist
Please check if your PR fulfills the following specifications:

[ ] Tests for the changes have been added
[ ] Docs have been added/updated

References

Closes #N/A

Hi
Here is some documentation to explain this PR: https://php.watch/versions/8.2/str_split-empty-string-empty-array

I updated test to run on PHP 8.2. Could you rebase your PR to see if it pass PHP 8.2 ?


Get this branch to the current state to avoid more severe merge conflicts later

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

164 of 225 (72.89%)  changed or added relevant lines in 48 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+10.2%) to 57.344%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

packages/Liquid-UI.package/LQAnswerPanel.class/instance/choiceAt.put..st
0
1
0.0%

packages/Liquid-UI.package/LQPollDraftBuilder.class/instance/maxNumberOfChoices..st
0
1
0.0%

packages/Liquid-UI.package/LQPollDraftBuilder.class/instance/resetUI.st
0
1
0.0%

packages/Liquid-UI.package/LQPollDraftBuilder.class/instance/runPoll.st
0
1
0.0%

packages/Liquid-UI.package/LQVisualizationControls.class/instance/buildWith..st
0
1
0.0%

packages/Liquid-UI.package/LQVisualizationView.class/instance/visualizeResultsFor..st
0
1
0.0%

packages/Liquid-UI.package/LQPollDraftBuilder.class/instance/choiceList..st
0
2
0.0%

packages/Liquid-UI.package/LQVisualizationControls.class/instance/initialize.st
0
2
0.0%

packages/Liquid-UI.package/LQVisualizationControls.class/instance/questionListFrame..st
0
2
0.0%

packages/Liquid-Core.package/LQChoice.class/instance/excludedGroups.st
0
3
0.0%

Totals

Change from base Build<PHONE_NUMBER>:
10.2%

Covered Lines:
1101

Relevant Lines:
1920

💛  - Coveralls


Make PipelineData helpers collect rawstreams

Description
Try out having pipelinedata helpers collect rawstreams before they apply functions. This fixes up str trim for me, and might be worth a try for #968 and similar.
cc @fdncred
Tests
Make sure you've run and fixed any issues with these commands:

[x] cargo fmt --all -- --check to check standard code formatting (cargo fmt --all applies these changes)
[x] cargo clippy --all --all-features -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect to check that you're using the standard code style
[x] cargo build; cargo test --all --all-features to check that all the tests pass

@jntrnr This issue sounds related: https://github.com/nushell/engine-q/issues/911

yes, this fixes #968 for me. thanks!

Cool, since this seems to be an improvement, I'll go ahead and land.


TypeError: '<' not supported between instances of 'NoneType' and 'int'

Bug 💥
Occurs here and there in the HF-Space.
It looks like there are some layout sections that do not get into text ordering which should not happen.
Expected behavior  🧮
No TypeError
Screenshots 🖼
NN
Desktop (please complete the following information, if any other than the one in the install requirements):
NN
Additional context 🧬
NN

Closing after having observed that bug was due to code error in HF-space.


Verification error for release 2.6.1

We are using Posthresql v14. I encountered the verification error as below
sh-5.2$ ./backup.sh -s -v all
Verifying backup ...
Settings:

Database: postgres=zeva-spilo-test:5432/zeva
Backup file: /backups/2023-04-21/zeva-spilo-test-zeva_2023-04-21_15-22-37.sql.gz

Already clean ...
waiting for server to start.................
The server started in 0h:0m:21s.
Restoring from backup ...
waiting for server to shut down.... done
server stopped
Cleaning up ...
[!!ERROR!!] - Backup verification failed: /backups/2023-04-21/zeva-spilo-test-zeva_2023-04-21_15-22-37.sql.gz
The following issues were encountered during backup verification;
Restoring '/backups/2023-04-21/zeva-spilo-test-zeva_2023-04-21_15-22-37.sql.gz' to '<IP_ADDRESS>/zeva' ...
ERROR:  could not open extension control file "/usr/share/pgsql/extension/pg_stat_kcache.control": No such file or directory
Restore failed.
Elapsed time: 0h:0m:26s - Status Code: 3

@kuanfandevops, This looks like an extension you've added to your Postgres servers, you'll need to add the same extension to the backup server image.  I suspect you added it to the previous version of your backup-container image if you were not getting this error message before.

The pg_stat_kcache extension requires pg_stat_kcache package installed in system, but Fedora doesn't have it. I tried to build it from source pg_stat_kcache, it wasn't successful. I'll try to skip the verification step and directly try the store.
Close this issue.


[elasticinframetrics]Fix failing test due to upgrade to v0.105.0

This is a minimal fix for the current failures happening in main. Creating this since https://github.com/elastic/opentelemetry-collector-components/pull/38 had concerns about backward compatibility.

I think we should go with https://github.com/elastic/opentelemetry-collector-components/pull/38
This fix seems not to solve the linter issue yet:
Error: elasticinframetricsprocessor/factory.go:50:6: SA1019: processor.CreateSettings is deprecated: [v0.103.0] Use processor.Settings instead. (staticcheck)

Wdyt?

I think we should go with https://github.com/elastic/opentelemetry-collector-components/pull/38

+1, the only reason would be the backward compatibility. I am not sure if that would be a concern but I am not familiar with the details of our release distro.

Closing this as there is agreement to go with #38


Add --access option

This option allows the command (or block of code) be executed when watching files were accessed (readed).
Resolve #55

Is there any performance penalty detecting file read events?
Another problem may be Windows, where storing files last access time is turned off by default to speed up the system:
Some systems also provide options to disable access time updating altogether. In Windows, starting with Vista, file access time updating is disabled by default.[6]

Source: https://en.wikipedia.org/wiki/MAC_times
The functionality is really cool, since not all filewatchers have it. The Readme file should have a section that tells that that for Windows users, storing files last access time have to be turned on.
Consider renaming the -access option --detect-filereads.

Is there any performance penalty detecting file read events?

Insignificant.

Another problem may be Windows, where storing files last access time is turned off by default to speed up the system:

Yes, as I wrote here.

The functionality is really cool, since not all filewatchers have it. The Readme file should have a section that tells that that for Windows users, storing files last access time have to be turned on.

But what about:

Having no platform specific code or native C code, is what makes filewatcher robust. Maybe this feature could be published as a separate gem that monkeypatches or extends filewatcher?

(source)
?

Consider renaming the -access option --detect-filereads.

Why? It's longer, and even in Windows this event calls access, doesn't it?

Yes, I remember now. I guess this feature probably is of interest only for a few unix and mac developers.

Suppressed by external plugin (still doesn't work in Windows due to NTFS restrictions): https://github.com/filewatcher/filewatcher-access
Let's switch to Filewatcher v2 flexible ecosystem!


Upstream updates

🗣 Description
This PR pulls in the latest upstream to bring this repository into the future.

💭 Motivation and Context
This repo is consuming apb dispatches without the ability to process them. Also, it is out of date.

🧪 Testing
Automated tests pass.

📷 Screenshots (if appropriate)
🚥 Types of Changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (causes existing functionality to change)

✅ Checklist

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[x] All new and existing tests passed.

[ ] ⚠ Let's get some branch protection all up in here.

This looks great. Thanks for waiting while I did a little more modernization.

Thanks for taking care of that!


Tab names do not retain their white font highlight.

Clicking on a tab name, highlights the tab that you are using. For example, if you click on "ALL CHANNELS" it is highlighted in white font:

However, once you click on something within that tab, the highlight for the tab name disappears. For example, once you click on a channel, the "ALL CHANNELS" name is no longer highlighted in white. This can be confusing (especially for new users) since they can't tell which of the three tabs they are currently viewing.

The above example is when using dark theme. The same issue occurs (except with opposite colors) when using light theme.
This issue occurs on Windows 10, Linux and macOS.

Changed now, thanks!


Remove Shipping and Taxes

This PR removes x_amount_shipping and x_amount_taxes as parameters.
Attn:
@girasquid @aprofeit @andrewpaliga

:+1:

We're absolutely sure these are always 0 when we send it to gateways? Also, we'll wait for Paliga to send an email I assume?

@aprofeit : I'll ship it when I get the :green_apple: for adding the billing address as well.

:+1:  -- we need to wait until March 7th to ship this.

I've been thinking about how to implement a versioned HPSDK so we don't always have to email updates, but it's still just a thought.

Thanks!
On Mon, Mar 7, 2016 at 12:39 PM Kae Bagg<EMAIL_ADDRESS>wrote:

Closed #195 https://github.com/activemerchant/offsite_payments/pull/195
via 9c30085
https://github.com/activemerchant/offsite_payments/commit/9c30085120eb623b917a36e3b48b49460a2e45ec
.
—
Reply to this email directly or view it on GitHub
https://github.com/activemerchant/offsite_payments/pull/195#event-580569747
.


register_models() plugin hook

#53

TODO:

[ ] Get continue mode working again
[x] Get logging to the database working again
[ ] Write documentation
[x] Figure out how to handle setting the OpenAI API key
[x] Demonstrate the design works with a PaLM 2 plugin prototype, refs #20
[ ] Prototype of functions support to further validate the design
[x] Get all the tests passing again

For continue mode, it's all about building the prompt.
Here's the current implementation:
https://github.com/simonw/llm/blob/9190051048be90df2761c3c94589859d01006467/llm/cli.py#L176-L184
The key bit of get_history() is this:
https://github.com/simonw/llm/blob/9190051048be90df2761c3c94589859d01006467/llm/cli.py#L552-L555
So maybe this is all about building a prompt an alternative way, perhaps like this:
# History is a list of logged messages:
prompt = Prompt.from_history(history)

OpenAI chat models work by building up that messages= array, but other models could potentially also support conversations through custom prompt assembly - by putting together a User: X\nAssistant: Y\nUser: Z\nAssistant: string for example.

It looks like PaLM2 has a separate chat mode, which is mainly documented through code examples:
https://github.com/GoogleCloudPlatform/python-docs-samples/blob/main/generative_ai/chat.py
https://console.cloud.google.com/vertex-ai/publishers/google/model-garden/text-bison?project=cloud-vision-ocr-382418

Untangling chat mode for PaLM 2 is a bigger job than I want to take on right now - more notes here: https://github.com/simonw/llm/issues/20#issuecomment-1607754615

I'm not happy with how streaming works yet.
I think models should have the following methods:

.prompt(...) - run a prompt and return the response
.stream(...) - run a prompt and return an object that can be used to iterate over a streaming response
.chain(...) - run a prompt with the ability for it to trigger additional prompt executions (needed for things like OpenAI functions). Returns an iterator over those prompt responses.
.chain_stream(...) - same again, but each returned object can also be iterated over to obtain tokens from a stream.

I was contemplating if a Model even needs to be a class - could I instead define it purely in terms of functions?
Then I remembered that there are some LLMs that run locally that have significant startup costs (loading data into memory/the GPU) - which is a good fit for a class, because that way they can be loaded once and then reused for subsequent calls to .prompt() and suchlike.

Copying in some miscellaneous notes I made on the API design:
The streaming vs not streaming thing still feels a bit ugly.
The key problem I'm trying to solve is providing a generic interface to an LLM that will enable the following:

CLI access through the same tool
Comparison prompts across multiple LLMs
Unified storage of prompts in SQLite
Web interface for all of them
Unified prompt template interface
The continue conversation feature
Basic secret key management

I want the simplest possible API to implement in order to solve all of these - adding new models should be as easy as possible.
So maybe two methods: prompt(...) and stream(...)
Perhaps have a Model and StreamingModel(Model) class - models can subclass either, if you subclass StreamingModel then you have to implement both methods.
If there is ever a model that only does streaming the subclass can still implement prompt() and have that call stream().
Maybe Prompt and Response could be inner classes on the Model subclass:
class VertexModel(Model):
    class Options(Model.Options):
        temperature: float
    class Response:
        ..

Prompt andResponse chould be responsible for turning themselves into loggable records.
It would be good to have a Python library utility mechanism for this logging, since then users could log prompts that their app is using easily.
Maybe it's the Model that manages logging since that can see both prompt and response?
Logging should be off by default but easy to opt into, maybe by providing a Log instance or callback?
A Record dataclass might be good for representing that. And a llm.log.Database class that can write those to SQLite.
So the internals documentation ends up covering these concepts:

Model
Template
Prompt
Response
Result
Logger?
Key
alias

For OpenAI functions I could introduce a .sequence() method which can run more then one prompt (or .multi() or .chain()).
So the API for a model is
response = model.prompt(...)

Or
for chunk in model.stream(...)

for response in model.chain(...):
    print(response.text())

Maybe this:
for response in model.chain_stream(...):
    for token in response:
        print(token, end="")

I'm going with response.text() and not response.text because you cannot await a property.
What abstraction could I build so other chains can be easily constructed? Like there's a thing where the user gets to define a function that takes a response and decides if there should be another prompt (which the functions stuff can then be built on):
def next(response, prompt):
    return None # or str or Prompt

model.chain(next)

model.chain(lambda response: "fact check this: $input", once=True)

So once=True means only do the chain function once. Or should they be the default and use repeat=True to keep it going?

I'm going to try a bit of documentation-driven development here.

Here's that first draft of the internals documentation - next step, actually implement what's described there: https://github.com/simonw/llm/blob/c2ec8a9c60ac38d152ed48ba8c7c067c8d2c9859/docs/python-api.md

The default implementation of continue mode can be really basic: it just grabs the text version of the prompts and responses from the previous messages into a list and joins them with newlines.

I'm still not clear on the best way to truncate messages in continue mode, right now I'm going to leave that and allow the model to return an error - but it would be good to have a strategy for that involving automatic truncating later on.

I don't think I should solve continuation mode until I've re-implemented logging to the database.
Although... I do want continue mode to be possible even without having database logging, at least at the Python API layer.

Got all the tests passing, partly by disabling the DB logging test.

E       AssertionError: assert 'id          ...03T20:26:40\n' == 'id          ...03T13:26:40\n'
E         Skipping 89 identical leading characters in diff, use -v to show
E         - 020-05-03T13:26:40
E         ?           ^^
E         + 020-05-03T20:26:40
E         ?           ^^
E         - babbage:2020-05-03    openai      2020-05-03T13:26:40
E         ?                                              ^^
E         + babbage:2020-05-03    openai      2020-05-03T20:26:40
E         ?                                              ^^

I think that test fails because of timezone differences between GitHub Actions and my laptop.

Docs can also be previewed here: https://llm--65.org.readthedocs.build/en/65/python-api.html

I think a Response should have a .prompt property that leads back to the prompt used for that response.
Maybe there should be a way in which these chain together, for the purposes of modeling conversations in situations where the SQLite log isn't being used?
Then perhaps you could do this:
response = model.prompt("Ten names for a pet pelican")
print(response.text())
response2 = response.reply("Make them more grandiose")
print(response2.text())

Problem with .reply() is: is that the same thing as .prompt() or .stream() or something else?

Another option: the model itself could keep a in-memory cache of its previous prompts, such that you can then reply via the model.
I'm not keen on this though, because the conversation state shouldn't be shared by every user of the model instance in a situation like llm web where the model may be serving multiple conversations at once.

The current signature for Response: https://github.com/simonw/llm/blob/69ce584ee8fa7608afa3b6541ccc5ffbf9f93623/llm/models.py#L28-L33
If it also grew a .model property for tracking the model instance that created it, it would have enough to execute replies to itself.

What would this look like if everything was represented in terms of chains of Prompts and Responses? Those chains could then be serialized and deserialized to SQLite, or to JSON or other formats too.
Especially when function start coming into play, there's something very interesting about storing a high fidelity representation of the full sequence of prompts and responses that got to the most recent state.

Whether or not something should stream is currently a property of the Response. That works: the .reply() method could pass on the streaming state to the next prompt that is executed.

_______________ ERROR collecting tests/test_cli_openai_models.py _______________
ImportError while importing test module '/home/runner/work/llm/llm/tests/test_cli_openai_models.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_cli_openai_models.py:2: in <module>
    from llm.cli import cli
/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/site-packages/llm/cli.py:7: in <module>
    from .plugins import pm, get_plugins, get_model_aliases, get_models_with_aliases
/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/site-packages/llm/plugins.py:21: in <module>
    mod = importlib.import_module(plugin)
/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
E   ModuleNotFoundError: No module named 'llm.default_plugins'

Something very broken there. I tried python setup.py sdist and then installing llm into a fresh virtualenv:
pip install ~/Dropbox/Development/llm/dist/llm-0.4.1.tar.gz

And I get this error when I run it:
% llm --help
Traceback (most recent call last):
  File "/Users/simon/.local/share/virtualenvs/foop-q39PfZK-/bin/llm", line 5, in <module>
    from llm.cli import cli
  File "/Users/simon/.local/share/virtualenvs/foop-q39PfZK-/lib/python3.11/site-packages/llm/cli.py", line 7, in <module>
    from .plugins import pm, get_plugins, get_model_aliases, get_models_with_aliases
  File "/Users/simon/.local/share/virtualenvs/foop-q39PfZK-/lib/python3.11/site-packages/llm/plugins.py", line 21, in <module>
    mod = importlib.import_module(plugin)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File<EMAIL_ADDRESS>line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'llm.default_plugins'

Fixed the setup.py bundling issue. Tests all pass now!

Now that some models live in llm.default_models.openai_models they are pretty inconvenient to import.
I'm tempted to add this helper:
from llm import get_model
gpt4 = get_model("gpt-4")

This will provide Python API level access to both the model plugins mechanism and the aliases mechanism.

Prototype of reply():
diff --git a/llm/default_plugins/openai_models.py b/llm/default_plugins/openai_models.py
index aff8c26..6e2bac3 100644
--- a/llm/default_plugins/openai_models.py
+++ b/llm/default_plugins/openai_models.py
@@ -62,8 +62,7 @@ def register_commands(cli):

 class ChatResponse(Response):
     def __init__(self, prompt, model, stream, key):
-        super().__init__(prompt, model)
-        self.stream = stream
+        super().__init__(prompt, model, stream)
         self.key = key

     def iter_prompt(self):
diff --git a/llm/default_plugins/vertex_models.py b/llm/default_plugins/vertex_models.py
index c59130c..aeb22de 100644
--- a/llm/default_plugins/vertex_models.py
+++ b/llm/default_plugins/vertex_models.py
@@ -11,7 +11,7 @@ def register_models(register):
 class VertexResponse(Response):
     def __init__(self, prompt, model, key):
         self.key = key
-        super().__init__(prompt, model)
+        super().__init__(prompt, model, stream=False)

     def iter_prompt(self):
         url = (
diff --git a/llm/models.py b/llm/models.py
index f6cae4a..bb3ec3f 100644
--- a/llm/models.py
+++ b/llm/models.py
@@ -26,13 +26,26 @@ class OptionsError(Exception):

 class Response(ABC):
-    def __init__(self, prompt: Prompt, model: "Model"):
+    def __init__(self, prompt: Prompt, model: "Model", stream: bool):
         self.prompt = prompt
         self.model = model
+        self.stream = stream
         self._chunks = []
         self._debug = {}
         self._done = False

+    def reply(self, prompt, system=None, **options):
+        new_prompt = self.prompt.prompt + "\n" + prompt
+        return self.model.execute(
+            Prompt(
+                new_prompt,
+                system=system or self.prompt.system or None,
+                model=self.model,
+                options=options,
+            ),
+            stream=self.stream,
+        )
+
     def __iter__(self):
         if self._done:
             return self._chunks

I tried it out:
>>> from llm.default_plugins.openai_models import Chat
>>> gpt4 = Chat("gpt-4")
>>> r = gpt4.prompt("three names for a pelican")
>>> r.text()
'1. Peter the Pelican\n2. Patty the Pelican \n3. Percy the Pelican'
>>> r.reply('2 more and  make them majestic')
<llm.default_plugins.openai_models.ChatResponse object at 0x103c9f6d0>
>>> r2 = _
>>> r2.text()
'1. Crested Skysurfer\n2. Coastal Guardian\n3. Piercing Bill\n4. Sovereign Sea Drake\n5. Crowned Aquatic Phoenix'
>>> r2.prompt
Prompt(prompt='three names for a pelican\n2 more and  make them majestic', model=<llm.default_plugins.openai_models.Chat object at 0x10244e990>, system=None, prompt_json=None, options={})

That's not quite right, I forgot to include both the previous prompt and the previous response.

This is better:
    def reply(self, prompt, system=None, **options):
        new_prompt = [self.prompt.prompt, self.text(), prompt]
        return self.model.execute(
            Prompt(
                "\n".join(new_prompt),
                system=system or self.prompt.system or None,
                model=self.model,
                options=options,
            ),
            stream=self.stream,
        )

>>> from llm.default_plugins.openai_models import Chat
>>> gpt4 = Chat("gpt-4")
>>> r = gpt4.prompt("three names for a pelican")
>>> list(r)
['1. Seabird\n2. Water bird\n3. Pelecanus (genus name)']
>>> r2 = r.reply('2 more and  make them majestic')
>>> for token in r2: print(token)
...
4. Ocean Sovereign
5. Sky Monarch
>>> r2.text()
'4. Ocean Sovereign\n5. Sky Monarch'
>>> print(r2.prompt.prompt)
three names for a pelican
1. Seabird
2. Water bird
3. Pelecanus (genus name)
2 more and  make them majestic

This default strategy of joining them with \n does work, but the OpenAI Chat APIs should do it more like this instead:
messages = []
for entry in history:
    if entry.get("system"):
        messages.append({"role": "system", "content": entry["system"]})
    messages.append({"role": "user", "content": entry["prompt"]})
    messages.append({"role": "assistant", "content": entry["response"]})

Maybe .reply() should be even simpler than that - it just takes a prompt, with the system prompt and options from the previous prompt always being reused.
If you want to go fancier than that you need to assemble it yourself.

I can't remember why I decided that .stream() and .prompt() should exist separately - why not just have .prompt() and it always returns  a Response which can be iterated over or have .text() called on it?

I could have an assemble_conversation() method on Response which can be over-ridden by a ChatResponse subclass.
Challenge though is that I really need to be storing the original JSON used by those Chat prompts - my previous code reassembled it from SQLite logs, but if I want to be able to assemble it from just in-memory Python classes I need to rethink how it works.

PaLM 2 has its own way of representing prompts as well.
It's infuriatingly hard to figure out what that representation looks like, because they really want you to use one of their various client libraries rather than telling you how their HTTP+JSON API works, but there's a clue in here: https://github.com/google/generative-ai-docs/blob/main/site/en/tutorials/chat_quickstart.ipynb
[{'author': '0', 'content': 'Hello'},
 {'author': '1', 'content': 'Hi there! How can I help you today?'},
 {'author': '0', 'content': "Just chillin'"},
 {'author': '1',
  'content': "That's great! Chilling is a great way to relax and de-stress. I hope you're having a good day."}]

Extracted the PaLM stuff out into a separate plugin: https://github.com/simonw/llm-palm

A sticking point for me now is the database schema design. I think I should be storing a bunch more stuff in JSON for the more complicated model prompts.
I was worried about space... but no, that's not worth worrying about at all. I think I should optimize for storing as much detail as possible in SQLite - if people find their database growing too large as a result I can provide tools later on for cleaning up old data, or archiving it out, or even dropping the detailed JSON from older records in favour of just storing the text prompt and text response.
I doubt many people will ever notice even if their LLM database grows to 100s of MBs.

Bonus: if I'm storing verbose JSON, I can implement conversations using JUST the previous prompt - I won't need to crawl back through the history to build up the prior conversation snippets.

Here's the current Prompt class design:
https://github.com/simonw/llm/blob/e1d2a71fd6692bf50831343b21a1b4587b8c59fb/llm/models.py#L8-L21
I could add a to_log() method which turns that into something suitable to be stored in SQLite or elsewhere.
The Response class is going to be more complicated because responses differ much more extensively between models. But models are already expected to subclass Response.
Maybe they should be expected to subclass Prompt too? The current design attempts to be flexible enough that they don't need to, but I'm not at all convinced that will hold up as I start adding features like OpenID functions. Right now I'd have to cram all of the function stuff into that one prompt_json property.
As a reminder, here's the current SQL schema for the log table, which I should probably reconsider as well:
https://github.com/simonw/llm/blob/e1d2a71fd6692bf50831343b21a1b4587b8c59fb/docs/logging.md?plain=1#L78-L88

I think the Prompt class has nothing to do with logging at all - it all happens on the Response class, which has a self.prompt it can use to access details of the associated prompt.

I started with this on Response:
    @abstractmethod
    def to_log(self) -> Dict[str, Any]:
        "Return a dict of data to log"
        pass

But then I realized that since this record will usually be written to a SQLite table with a fixed schema, it would be better to have it return a dataclass:
    @abstractmethod
    def to_log(self) -> LogMessage:
        "Return a LogMessage of data to log"
        pass

Aside: this makes me think it might be cool if sqlite-utils had a way of working with dataclasses rather than just dicts, and knew how to create a SQLite table to match a dataclass and maybe how to code-generate dataclasses for a specific table schema (dynamically or even using code-generation that can be written to disk, for better editor integrations).

Here's a log entry (currently in the log2 table) for this prompt:
llm '2 dog emojis'

The prompt_json and response_json fields are obviously wrong. Still trying to get my head around how those should work.

I'm going to implement -o/--option for the OpenAI model to help decide how the database rows should work.

Test PaLM message logged like this:
llm -m palm 'two dog emojis

The key to figuring out what should go in the prompt_json logging field is to think about how conversation history mode used to work:
    if history:
        for entry in history:
            if entry.get("system"):
                messages.append({"role": "system", "content": entry["system"]})
            messages.append({"role": "user", "content": entry["prompt"]})
            messages.append({"role": "assistant", "content": entry["response"]})
            history_model = entry["model"]
    if system:
        messages.append({"role": "system", "content": system})
    messages.append({"role": "user", "content": prompt})

That's the JSON that should be stored.
And the bonus here is that I can simplify how -c mode works: it always pulls the most recent message sent to the same model - or if --chat is passed it uses that message. It copies the logged prompt_json and response from that and uses those to construct the new, longer prompt JSON.
It doesn't need to consult previous history because each logged prompt has all of the information in it already.

Or... maybe I just log the full JSON that would be sent and recieved from the API. According to https://platform.openai.com/docs/api-reference/making-requests
{
     "model": "gpt-3.5-turbo",
     "messages": [{"role": "user", "content": "Say this is a test!"}],
     "temperature": 0.7
}

And for the response:
{
   "id":"chatcmpl-abc123",
   "object":"chat.completion",
   "created":1677858242,
   "model":"gpt-3.5-turbo-0301",
   "usage":{
      "prompt_tokens":13,
      "completion_tokens":7,
      "total_tokens":20
   },
   "choices":[
      {
         "message":{
            "role":"assistant",
            "content":"\n\nThis is a test!"
         },
         "finish_reason":"stop",
         "index":0
      }
   ]
}

There's a little bit of duplicate information in there, but maybe I should stop worrying so much about database storage space and ship the feature already!

The challenge is what to store here for streaming mode, where the JSON comes in as separate chunks. I'm not too excited about storing those chunks, they're pretty verbose for large responses.

Another challenge: how to get the OpenAI client libraries to expose that raw JSON to me?

https://github.com/openai/openai-python/blob/041bf5a8ec54da19aad0169671793c2078bd6173/openai/openai_response.py looks promising:
from typing import Optional

class OpenAIResponse:
    def __init__(self, data, headers):
        self._headers = headers
        self.data = data

    @property
    def request_id(self) -> Optional[str]:
        return self._headers.get("request-id")

    @property
    def retry_after(self) -> Optional[int]:
        try:
            return int(self._headers.get("retry-after"))
        except TypeError:
            return None

    @property
    def operation_location(self) -> Optional[str]:
        return self._headers.get("operation-location")

    @property
    def organization(self) -> Optional[str]:
        return self._headers.get("OpenAI-Organization")

    @property
    def response_ms(self) -> Optional[int]:
        h = self._headers.get("Openai-Processing-Ms")
        return None if h is None else round(float(h))

https://github.com/openai/openai-python/blob/041bf5a8ec54da19aad0169671793c2078bd6173/openai/openai_object.py#L189-L208
        if stream:
            assert not isinstance(response, OpenAIResponse)  # must be an iterator
            return (
                util.convert_to_openai_object(
                    line,
                    api_key,
                    self.api_version,
                    self.organization,
                    plain_old_data=plain_old_data,
                )
                for line in response
            )
        else:
            return util.convert_to_openai_object(
                response,
                api_key,
                self.api_version,
                self.organization,
                plain_old_data=plain_old_data,
            )

https://github.com/openai/openai-python/blob/041bf5a8ec54da19aad0169671793c2078bd6173/openai/util.py#L101 is that convert_to_openai_object() function.

Found this with the debugger - response is a neat object:
pprint(response.to_dict_recursive())
{'choices': [{'finish_reason': 'stop',
              'index': 0,
              'message': {'content': 'One.', 'role': 'assistant'}}],
 'created':<PHONE_NUMBER>,
 'id': 'chatcmpl-7Y2uAWr7wggQVcH485MDmA7JCvOLj',
 'model': 'gpt-4-0613',
 'object': 'chat.completion',
 'usage': {'completion_tokens': 2, 'prompt_tokens': 9, 'total_tokens': 11}}

For the streaming one, I captured all chunks for this prompt:
llm 'say dog twice.'

Then used a breakpoin() to run this:
print(json.dumps([chunk.to_dict_recursive() for chunk in chunks], indent=2))

And got:
[
  {
    "id": "chatcmpl-7Y2yYxKw3ujWjJdTWHCRs3XzN08FB",
    "object": "chat.completion.chunk",
    "created":<PHONE_NUMBER>,
    "model": "gpt-4-0613",
    "choices": [
      {
        "index": 0,
        "delta": {
          "role": "assistant",
          "content": ""
        },
        "finish_reason": null
      }
    ]
  },
  {
    "id": "chatcmpl-7Y2yYxKw3ujWjJdTWHCRs3XzN08FB",
    "object": "chat.completion.chunk",
    "created":<PHONE_NUMBER>,
    "model": "gpt-4-0613",
    "choices": [
      {
        "index": 0,
        "delta": {
          "content": "Dog"
        },
        "finish_reason": null
      }
    ]
  },
  {
    "id": "chatcmpl-7Y2yYxKw3ujWjJdTWHCRs3XzN08FB",
    "object": "chat.completion.chunk",
    "created":<PHONE_NUMBER>,
    "model": "gpt-4-0613",
    "choices": [
      {
        "index": 0,
        "delta": {
          "content": ","
        },
        "finish_reason": null
      }
    ]
  },
  {
    "id": "chatcmpl-7Y2yYxKw3ujWjJdTWHCRs3XzN08FB",
    "object": "chat.completion.chunk",
    "created":<PHONE_NUMBER>,
    "model": "gpt-4-0613",
    "choices": [
      {
        "index": 0,
        "delta": {
          "content": " dog"
        },
        "finish_reason": null
      }
    ]
  },
  {
    "id": "chatcmpl-7Y2yYxKw3ujWjJdTWHCRs3XzN08FB",
    "object": "chat.completion.chunk",
    "created":<PHONE_NUMBER>,
    "model": "gpt-4-0613",
    "choices": [
      {
        "index": 0,
        "delta": {
          "content": "."
        },
        "finish_reason": null
      }
    ]
  },
  {
    "id": "chatcmpl-7Y2yYxKw3ujWjJdTWHCRs3XzN08FB",
    "object": "chat.completion.chunk",
    "created":<PHONE_NUMBER>,
    "model": "gpt-4-0613",
    "choices": [
      {
        "index": 0,
        "delta": {},
        "finish_reason": "stop"
      }
    ]
  }
]

Got Code Interpreter to write code to build a single object out of that for me: https://chat.openai.com/share/c8d2accd-798b-41b0-937e-18014151c79a

After combining chunks it looks like this in the logs:

And saving prompt JSON too:

To land this I need to:

Implement --continue/--chat
Implement a migration to the new database schema
Add tests for the new database schema

I don't like timestamp_utc as the column names it hints at an integer. datetime_utc would be better.

OK, I think I understand the database schema now:

id - incrementing ID
model - model ID string
prompt - simple string version of the prompt
system - simple string version of the system prompt (if set)
options_json - JSON dictionary of options that were set
prompt_json - detailed JSON version of the prompt, with enough information to repeat it and to build an ongoing conversation in case of replies
response - simple string version of the response
response_json - high fidelity JSON version of the response, which should also encapsulate enough information to continue the conversation if necessary
reply_to_id - foreign key ID reference to the message this was a reply to (or null)
chat_id - foreign key ID reference to the first message in the thread, if part of a thread
debug_json - any debug information that came back (or maybe drop this?)
duration_ms - duration measured for prompt, as integer ms
datetime_utc - an ISO UTC datestamp string of when this was recorded

I've made a few changes here:

options  is now called options_json for consistency with the other JSON fields
Added reply_to_id to explicitly track the previous message in a conversation
chat_id is now a denormalized field - it could be figured out by following reply_to_id back multiple times, but I think having a single ID for the conversation will be more convenient. This is the ID that will be passed to --chat so you can continue a conversation without constantly having to update that parameter to the ID of the most recent message
Renamed timestamp_utc to datetime_utc

One open question: should I drop debug_json? If I'm storing the whole response in response_json maybe that covers the same ground?

Confirmed: debug_json duplicates information from response_json - I'm going to drop it entirely:

I need to rewrite or replace the existing log() function.

I'm renaming the log table to logs because that is what the command and file are called.

Now I need to write the migration to upgrade the existing log table.
Here is the old log and the new logs schema:
sqlite-utils schema "$(llm logs path)" log logs

CREATE TABLE "log" (
   [id] INTEGER PRIMARY KEY,
   [model] TEXT,
   [timestamp] TEXT,
   [prompt] TEXT,
   [system] TEXT,
   [response] TEXT,
   [chat_id] INTEGER REFERENCES [log]([id])
, [debug] TEXT, [duration_ms] INTEGER, [prompt_json] TEXT)
CREATE TABLE [logs] (
   [id] INTEGER PRIMARY KEY,
   [model] TEXT,
   [prompt] TEXT,
   [system] TEXT,
   [options_json] TEXT,
   [prompt_json] TEXT,
   [response] TEXT,
   [response_json] TEXT,
   [reply_to_id] TEXT,
   [chat_id] TEXT,
   [duration_ms] INTEGER,
   [datetime_utc] TEXT
)

Except those types are wrong, should be:
   [reply_to_id] INTEGER,
   [chat_id] INTEGER,

I don't like how sqlite-utils added columns cause the schema to look a bit ugly.
I just spotted that a fix for this is to run transform() after the column has been added:
>>> import sqlite_utils
>>> db = sqlite_utils.Database(memory=True)
>>> db["log"].insert({"id": 1, "name": "Hello", "age": 31}, pk="id")
<Table log (id, name, age)>
>>> db["log"].insert({"name": "Hello", "age": 31, 'weight':3.5}, alter=True)
<Table log (id, name, age, weight)>
>>> print(db["log"].schema)
CREATE TABLE [log] (
   [id] INTEGER PRIMARY KEY,
   [name] TEXT,
   [age] INTEGER
, [weight] FLOAT)
>>> db["log"].transform(rename={"name": "name2"})
<Table log (id, name2, age, weight)>
>>> print(db["log"].schema)
CREATE TABLE "log" (
   [id] INTEGER PRIMARY KEY,
   [name2] TEXT,
   [age] INTEGER,
   [weight] FLOAT
)

... and it turns out running .transform() with no arguments still fixes the format of the schema!
>>> db["log"].add_column("foo", str)
<Table log (id, name2, age, weight, foo)>
>>> db["log"].add_column("bar", str)
<Table log (id, name2, age, weight, foo, bar)>
>>> db["log"].add_column("baz", str)
<Table log (id, name2, age, weight, foo, bar, baz)>
>>> print(db["log"].schema)
CREATE TABLE "log" (
   [id] INTEGER PRIMARY KEY,
   [name2] TEXT,
   [age] INTEGER,
   [weight] FLOAT
, [foo] TEXT, [bar] TEXT, [baz] TEXT)
>>> db["log"].transform()
<Table log (id, name2, age, weight, foo, bar, baz)>
>>> print(db["log"].schema)
CREATE TABLE "log" (
   [id] INTEGER PRIMARY KEY,
   [name2] TEXT,
   [age] INTEGER,
   [weight] FLOAT,
   [foo] TEXT,
   [bar] TEXT,
   [baz] TEXT
)

I find myself wanting two new features in sqlite-utils:

The ability to have the new transformed table set to a specific name, while keeping the old table around
The ability to rename a table (sqlite-utils doesn't have a table rename function at all right now)

I'm going to make the changes to the old log table and then rename it to logs.

Schema after those migrations: https://github.com/simonw/llm/blob/1d5c429d3213c3c16de1c8e2e961ccf26760408a/docs/logging.md?plain=1#L78-L91

Still needed:

The conversation mode, including recording reply_to_id and chat_id
Tests for database logging

This bit is out of date now: https://github.com/simonw/llm/blob/5652b52f47a8dc4f28b7b2e6aad0f83543afe887/docs/python-api.md?plain=1#L18-L28

This bit too - there are some extra methods now: https://github.com/simonw/llm/blob/5652b52f47a8dc4f28b7b2e6aad0f83543afe887/docs/python-api.md?plain=1#L30-L34

Made a start on the docs:
LLM model plugin documentation
To add support for a different language model in LLM you need to implement a subclass of the llm.Model class.
Your model class will do the following:

Implement model_id(self) as a method that returns a unique string identifying your model. This will be logged in the database and used with the llm -m/--model option to select your model, so it should aim to be unique even when combined with other LLM plugins.
Optionally specify options that can be passed to your model, if there are any - things like temperature and top_p. These are provided on an Options inner class.
Implement an inner class called Response that subclasses llm.Response - this class will be responsible for executing prompts and returning or streaming the response.
Optionally specify if it needs an API key retrieved from LLM key storage or from an environment variable

Most of your model's work will happen in the Response subclass, most specifically in the iter_prompt() method.
This method should return an iterator over strings. Models that stream content can use this method to return tokens as they become available - models which can't stream should instead return an iterator which produces a single string with the full result.
If the underlying model returns more detailed information about the response, that information can be added to the ._response_json dictionary. This will be logged to the database when the prompt completes.
Here's a basic example of an implementation for the Google PaLM 2 model, using Google's google-generativeai library:
# code goes here ...

Registering your models
Your plugin should provide an implementation of the register_models() plugin hook which registers one or more instances of the model with the LLM tool.
The simplest version of registration looks like this:
from llm import hookimpl

@hookimpl
def register_models(register):
    register(CustomModel())

You can call register multiple times to register multiple models. If your model constructor takes different backend model IDs you could call register() like this:
@hookimpl
def register_models(register):
    register(CustomModel("text-alpha-001"))
    register(CustomModel("text-beta-001"))

If a model should be made available via some aliases you can pass those to register() like this:
register(CustomModel(
    "text-alpha-001",
    aliases=("alpha", "alpha1")
))

Users will then be able to use your model like this:
llm "say hi" -m alpha
llm "say hi" -m alpha1

I'm going to see what it looks like if I move the Response class to be an inner class on Model.

I'm reconsidering if the Prompt class is necessary at all now. I think maybe I can get everything done with:

Options - validating and transmitting the options passed to a particular model
Model - the model itself, mainly for setup for things like API keys and keep stuff in-memory for larger on-device models
Response - the response to a prompt, which also encapsulates whatever the prompt was

The model.prompt(...) method takes whatever stuff the prompt accepts and returns a Response that is ready to execute when it gets iterated on - or has .text() or similar called on it.
So what's the purpose of the prompt class? Right now it looks like this:
https://github.com/simonw/llm/blob/4b6f08abee17e7d40ab14b0622ea2deeb51d6d4d/llm/models.py#L10-L23
I guess the real answer here comes down to how stuff like OpenAI Functions work.

I remember the point of the Prompt object - it's so that a Template can return a Prompt rather than having to produce a bunch of text and system and options and soon function definitions as well.

I have two example plugins now:

https://github.com/simonw/llm-palm
https://github.com/simonw/llm-markov

I haven't released either of them to PyPI, will do that after I ship the changes from this PR.

Looking at this code:
https://github.com/simonw/llm-markov/blob/c871593fbd0675d8230eaa5c72ac04bc89a28920/llm_markov/__init__.py#L57-L58
    def execute(self, prompt: Prompt, stream: bool = True) -> Response:
        return self.Response(prompt, self, stream)

There's no reason that couldn't be the default implementation on the base Model class.

I'd like to make options validation errors a bit neater.

Deleting and recreating the virtual environment on my laptop (with pip install -e '.[test]' again) helped me recreate the errors.
types-click seems to fix it.

I wrote a VERY detailed tutorial, which helped me iron out all sorts of kinks in the API: https://github.com/simonw/llm/blob/a64b83adb599e285b5d6fc1ff786daef1f60118c/docs/tutorial-model-plugin.md

I really want to merge this thing, but it's still broken --continue mode.

Here's the rendered tutorial: https://llm--65.org.readthedocs.build/en/65/tutorial-model-plugin.html

Now have three plugins that use this new hook:

https://github.com/simonw/llm-palm
https://github.com/simonw/llm-markov
https://github.com/simonw/llm-mpt30b

I think the key to implementing continue mode is to realize that a response needs to be able to round-trip to the SQLite database - so a logs record is actually everything that needs to be stored in order to re-inflate a full Response object along with the prompt and options that were used to create it.
Then Response can have a reply() method which knows how to construct a new prompt combining together the previous prompt/response and the new prompt.
Different models will need to provide different implementations of the code that combines prompts and responses together. We have three different ways of doing that already.
For OpenAI it's:
messages = []
messages.append({"role": "system", "content": previous_prompt.system})
messages.append({"role": "user", "content": prevous_prompt.prompt})
messages.append({"role": "assistant", "content": previous_response.text()})
messages.append({"role": "assistant", "content": prompt.prompt})

For PaLM it's:
 messages = [{'author': '0', 'content': 'Hello'},
 {'author': '1', 'content': 'Hi there! How can I help you today?'},
 {'author': '0', 'content': "Just chillin'"},
 {'author': '1',
  'content': "That's great! Chilling is a great way to relax and de-stress. I hope you're having a good day."}]

With system provided as the context= argument.
For MPT30B it's:
prompt_lines = [
    # System prompt
    f"<|im_start|>system\n{system_prompt}<|im_end|>\n",
    # User prompt
    f"<|im_start|>user\n{prompt.prompt}<|im_end|>\n",
    # Start of assistant prompt
    f"<|im_start|>assistant\n",
    f"{self.text()}\n"
    f"<|im_start|>user\n{prompt.prompt}<|im_end|>\n"
    f"<|im_start|>assistant\n"
]
prompt_text = "".join(prompt_lines)

On thinking further about the --continue challenge, I'm beginning to reconsider the core data structures involved.
It strikes me that there's a key concept here which isn't being captured: that of a conversation.
ChatGPT has these - it calls them "chats". A chat is a bundle of ordered messages. They have an ID and an optional title (which ChatGPT picks itself but the user can then edit) - they are the units that can be shared, e.g. https://chat.openai.com/share/00d0ce07-deab-4d51-be84-e5e20aa07039 (a long-running conversation where I brainstormed names to use for this and other related concepts).
I thought about a bunch of different names for these - threads, sequences, chats - but I think I'm going to go with "conversation".
So a conversation is an ordered sequence of X - where X could be one of the following nouns:

message
response
reply
item
round
step
completion

The challenge with picking this name is that at actually represents a pair: there's a prompt, and then there's the response from the LLM to that prompt.
I think I'm going to say that a conversation is a collection of completions.
A prompt is always the first half of a completion (the response is the second half). That completion may end up comprising multiple messages - a system message, a user message, then an assistant message for example. The thing that makes it a completion is that it was sent to the LLM and the response from the LLM was then persisted.
In the common case of only running a single prompt (llm "three names for a pet skunk" for example) that's still a conversation, it's just a conversation that ended after the first completion.
I think I log conversations and completions to the database - potentially via a plugin hook which will allow users to swap out alternative persistence layers, though that one can come later.

If I'm making that radical a change to the DB schema I'm tempted to just leave log/logs as it is - users who upgrade to llm 0.5 will keep their old table but get new tables for the new design.

If I do model conversations at the persistence level I think it will be easier to implement --continue efficiently - previously I was planning to serialize the entire and prompt to a logs entry every single time, with that prompt getting longer and longer (and more heavily duplicated) every time a new message was added to that conversation.
With conversations as a high level concept I'm more comfortable with an efficient database design where the prompt has to be assembled from multiple rows every time a completion is generated.

To recap then, the concepts LLM will deal with are:

Models are GPT-4, MPT-30B, Falcon 7B etc.
Conversations are a sequence of completions (prompt-and-responses) to a model
Completions are single round-trip completions sent to a model - a prompt, options and the associated response
Prompts are the data provided by the user each turn - usually a text prompt and occasionally a text system prompt
Responses are the fully detailed responses that come back from the model
Options are the additional options sent as part of a completion

I'm still not 100% sure how this terminology works with functions - where you send a prompt, the LLM replies with a demand to run a function, then you execute that function and send its answer as part of the next prompt back to the LLM.
I think that just means that a request-to-run-a-function is a form of response, and a results-from-a-function is a form of prompt.

I'm messing around with these as the in-memory structures:
@dataclass
class Conversation:
    id: str = field(default_factory=lambda: str(ULID()).lower())
    name: Optional[str] = None
    model: Optional[str] = None
    completions: List["Completion"] = field(default_factory=list)

@dataclass
class Completion:
    id: str = field(default_factory=lambda: str(ULID()).lower())
    conversation: Conversation = None
    prompt: Prompt = None
    prompt_json: Dict[str, Any] = None
    options_json: Dict[str, Any] = None
    response: str = None
    response_json: Dict[str, Any] = None

    def __post_init__(self):
        if self.conversation is not None:
            self.conversation.completions.append(self)

Trying out ULIDs from https://pypi.org/project/python-ulid/ - which are neat in that they incorporate a timestamp as well as a UUID-style random element, so they can be sorted alphabetically to sort in time order.

So maybe the .reply() method ends up on the Conversation, not on the Completion or Response.

From a Python API point of view, I think you can instantiate a model and fire off prompts and get back responses directly... but if you want to hook into the conversation persistence layer you should do so by using llm.Conversation() instead.

So the simplest way to use the Python API would be:
import llm
gpt4 = llm.get_model("gpt-4")
print(gpt4.prompt("3 names for a pet llama").text())

And a more complex way would be:
import llm
conversation = llm.Conversation.for_model("gpt-4")
one = conversation.prompt("3 names for a pet llama")
print(one.text())
two = conversation.prompt("2 more")
print(two.text())

Crucial detail: a conversation can be rehydrated from persistent storage and continued. That's how -c mode for the llm CLI tool will work. It's likely how chat mode in the web UI will work too.

Next question: where does the custom logic go for assembling a new prompt to follow-up a previous one? As described in:

https://github.com/simonw/llm/pull/65#issuecomment-1626670374

Does that go on Model or Prompt or Conversation or Response?
I'm tempted to put it on Model or Response simply because those are the main class that the model plugin author is already subclassing.

I'm going to put it on Model because Response on its own doesn't necessarily have all of the information it needs - it might not have the full Conversation needed to build up the entire complex prompt.
Maybe this:
def combined_prompt(self, conversation, prompt, system, ...):`

Other names:

extend_prompt()
continue_prompt()
continue_conversation()

Current model methods are prompt() and execute().

Another option: conversation becomes an optional argument to the existing model.prompt(...) method.

Added a fourth plugin using this hook:

https://github.com/simonw/llm-gpt4all

Another concept:
r1 = gpt4.prompt(
    "What is the meaning of life?",
)
print(r1.text())

r2 = gpt4.prompt(
    "Are you sure about that?",
    conversation=[r1],
)
print(r2.text())

r3 = gpt4.prompt(
    "Clarify further",
    conversation=[r1, r2],
)
print(r3.text())

Here conversation= can be a straight up list of previous responses. Maybe it can be that OR a Conversation dataclass instance.

I'm not convinced by the way the Prompt() class works right now.
Should it represent the final prompt that was sent (with the JSON message list or whatever), or should it just encapsulate the message that came from the user? Should it even exist at all?

Unfortunately I feel like I need to solve all of these problems in order to land this:

Persistence (aka logging) to the DB and for a future plugin hook
Continue conversations for in-memory Python usage with no persistence at all
Continue mode for the CLI tool
Prompt combination - how does continue mode actually construct a prompt that works with the given plugin model?
Functions. I need to know how they're going to fit into all of this

Here's a sketched out version of what I've been trying so far:
diff --git a/llm/models.py b/llm/models.py
index 20b81a6..8108a4a 100644
--- a/llm/models.py
+++ b/llm/models.py
@@ -1,24 +1,23 @@
-from dataclasses import dataclass, asdict
+from dataclasses import dataclass, asdict, field
 import datetime
 from .errors import NeedsKeyException
 import time
-from typing import cast, Any, Callable, Dict, Iterator, List, Optional, Set
+from typing import cast, Any, Callable, Dict, Iterator, List, Optional, Set, Union
 from abc import ABC, abstractmethod
 import os
 from pydantic import ConfigDict, BaseModel
+from ulid import ULID

 @dataclass
 class Prompt:
     prompt: str
-    model: "Model"
     system: Optional[str]
     prompt_json: Optional[str]
     options: "Options"

-    def __init__(self, prompt, model, system=None, prompt_json=None, options=None):
+    def __init__(self, prompt, system=None, prompt_json=None, options=None):
         self.prompt = prompt
-        self.model = model
         self.system = system
         self.prompt_json = prompt_json
         self.options = options or {}
@@ -28,6 +27,51 @@ class OptionsError(Exception):
     pass

+@dataclass
+class Conversation:
+    id: str = field(default_factory=lambda: str(ULID()).lower())
+    name: Optional[str] = None
+    model: Optional[str] = None
+    completions: List["Completion"] = field(default_factory=list)
+
+    @classmethod
+    def for_model(cls, model_id: str) -> "Conversation":
+        from llm.plugins import get_model
+
+        model = get_model(model_id)
+        return cls(model=model)
+
+    def prompt(
+        self, prompt: str, system: str = None, stream: bool = True, options: dict = None
+    ) -> "Response":
+        return self.model.prompt_continue(
+            self, prompt, system=system, stream=stream, options=options
+        )
+
+    def execute(self, prompt: Prompt, stream: bool = True) -> "Response":
+        response = self.model.execute(prompt, stream)
+        response.on_complete(self.record_completion)
+        return response
+
+    def record_completion(self, completion: "Completion"):
+        self.completions.append(completion)
+
+
+@dataclass
+class Completion:
+    id: str = field(default_factory=lambda: str(ULID()).lower())
+    conversation: Conversation = None
+    prompt: Prompt = None
+    prompt_json: Dict[str, Any] = None
+    options_json: Dict[str, Any] = None
+    response: str = None
+    response_json: Dict[str, Any] = None
+
+    def __post_init__(self):
+        if self.conversation is not None:
+            self.conversation.completions.append(self)
+
+
 @dataclass
 class LogMessage:
     model: str  # Actually the model.model_id string
@@ -52,18 +96,8 @@ class Response(ABC):
         self._chunks: List[str] = []
         self._done = False
         self._response_json = None
-
-    def reply(self, prompt, system=None, **options):
-        new_prompt = [self.prompt.prompt, self.text(), prompt]
-        return self.model.execute(
-            Prompt(
-                "\n".join(new_prompt),
-                system=system or self.prompt.system or None,
-                model=self.model,
-                options=options,
-            ),
-            stream=self.stream,
-        )
+        self._complete_callbacks = []
+        self._completion: Completion = None

     def __iter__(self) -> Iterator[str]:
         self._start = time.monotonic()
@@ -75,12 +109,32 @@ class Response(ABC):
             self._chunks.append(chunk)
         self._end = time.monotonic()
         self._done = True
+        self._completion = Completion(
+            prompt=self.prompt,
+            prompt_json=self._prompt_json,
+            options_json={
+                key: value
+                for key, value in self.prompt.options.model_dump().items()
+                if value is not None
+            },
+            response=self.text(),
+            response_json=self.json(),
+        )
+        for callback in self._complete_callbacks:
+            callback(self._completion)
+
+    def completion(self):
+        self._force()
+        return self._completion

     @abstractmethod
     def iter_prompt(self, prompt: Prompt) -> Iterator[str]:
         "Execute prompt and yield chunks of text, or yield a single big chunk"
         pass

+    def on_complete(self, callback) -> None:
+        self._complete_callbacks.append(callback)
+
     def _force(self):
         if not self._done:
             list(self)
@@ -103,7 +157,7 @@ class Response(ABC):

     def log_message(self) -> LogMessage:
         return LogMessage(
-            model=self.prompt.model.model_id,
+            model=self.model.model_id,
             prompt=self.prompt.prompt,
             system=self.prompt.system,
             prompt_json=self._prompt_json,
@@ -164,14 +218,40 @@ class Model(ABC):
         self,
         prompt: Optional[str],
         system: Optional[str] = None,
+        conversation: Optional[Union[List[str], Conversation]] = None,
         stream: bool = True,
         **options
     ):
         return self.execute(
-            Prompt(prompt, system=system, model=self, options=self.Options(**options)),
+            Prompt(prompt, system=system, options=self.Options(**options)),
             stream=stream,
         )

+    def prompt_continue(
+        self,
+        conversation: Conversation,
+        prompt: Optional[str],
+        system: Optional[str] = None,
+        stream: bool = True,
+        **options
+    ):
+        if system is None:
+            # Use most recent system prompt in conversation
+            for completion in reversed(conversation.completions):
+                if completion.system:
+                    system = completion.prompt
+                    break
+        # Build prompt from conversation
+        bits = []
+        for completion in conversation.completions:
+            if completion.system:
+                bits.append(completion.system)
+            bits.append(completion.prompt)
+            bits.append(completion.response)
+        if prompt:
+            bits.append(prompt)
+        return self.prompt("\n".join(bits), system=system, stream=stream, **options)
+
     def execute(self, prompt: Prompt, stream: bool = True) -> Response:
         r = cast(Callable, getattr(self, "Response"))
         kwargs = {}

I figured out how to use functions= with the openai.ChatCompletion.create() method - TIL: https://til.simonwillison.net/gpt3/openai-python-functions-data-extraction

I tried adding this method to Model:
    def chain(
        self,
        prompt: Optional[str],
        system: Optional[str] = None,
        stream: bool = True,
        proceed: Optional[Callable] = None,
        **options
    ):
        if proceed is None:
            proceed = lambda response: None
        while True:
            if isinstance(prompt, str):
                prompt = Prompt(prompt, model=self, system=system, options=self.Options(**options))
            response = self.execute(
                prompt,
                stream=stream,
            )
            yield response
            next_prompt = proceed(response)
            if not next_prompt:
                break
            prompt = next_prompt

Then used it like this:
import llm

g = llm.get_model("chatgpt")

done = [False]
def proceed(response):
    if not done[0]:
        done[0] = True
        return llm.Prompt("Answer again but with much more passion!", response.model)
    else:
        return None

for r in g.chain("What is blue?", proceed=proceed):
    for token in r:
        print(token, end='')
    print()

Result:

Blue is one of the primary colors and is often described as a color located between green and violet on the color wheel. It is associated with the sky, water, and tranquility. Blue is considered a cool color and is commonly used to symbolize calmness, serenity, and stability.
Absolutely! I would be more than thrilled to answer again with a resounding passion that cannot be contained!
Every word that I speak shall be filled with an overwhelming fervor that electrifies the air around me. My enthusiasm knows no bounds, and my heart beats with a fiery passion that ignites my soul!
Get ready to feel the intensity as I dive headfirst into answering your question with a fiery passion!

Though note that the passionate answer wasn't actually a reply to the previous one because I haven't implemented replies/conversations yet.
But I do like the .chain(..., proceed=) pattern.
A FunctionChat class could implement a default .chain() method that knows how to execute and continue functions.

Does putting .chain() on model really make sense, or should it be on Conversation? Chaining doesn't make sense outside of the context of a conversation.

What is a Prompt anyway?
Currently it's shaped like this:
@dataclass
class Prompt:
    prompt: str
    model: "Model"
    system: Optional[str]
    prompt_json: Optional[str]
    options: "Options"

    def __init__(self, prompt, model, system=None, prompt_json=None, options=None):
        self.prompt = prompt
        self.model = model
        self.system = system
        self.prompt_json = prompt_json
        self.options = options or {}

So it sort-of represents the input from the user (the prompt and system strings, plus the options) - but it also has that confusing prompt_json option which I don't think I've ever actually used.
I'm mixing several concepts here:

The prompt that the user provided as string through the CLI interface or eventually through the llm web UI
In Python library land, this is also the prompt/system prompt strings passed to model.prompt(...)
The options being used for this execution
The model that the user is using (I think that should be completely independent of the prompt)
The actual complex JSON-or-similar data structure that's passed to the low-level library. This could potentially include previous chat messages or even things like function results
The function definitions themselves need to be considered too
Parts of the prompt that are later serialized to the database

Having that one class handle all of the above doesn't make sense to me.

Passing system= around is a little confusing, since some models don't support a system prompt at all and, for those that do, the system prompt is usually set once at the start of a conversation (though it could be tweaked later on in some weird edge cases).
The system prompt actually ends up being a separate message in a conversation, albeit one that's sent in a bundle with a user's prompt, not sent independently.

Important to note that calls to .chain() that result in multiple round-trips to the LLM should also result in multiple records being written to the log database.

If the API was built around conversations instead, it could look like this:
import llm

gpt4 = llm.get_model("gpt-4")

conversation = gpt4.conversation(system="You are a sentient cheesecake")
print(conversation.prompt("What are you made of?").text())
print(conversation.prompt("Why those ingredients?").text())

Maybe functions would look like this:
from my_functions import execute_sql

gpt4func = llm.get_model("gpt-4-functions")

conversation = gpt4.conversation(
    system="You are a sentient cheesecake and a SQL expert",
    functions=[execute_sql]
)
for response in conversation.chain("How many cheesecakes did you sell?"):
    print(response.text())

And if you're logging to the DB:
gpt4 = llm.get_model("gpt-4")

conversation = gpt4.conversation(system="You are a sentient cheesecake", log=True)
print(conversation.prompt("What are you made of?").text())

That log=True causes the plugin hook for logging to be fired.
As an alternative, maybe pass logger=my_log_object to instead explicitly log things by calling that function.

The key thing about this design is that it becomes the job of the Conversation to now how to build prompts and combine them together in a way that makes sense for the specific model.
So maybe... models provide an inner Conversation class instead of focusing on that inner Response class (or in addition to that).
And the .prompt() method on model is actually just a wrapper that creates a new conversation, runs a single prompt through it and returns the response.

Writing an inner Conversation subclass inside of Model just feels weird to me. Can I get away with writing custom methods on Model instead for this? Or on Response?
Could I ditch the custom Response thing in favour of methods on model?

Yeah, I think the Response class shouldn't actually differ between models. It's job is mainly to know how to either iterate through streaming model or call self._force() to force that execution in order to answer .text() and the like.
I'd like a design where all of the customization happens in a small number of model methods.

So maybe Response.iter_prompt() becomes Model.execute_prompt(), and one of the things that method is expected to do is concatenate together the JSON/syntax necessary to hand the previous conversation over to the LLM.

def execute_prompt(self, prompt_text=None, system_text=None, conversation=None, options=None):

But where do functions= go?

Found this uncommitted experimental code in llm-mpt30b:
        def reply_prompt(self, prompt: llm.Prompt) -> llm.Prompt:
            prompt_lines = self._prompt_json["prompt_lines"]
            prompt_lines.extend([
                f"{self.text()}\n"
                f"<|im_start|>user\n{prompt.prompt}<|im_end|>\n"
                f"<|im_start|>assistant\n"
            ])
            return llm.Prompt(prompt.prompt, self.model, system=prompt.system or self.prompt.system, prompt_json={"prompt_lines": prompt_lines})

Here's the updated tutorial: https://llm--65.org.readthedocs.build/en/65/tutorial-model-plugin.html

This branch has gone on long enough. I'm going to merge it without the --continue feature and continue development on the main branch.

I don't like how that git push --force-with-lease seems to have reset the commit date on all of my commits though.

Looks like I can fix that with:
git filter-repo --commit-callback '
    if commit.committer_date == b"Mon Jul 10 08:39:00 2023 -0700":
        commit.committer_date = commit.author_date
' --force

Parsed 172 commits
New history written in 0.03 seconds; now repacking/cleaning...
Repacking your repo and cleaning out old unneeded objects
HEAD is now at 7ac007a Show error for --continue mode, remove deleted code
Enumerating objects: 907, done.
Counting objects: 100% (907/907), done.
Delta compression using up to 12 threads
Compressing objects: 100% (271/271), done.
Writing objects: 100% (907/907), done.
Total 907 (delta 621), reused 907 (delta 621), pack-reused 0
Completely finished after 0.16 seconds.

That didn't quite work - it didn't actually update the commit dates.
This helped debug it:
git filter-repo --commit-callback '
    print(repr(commit.committer_date))
' --force

Turns out those dates look like this:
b'1689001152 -0700'
b'1689002848 -0700'
b'1689003110 -0700'
b'1689003276 -0700'

So I ran this command:
git filter-repo --commit-callback '
    if commit.committer_date == b"1689003540 -0700":
        commit.committer_date = commit.author_date
' --force

And then force pushed it all to main:
git push --force-with-lease origin main

Turned that into a TIL: https://til.simonwillison.net/git/git-filter-repo


transport_enabled setting on an auth domain and authorizer may be unnecessary after transport client removal

Since OpenSearch 2.0, support for the transport client has been removed. The transport client was removed from the security plugin in this PR: https://github.com/opensearch-project/security/pull/1701
My understanding of transport_enabled is that it is used to enable an auth domain for authentication on the transport layer. There is a separate setting called http_enabled to enable the auth domain on the REST layer. This setting also appears to be applicable to authorizers in the authz section.
Below is an example of a basic auth entry in the authc: section of the security plugin's config.yml file from the demo configuration:
basic_internal_auth_domain:
    description: "Authenticate via HTTP Basic against internal users database"
    http_enabled: true
    transport_enabled: true
    order: 4
    http_authenticator:
      type: basic
      challenge: true
    authentication_backend:
      type: intern

The transport_enabled setting may not be needed here after the TransportClient's removal.
First: Determine if it is safe to remove values from settings.
Second: If safe to remove, remove. If not, update documentation.

[Triage] Hi @cwperks, thank you for filing this issue. It is not clear whether the goal of this issue is the investigation, or you are going to add an action task to this. For the time being I will leave this without the triaged label.

@cwperks @scrawfor99 I'm interested in working on this issue. How would one begin to determine if this change is safe? Does this just mean the tests work correctly? Also is the config.yml file the only file where changes need to be made?

Hi @prabhask5, thanks for checking in on this issue. As Craig mentioned, this setting is a pretty old one based on enabling/disabling authentication on the transport layer. That being said, we don't have the Transport Client anymore. The two settings Craig mentioned are defined here:
| `http_enabled` | Enables or disables authentication on the REST layer. Default is `true` (enabled). |
| `transport_enabled` | Enables or disables authentication on the transport layer. Default is `true` (enabled). |

To test whether you can remove the transport setting, there are two options. First, you can try swapping it back and forth and on the config.yml settings and see what happens. If you get the same behavior with it on as with it off it is a good indicator it may not matter.
The second way to determine whether it is required is do a code search for where the setting is used and determine the extent to which it is called or checked. This should either be done instead of the first option or after having completed the first option. By checking the code use, you can get the best idea of whether the code is actually necessary.
You are right that the only changes needed here should be in the config.yml file.

These values are also displayed on the UI. Looks like they can be deleted from there as well.

@scrawfor99 @cwperks Looks like both settings are used heavily within DynamicConfigModelV7.java and DynamicConfigModelV6.java to generate AuthDomains and Authorizers SortedSets, which are then used to generate ConfigModels which are used in a lot of other files. Although transportAuthDomains and transportAuthorizers are not used, so maybe (?) they aren't needed anymore and can be safely deleted alongside the config settings, this would require significant refactor which I just want to check with before doing so. Thanks!


Add .Raw() fallback API

A fallback method to send raw sql is needed so that there's an escape hatch for queries which are not yet supported by the client.

Old spec: https://github.com/prisma/specs/blob/raw-spec/photon-raw-api/Readme.md#photon-go
Should definitely investigate how the JS client does it nowadays and see if we can align. This spec predates that implementation.


Lab 2.3 - Cannot export data

Lab 2.3
Exercise 1
I'm not 100% if this is my mistake...  :)
Exercise 1 has you export data from the Project table.   However, when you do the export, an error pops up:
"Connection to dataverse failed.  Please check the link below on how to fix this issue."
When I click the link to investigate, it takes me to https://learn.microsoft.com/en-us/power-apps/maker/data-platform/data-platform-import-export#troubleshoot-connection-issues   when says "Get Data from Excel and Export Data features are currently not included in the Power Apps Developer Plan."   But that is the type of plan we use in these labs.
So I'm not sure if it's my issue, or the lab needs to be changed, or delete this exercise.

Thanks Julian.  Is there an ETA on that next major update?

Due January 2024 but will likely need to update this lab beforehand

If I can help with beta courses, I'd be happy to, been teaching the PL-900, 100, 200, 400 and 500 for a year now.

The export option fails everywhere not just in the lab, so i think its a general issue

Please see updated labs merged today! Thanks.


patch: Redirect app-types docs to fleet-types

We probably missed redirecting app-types documentation to fleet-types page during https://github.com/balena-io/docs/issues/1969
Signed-off-by: Vipul Gupta (@vipulgupta2048) <EMAIL_ADDRESS>

Please make sure to read the CONTRIBUTING document before opening the PR for relevant information on contributing to the documentation. Thanks!

@balena-ci rebase

@balena-ci I self-certify!