[u-u] AI - Llama 2 an open source AI that can run on a Raspberry PI

[Colin McGregor]

This is a lightly edited version of a post that I made to another
mailing list recently (sorry, not sorry).

As noted before this evening's meeting I am running the open source
(but NOT GPL) Llama 2  AI on a Raspberry PI 5. Llama 2 is a product of
Meta (parent company of Facebook), and as you will see in clause 2 of
their licence agreement, there is a poison pill clause aimed at the
likes of Google (for a hobbyist interested in seeing / playing with a
local AI there are no issues) :  https://ai.meta.com/llama/license/ .

Here is the guide I used for getting Llama 2 up and running on a
Raspberry PI (I am using the PI 5, but as the video notes (subject to
conditions) it can run on a PI 4) :
https://youtu.be/idZctq7WIq4?si=kekFWQHty60dp2ce

There is an old saying "The amazing thing about a dancing bear is not
how well the bear dances, but that the bear can dance at all.". So,
yes, I will to some degree mock Llama 2 on a Raspberry PI in the
following, but... It is kind of amazing that you can get a functional,
local AI running on a machine as modest as Raspberry Pi, so the flaws
are very forgivable.

I have run Llama 2 on a Raspberry Pi 5 with 4GB of RAM and one with
8GB of RAM, with no obvious difference in performance. At present the
key issue appears to be storage I/O performance, to try to attack that
I currently have a PCIe to NVMe adapter on order from Pimoroni that
will let me use NVMe SSD storage with the PI (I want to see if/how
much that helps). I have also tried the 7b and 13b language models on
the Pi, the 7b runs (crawls?) faster than the 13b model, but the 13b
model gives better (but far from perfect) answers. I am currently
normally using the 13b model and accept the slowness. As it is,
obviously different questions take different amounts of time, but as
things stand, I will ask a question in the evening and see how things
are coming the next morning after a good night's sleep.

I have been keeping a log of the questions / answers I have had with
Llama 2 and I will start with a question I asked both the 7b and 13b
models. For those who didn't take (or failed) Canadian history in
school, let me point out that Lester B. Pearson didn't have any
significant role in the Cuban Missile Crisis of the 1960s (but his
role in the 1950s Suez Crisis got him a Nobel Peace Prize). Also, Joe
Clark was Prime Minister of Canada from June 4, 1979 to March 3, 1980
so Pierre Trudeau both preceded and succeeded Joe Clark. These are two
of the issues (there are others) in the 7b answer. There is some
legitimate room for debate on which Prime Ministers should be on lists
like these, but I don't consider any of the names presented as clearly
wrong.

7b :
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> Who was Canada's best Prime Minister?
Canada has had several effective and well-respected Prime Ministers
throughout its history, making it difficult to pinpoint a single
individual as the "best." However, some notable Prime Ministers who
have made significant contributions to the country include:

1. Sir John A. Macdonald (1867-1873, 1878-1880): Often considered one
of Canada's most important Prime Ministers, Macdonald played a crucial
role in shaping the country's political and economic systems. He was
instrumental in creating the National Policy, which helped establish
Canada as a major industrial power, and he also negotiated the
Northwest Passage Agreement with the United States, securing Canada's
western border.
2. William Lyon Mackenzie King (1921-1930, 1935-1948): King was a
skilled diplomat and leader who guided Canada through two major
crises: World War I and the Great Depression. He implemented policies
that helped stabilize the economy and provided support to those
affected by the economic downturn. Additionally, he played a key role
in establishing the Canadian Broadcasting Corporation (CBC), which has
become an important institution for Canadian culture and identity.
3. Lester B. Pearson (1963-1967): Pearson is best known for his role
in resolving the Cuban Missile Crisis, which brought the world closer
to nuclear war. He also played a crucial part in establishing the
United Nations, for which he was awarded the Nobel Peace Prize in
1957. As Prime Minister, Pearson implemented policies that promoted
peace and disarmament, and he worked tirelessly to improve Canada's
international reputation.
4. Pierre Trudeau (1968-1984): Trudeau is remembered for his
charismatic leadership and his efforts to promote Canadian identity.
He implemented a number of progressive policies, including the
patriation of the Canadian Constitution, which gave the province of
Quebec greater autonomy, and he also legalized same-sex marriage in
1979. Additionally, Trudeau was instrumental in establishing the
Canadian Human Rights Commission, which has helped protect the rights
of marginalized communities.
5. Jean Chrétien (1993-2003): Chrétien is credited with helping to
stabilize Canada's economy and promote international trade. He also
played a key role in the Belfast Agreement, which brought an end to
the Troubles in Northern Ireland. Additionally, he implemented
policies that helped to promote diversity and inclusion within
Canadian society.
Of course, there are many other notable Prime Ministers in Canadian
history, but these five individuals have had particularly significant
impacts on the country's political landscape and international
reputation.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
13b
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> Who was Canada's best Prime Minister?
 Determining the "best" Prime Minister of any country is subjective
and can depend on various factors such as political affiliation,
policy preferences, and the current events that occurred during their
tenure. However, here are some of the most highly-regarded Canadian
Prime Ministers based on historical assessments and public opinion:

1. Pierre Trudeau (1968-1979, 1980-1984): Known for his charisma,
progressive policies, and efforts to promote Canadian unity, Trudeau
is often considered one of Canada's greatest Prime Ministers. He
implemented policies such as official bilingualism, the Canadian
Charter of Rights and Freedoms, and the creation of the Canadian Human
Rights Commission.
2. John A. Macdonald (1867-1873, 1878-1891): As Canada's first Prime
Minister, Macdonald played a crucial role in shaping the country's
Confederation and establishing many of its foundational institutions.
He is credited with developing the National Policy, which aimed to
promote economic growth and unity through tariffs and transportation
infrastructure.
3. Lester B. Pearson (1963-1968): Pearson's tenure as Prime Minister
saw significant social and political changes in Canada, including the
introduction of universal healthcare, the establishment of a new flag,
and the devolution of power to Quebec. He was also instrumental in
promoting Canadian diplomacy and peacekeeping efforts globally.
4. Wilfrid Laurier (1896-1911): Laurier is often praised for his
leadership during Canada's early years, when the country faced
challenges such as population growth, economic development, and the
need for greater autonomy from Britain. He is credited with promoting
French-English relations and expanding the scope of federal powers.
5. Jean Chrétien (1993-2003): Chrétien's tenure saw significant
economic growth, the repayment of Canada's national debt, and the
negotiation of several major international agreements, including the
Free Trade Agreement with the United States. He also played a key role
in promoting Canadian unity during the Quebec sovereignty referendum
in 1995.

Of course, these are just a few examples of highly-regarded Prime
Ministers in Canadian history. Other notable leaders include William
Lyon Mackenzie King, Joe Clark, and Brian Mulroney, among others.
Ultimately, the "best" Prime Minister is a matter of personal opinion
and depends on one's values, priorities, and perspectives on Canadian
history.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
As someone who lives near Yonge & Eglinton, I found the following
answer out of the 13b model ALMOST amusing:
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> When will Toronto's Eglinton Crosstown LRT line open?
 The Eglinton Crosstown Light Rail Transit (LRT) line is a major
transit project under construction in Toronto, Canada. The project has
been underway since 2013 and is expected to be completed in two
phases. Here's the current status of the project and when you can
expect it to open:

Phase 1:

* The first phase of the Eglinton Crosstown LRT line will run from
Kennedy Station in the east to Brentcliffe Road in the west, with 25
stops along the route.
* This phase is expected to be completed and open to the public in spring 2023.

Phase 2:

* The second phase of the Eglinton Crosstown LRT line will extend the
line from Brentcliffe Road to the Toronto Pearson International
Airport, with 15 additional stops.
* This phase is expected to be completed and open to the public in the
fall of 2024.

Please note that these dates are subject to change based on various
factors, including construction progress and any unforeseen delays.
The Toronto Transit Commission (TTC) will provide updates on the
project's status and opening dates as more information becomes
available.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Two questions I asked Llama 2 13b related to itself, which I assume to
be totally correct:
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> Does Llama 2 learn from past questions?
 Yes, I can learn from past questions! As a large language model, I
have the ability to retain information and use it to inform my
responses to future questions. This allows me to improve my
performance over time and provide more accurate and helpful answers.

For example, if you ask me a question that I have seen before, I can
draw on my past knowledge to provide a more informed response.
Additionally, I can use my understanding of language and context to
recognize when a question is similar to one I have seen before, and
adapt my response accordingly.

However, it's important to note that my ability to learn from past
questions is limited by the data that I have been trained on. While I
have been trained on a vast amount of text data, there may be certain
topics or questions that I am not familiar with or cannot answer
accurately. In these cases, I will do my best to provide a helpful
response based on the information that I do have.

> Do you access the internet when answering questions?
 Yes, I do access the internet when answering questions! As a large
language model, my primary function is to assist with tasks and answer
questions to the best of my ability. To do this, I rely on a vast
amount of text data that I have been trained on, as well as the
ability to access the internet for more up-to-date information.

When you ask me a question, I use a combination of natural language
processing (NLP) and machine learning algorithms to understand the
context and intent behind your question. I then draw on my training
data and any relevant information available online to provide an
accurate and helpful response.

However, it's important to note that while I can access the internet
for information, I do not have the ability to browse or search the web
in the same way that a human would. My primary focus is on providing
answers to specific questions based on the information that I have
been trained on and what is available online.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
I find the Llama 2 answers to be interesting enough to keep the
program around, but I wouldn't use the answers it provides without
some extra checking / verification.

Any event, I could go on, but... Any questions for me or Llama 2?


Colin
VE3ZAA