Chapter 2 Related Works
2.3 Expert Systems
By definition, expert system is a system which employs human expertise to solve
problems like the real mankind does. An expert system either supports or automates
decision making in an area of which experts perform better than non-experts.
Tzeng et al. [4] introduced the concept of “knowledge” used in expert system.
There are four main topics: knowledge representation, knowledge acquisition,
knowledge inference, and knowledge validation/verification. By these topics, we can
choose the suitable method for each topic to form our calligraphy system.
In the first topic, many representative mechanisms are used for knowledge
representation. Backus–Naur form is the most common one used as a meta-language
for defining the syntax of a language. Frame-based representation is suitable for
related knowledge about a narrow subject with much default knowledge. Case-based
representation is usually used to describe the experimental knowledge, and to solve
problems by the experience of old cases which are similar to the current one. Here in
our calligraphy system, we use rule-based knowledge for the knowledge
representation. This representation is appropriate for a kind of knowledge field which
requires inference. The basic rule form is if-else condition. With complete rules, all
situations can be inferred.
The goal of knowledge acquisition is to elicit expertise from domain experts. We
reference the calligraphy copybook of Yan’s Kai style for increasing the domain
knowledge of Chinese calligraphy. Then, a repertory grid is needed for knowledge
acquisition. In this method, we elicit elements to be classified. Within the modify
stage, each grid is rated by filling a rating. At last, the repertory grid contains the state
of the attributes in it. By this way, all rules can be generated from the grid. We show
the final rule set in appendix to express the concept of repertory grid.
The third topic is the method of inference which has a main theory called
syllogism. It is a simple, well-understood branch of logic that can be completely
proven. In general, a syllogism can be any valid deductive argument having two
premises and a conclusion. Using the syllogism with two ways of reasoning – forward
and backward, the decision of any situation can be made from the input conditions or
from the final conclusions on the contrary, respectively.
In the last topic, knowledge should be verified and validated. Most expert system
has non-exclusive rules and fuzzy decisions. It leads to some unexpected situations.
To avoid such kind of problems, we set the rules used in Chinese calligraphy mutually
exclusive. Therefore, the same attribute state will not go to different conclusions.
Chapter 3
System Structure
In this chapter, we will introduce the structure of the whole calligraphy system.
We propose an approach which integrates expert system into calligraphy by using
different developmenting environments, including the inference mechanism (using the
JAVA environment) and the result display (using the C ++ environment) of the
calligraphy. The proposed system consists of 5 modules according to the
developmenting environment. Figure 3.1 shows the system architecture. We are going
to describe the details of each module in Section 3.1. Besides, since we combined
another independent system (the 3rd module in Figure 3.1), some other descriptions
will be given in Section 3.2.
Figure 3.1 The system architecture
3.1 System Overview
The proposed calligraphy system consists of 5 modules: stroke trajectories
setting and analysis, stroke relationships decision and attributes setting, stroke style
determination, returns of the best stroke style, and the strokes generation. Among
them, the first and the last one use C++ as the developmental environment, and
devoted to input and output. The others (the middle three modules) are responsible for
inference mechanism and are developed in JAVA environment.
In the first module, we input the stroke trajectories which can be done with a
digitizing pen or a mouse to start the calligraphy system. In this part, our approach is
to imitate the calligraphic copybook of Yan’s. With the graphic API – OpenGL, we set
the size of the copybook be 400 x 400 pixels and 256 gray level for each pixel. It is
put on the screen as the background and the user can write Chinese characters in it.
When a character has been written, the first step of this module is accomplished at the
beginning. We can only see the mouse trajectories of the character by removing the
copybook. After that, the second step of this module can be carried on: strokes
analysis. The system will analyze every stroke roughly in this stage, and determine
the type of each stroke.
In the second module, the relationships among all strokes are determined.
According to the mouse trajectories that are set in the previous module, we can
measure the starting and ending positions of each stroke, check whether the strokes of
a character are connected or joined to each other, and calculate more information of
each stroke. Recording all these data set, we can enter the inference mechanism then.
In the previous two modules described before, all information of each stroke has
been already collected. Now, we are going to determine the stroke style. There are two
sub-routines here. First, we have categorized all strokes and arranged them in much
more kinds in terms of the writing style of Yan and the characteristics of his
calligraphic character. This procedure is named stroke type classification. Then, we
design the inference rules according to the stroke information collected before and use
expert system as the inference engine. Via these two sub-steps, the style of each stroke
can be determined.
After the inference, the fourth module handles the return of it. Because the
system has crossed different developmenting environment – JAVA and C++, we
utilize transmission of file in this part, and make them (JAVA and C++) communicate
to each other correctly.
Finally, when each stroke is allocated with a specific stroke type, we can
API – OpenGL is used in this module. We use glDrawPixels to render the calligraphic
character with specific stroke model and stroke profiles. And this character will
appear in Yan’s style at last.
3.2 DRAMA
In the last chapter, we have discussed expert systems about the four main topics
on “knowledge” in detail. Here in this section, the focus is put on the developmenting
environment – DRAMA, a JAVA-based expert system.
DRAMA is a rule-based expert platform. As to the general engineers who
construct the knowledge base, this is an intact knowledge structure that is easier to use.
By adding rules that are similar to human logical thinking [8] [14], DRAMA can have
the ability of knowledge inference, and offer service for lots of different knowledge
field.
There is an important design concept in DRAMA – NORM. It stands for “new
object-oriented rule model” [7] [15]. By the concept of object-oriented knowledge
management, this kind of system can help knowledge constructors, the engineers, to
develop knowledge applications in a manner closer to human thinking. The purpose
of this model is to consider the psychological aspect of real mankind.
This consideration is often neglected by traditional expert system which just simply
use procedural knowledge as the system basis. This makes the description of
knowledge often lack the accuracy. However, the system with NORM model takes
some characteristics of human logical thinking into account, to design different
knowledge fields related to each other.
In NORM model, knowledge class is designed to perform knowledge block or
knowledge concept. We have some operations moving among different knowledge
classes:
1. When a new knowledge is constructed after learning some expertise, we
usually connect the current knowledge with others which we have
learned before, to reduce the difficulties of learning the current one. This
is called “reference”.
2. Knowledge “modification” is a kind of operation we are familiar with.
The learner can increase learning efficiency by some well-modified
knowledge. Besides, “extension” is also important on knowledge
construction. According to the demand for advanced knowledge, learners
use existing knowledge to increase extra knowledge content.
3. When the problem in a knowledge class can be simplified or turned into
4. According to different decision conditions, we need utilizing different
knowledge classes to infer. At this time, “acquire” is needed. In other
words, we should find out the proper knowledge classes to handle the
sub-problem.
By the operations described above, the NORM model is complete and makes
DRAMA more friendly to use.
Finally, we show the system structure of DRAMA in Figure 3.2. It includes
calligraphy program handling I/O and processing, server and inference engine of
DRAMA, and the knowledge base constructed with NORM model.
Figure 3.2 Calligraphy system
Chapter 4
Knowledge-Based Calligraphy System
Based on the system structure described in Chapter 3, in this chapter we are
going to do more detailed discussion to every module. All discussions are given from
the point of view of implementing the calligraphy system. The order of explanation
may (1) be with the order of system procedure or (2) be with the order of system
design (or implementation order). Although (1) is a more logical formulation, (2) is
preferred since it is more intuitive for users to know the design procedure. Besides,
we can know how modules are linked to each other more completely by this way.
4.1 Stroke Model
Stroke model is the basic component in Chinese calligraphy synthesis. In
calligraphy system, it lies in the fifth module. Cooperating with setting of stroke
profiles, the final character will be presented by stroke generation.
In [9], the proposed virtual brush has several features. Here, we integrate some of
those features into our system as follows:
1. The modeling way allows the synthesis of gray-level calligraphic images
with fine ink trails.
2. The brush pressure is ignored since the main purpose of this system is to
generate characters in Yan’s style by the composition of all strokes, not
focusing on the interactions of each stroke that users write.
3. Realistic calligraphic effects can be produced by varying different
parameters which control the profile of brush orientation along a stroke
trajectory.
4. The contact between the bristles and the paper is modeled using an
elliptic footprint whose principal axes can be dynamically adjusted
according to different turning control.
Then, the brush model is applied. In normal state, it is formed like a brush bundle
which can be approximated by an inverted cone while suitably inked, as shown in
Figure 4.2. It has the necessary information relating to the geometry of the brush
bundle. For example, the radius of the brush stem (R), the length of the brush bundle
(L), and the number of bristles (M) that form the brush bundle.
A typical example of a real brush has the following values: the brush stem radius
(R) = 0.9 cm., the length of brush bundle (L) = 3 cm., and the number of bristles (M)
= 10 000 approximately.
In a digital environment, these values above should be adjusted. The radius (R)
and the length (L) change based on the system size. In general case, each piece of the
brush bristle paints just one pixel or more. Here in our system, the bristles are ignored
because we do not focus on the simulation of stroke movement interactively. Thereby,
the radius and the length dominate the stroke generation.
Although the parameters (R and L) are fixed for a given brush, the radius of the
contact area between the virtual brush and the paper varies as the brush model rises
and falls on the paper. In other words, the larger the contact area between paper and
brush stem, the more pixels it needs to fill the canvas. This effect is shown in Figure
4.3. Except for the contact radius, another factor also determines the brush effect,
which is the orientation of the brush. It depends on the direction that the brush moves
along and appears in an ellipsoid shape. For example, when a vertical or horizontal
stroke is written, the ellipse should be a or a shape, respectively, and, for a
slanting stroke, it will be a or a shape, respectively. Combining the radius of
contact area and the ellipse orientation, we can achieve some special properties of the
Chinese calligraphy, especially on the start or the end of each stroke. In this way, all
kinds of stroke type can be generated. Figure 4.3 shows some examples.
Figure 4.3 The changes of the ellipsoid contact area in the size and the orientation.
In the stroke profile, we apply the radius and orientation parameters to specify the
stroke shape. The format consists of several data, including the percentage range on a
stroke, the deviation degree in this range, the ellipse radius and orientation specifying
the brush drops on this range. If the better result of stroke is preferred, the percentage
range should be narrowed. Therefore, we can describe the stroke detail more
explicitly in each range.
4.2 Stroke Type Classification
Before the progress entering the expert systems, first we should know which
kind of form the result would be presented after inference. In other words, the best
stroke type is chosen for the appearance of the stroke each time. This work is done by
the stroke type classification, the third module in the system structure as shown in
Figure 4.1.
In the classification of the Chinese characters stroke (that we referred to is the
regular script, Kai, which we are going to synthesize here), the most famous one is the
eight-laws of character ‘yong’ (永). It is the basic rule of writing in Chinese
calligraphy and is extremely important for beginners. People who want to write
The eight-laws of character ‘yong’ contains: (1) spot (點,側), (2) horizontal
stroke (橫,勒), (3) vertical stroke (豎,努), (4) rise suddenly (鈎,趯), (5) horizontally
rise stroke (仰橫,策), (6) long left-falling stroke (長撇,掠), (7) short left-falling stroke
(短撇,啄), and (8) right-falling stroke (捺,磔). Figure 4.4 shows it.
Figure 4.4 The eight-laws of character ‘yong’ (永字八法).
We apply the concept of the eight-laws of character ‘yong’ into the calligraphy
system. For the purpose of imitating the calligraphy written by Yan, we made some
revision on the eight-laws and rearrange the strokes of Chinese characters following
the taxonomy of [1]. By the classifying rules, the stroke categories are modified to (A)
horizontal stroke (橫), (B) vertical stroke (豎), (C) left-falling stroke (撇), (D)
right-falling stroke (捺), (E) spot (點), (F) rise (挑), (G) arc (厥), and (H) long-arc
(鈎). And these eight kinds of stroke all have eight kinds of writing style, respectively.
At the same time, in order to let the strokes more suitable for our system, some
of the writing styles of certain stroke are added, and some are deleted. We also
simplify all strokes to one direction since the system load can be lowered in this way
(with the concept of the eight-law of the character ‘yong’). The final version of the
stroke set is showed in Figure 4.5.
4.2.1 Naming
According to the stroke types, we can design each stroke style by setting the
parameters of stroke profile with the stroke model discussed in Section 4.1. As noted
previously, the purpose of the stroke type classification is offering choice as the
inference result in the system. So we should name the strokes properly. The naming
rules comprise three components, which are the main type classification, secondary
type classification, and the inference classification. We can see more clearly in
Section 4.3. For example, “A1a” stands for a stroke in which ‘A’ means the horizontal
stroke, the first type in stroke type classification; ‘1’ means the 1st kind of ‘A’ type;
and ‘a’ indicate the choice ‘a’ of ‘A1’ type after the inference of expert system. The
naming is also shown in Figure 4.5.
A1a A1b A1c A1d A1e
A1f A1g A1h A1i A1j A1k
B1a B1b B1c B1d B1e B1f B1g B1h B1i B1j B2a
C1a C1b C1c C1d C1e C1f C1g C2a C2b C3a C3b
D1a D1b D1c D1d D1e
D1f D1g D1h D1i D1j
E1a E1b E1c E2a E3a E4a G1a H1a H1b
F1a F2a F3a F4a F5a F5b
Figure 4.5 The name and classification of strokes for Yan’s style.
4.3 System Progress: Two Phases
When stroke model and classification of stroke types both are ready, we can
enter the main decision mechanism in the system. There are two phases in the
progress. The first phase is the classification by direction. In this part, system will
make the preliminary categorization. This includes the main type classification and
the secondary type classification that has already been discussed in Section 4.2.1.
Through this phase, stroke will get the first two symbols of its name, for example, A1
or E1. This phase is executed in the C++ environment and it belongs to the first
module in the system structure as shown in Figure 4.1.
Then in the second phase, we will choose more suitable style for every kind of
stroke. Via the inference of the expert system, stroke will get the last symbol of its
name. This stroke appears with the description of its profile finally. This phase is
executed in the JAVA environment and it belongs to both the second and the third
module in the system structure as shown in Figure 4.1.
4.3.1 Phase 1: Classify by Direction
When a stroke is written (it means the trajectory not been dealt with yet), it
contains a lot of information immediately, such as the position on which the virtual
pen touches the paper, the length of the stroke, and the direction of the stroke, etc.
Among them, the obvious one is the direction. It dominates which type the current
stroke belongs to. The direction can be obtained via the stroke slope and the relative
relation of the start and end points in a stroke.
Figure 4.6 shows the direction graph describing the length and the slope of
strokes. Because the paper size in our system is 400 x 400 pixels, each stroke ranges
from 0 pixels to 400√2 pixels. The center point in Figure 4.6 stands for the start point
of one stroke. With the direction of it, the stroke ends in one range of certain stroke
type. From now, we can classify strokes by this graph.
The classification can proceed from inside to outside. First we compare the
current stroke and the strokes with short length, is that, E1, E2, E3, and E4, which are
independent to each other and the length of them are about 40 to 60 pixels at most.
The ‘independent’ here means the range of them doesn’t overlap those of the others.
In more detail, the range of E1 is 0 to 60 pixels with slope from –3 to –1, and 0 to 80
to ∞ and from -∞ to –3. E3 and E4 can be described in the same way.
If this stroke does not match these types above, we compare it with the strokes
that have the longer length about 60 to 120 pixels at most, such as F1, F2, F3, F4, and
F5. If the current stroke still not matches those ones, it will be compared with the
longest ones about 400 or more pixels at most that are C1, C2, C3, G1, B1, B2, H1,
D1, and A1 finally.
Another important decision factor can be used to categorize the stroke, which is
the deviation of it. We can see that the ranges of C1, C2 and C3 overlap to each other
in Figure 4.6. The same situation also takes place in B1 and G1, H1 and B2, and D1
and B2. The stroke cannot be classified just with the slope between the starting and
ending points of it here. But with the stroke deviation, we can achieve this goal. For
example, the comparison of deviation among C1 and C2 and C3 is C1<C2<C3; the
comparison of deviation between B1 and G1 is B1<G1; the shape of stroke B2 is like
a right angle but that of stroke D1 and H1 are not (that the shape also influence the
deviation). Finally, the directional classification is done and each stroke has the first
two symbols of its name now.
Figure 4.6 Direction graph with length (in pixels) and slope
4.3.2 Phase 2: Classify by Relationship
After the directional classification, the first two symbols in the name of the
current stroke can be determined. Then, getting the third symbol would be taken as
our goal that can be done by the relationship between the current stroke and the other
strokes in a character. There are two components in this phase, including the
strokes in a character. There are two components in this phase, including the